id,node_id,number,title,user,user_label,state,locked,assignee,assignee_label,milestone,milestone_label,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,repo_label,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason 267542338,MDU6SXNzdWUyNjc1NDIzMzg=,13,Add a syntax highlighting SQL editor,9599,simonw,closed,0,,,,,1,2017-10-23T05:03:33Z,2017-11-15T02:04:51Z,2017-11-15T02:04:51Z,OWNER,,https://ace.c9.io/#nav=embedding looks like a good option,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 267707940,MDU6SXNzdWUyNjc3MDc5NDA=,14,Datasette Plugins,9599,simonw,closed,0,,,,,22,2017-10-23T15:15:28Z,2019-05-13T18:58:20Z,2019-05-13T18:58:19Z,OWNER,,"It would be neat if additional functionality could be opted-in to the system in the form of easy-to-add plugins, hosted as separate packages. First example: a Google Analytics plugin, which adds GA tracking code with your tracking ID to the web interface for your dataset. This may be an opportunity to experiment with entry points: http://amir.rachum.com/blog/2017/07/28/python-entry-points/",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 267739593,MDU6SXNzdWUyNjc3Mzk1OTM=,18,See if I can get a websockets interface working,9599,simonw,closed,0,,,,,1,2017-10-23T16:46:41Z,2021-01-04T20:05:52Z,2021-01-04T20:05:48Z,OWNER,,"Since I am already running on Sanic, how hard would it be to add a websocket ebdpoint that lets you talk to sqlite interactively? Could this be used to efficiently support streaming in answers to giant queries?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/18/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 267857622,MDU6SXNzdWUyNjc4NTc2MjI=,25,Endpoint that returns SQL ready to be piped into DB,9599,simonw,closed,0,,,,,2,2017-10-24T00:19:26Z,2017-11-15T05:11:12Z,2017-11-15T05:11:11Z,OWNER,,It would be cool if I could figure out a way to generate both the create table statements and the inserts for an individual table or the entire database and then stream them down to the client.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 267886330,MDU6SXNzdWUyNjc4ODYzMzA=,27,Ability to plot a simple graph,9599,simonw,closed,0,,,,,3,2017-10-24T03:34:59Z,2018-07-10T17:52:41Z,2018-07-10T17:52:41Z,OWNER,,"Might be as simple as: pick he type of chart (bar, line) and then pick the column for the X axis and the column for the Y axis. Maybe also allow a pie chart. It’s up to the user to come up with SQL that gets the right values.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 268078453,MDU6SXNzdWUyNjgwNzg0NTM=,30,Do something neat with foreign keys,9599,simonw,closed,0,,,,,1,2017-10-24T15:29:29Z,2017-11-14T18:29:08Z,2017-11-14T18:29:01Z,OWNER,,"https://www.sqlite.org/pragma.html#pragma_foreign_key_list SQLite has robust support for introspecting foreign keys. I could use that to automatically link to the corresponding record from my tables.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/30/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 268176505,MDU6SXNzdWUyNjgxNzY1MDU=,34,Support CSV export with a .csv extension,9599,simonw,closed,0,,,,,1,2017-10-24T20:34:43Z,2021-06-17T18:14:48Z,2018-05-28T20:45:34Z,OWNER,,"Maybe do this using streaming with multiple pagination SQL queries so we can support arbritrarily large exports. How would this work against a view which doesn’t have an obvious efficient pagination mechanism? Maybe limit views to up to 1000 exported records? Relates to #5 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/34/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 268262480,MDU6SXNzdWUyNjgyNjI0ODA=,36,"date, year, month and day querystring lookups",9599,simonw,closed,0,,,,,3,2017-10-25T04:23:45Z,2018-05-28T17:30:53Z,2018-05-28T17:30:53Z,OWNER,,"- [ ] `?timestamp___date=2017-07-17` - return every item where the timestamp falls on that date - [ ] `?timestamp___year=2017` - return every item where the timestamp falls within 2017 - [ ] `?timestamp___month=1` - return every item where the month component is January - [ ] `?timestamp___day=10` - return every item where the day-of-the-month component is 10 Follow on from #23 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/36/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 268453968,MDU6SXNzdWUyNjg0NTM5Njg=,37,Ability to serialize massive JSON without blocking event loop,9599,simonw,closed,0,,,,,2,2017-10-25T15:58:03Z,2020-05-30T17:29:20Z,2020-05-30T17:29:20Z,OWNER,,"We run the risk of someone attempting a select statement that returns thousands of rows and hence takes several seconds just to JSON encode the response, effectively blocking the event loop and pausing all other traffic. The Twisted community have a solution for this, can we adapt that in some way? http://as.ynchrono.us/2010/06/asynchronous-json_18.html?m=1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/37/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 268462768,MDU6SXNzdWUyNjg0NjI3Njg=,38,Experiment with patterns for concurrent long running queries,9599,simonw,closed,0,,,,,5,2017-10-25T16:23:42Z,2018-05-28T20:47:31Z,2018-05-28T20:47:31Z,OWNER,,I want to understand how the system could perform under load with many concurrent long-running queries. Can we serve these without blocking the event loop?,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/38/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 268591332,MDU6SXNzdWUyNjg1OTEzMzI=,42,Homepage UI for editing metadata file,9599,simonw,closed,0,,,,,4,2017-10-26T00:22:03Z,2017-12-10T03:02:14Z,2017-12-10T03:02:14Z,OWNER,,"Since we are going to have a metadata file which sets the title/description/etc for each database, why not allow you to run the app in —dev mode which makes the homepage into a WYSIWYG editor that can save to that file format.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/42/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 269731374,MDU6SXNzdWUyNjk3MzEzNzQ=,44,?_group_count=country - return counts by specific column(s),9599,simonw,closed,0,,,,,7,2017-10-30T19:50:32Z,2018-04-26T15:09:58Z,2018-04-26T15:09:58Z,OWNER,,"Imagine if this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283/airports.jsono?country__contains=gu&_group_count=country Turned into this: https://stateless-datasets-jykibytogk.now.sh/flights-07d1283?sql=select%20country,%20count(*)%20as%20group_count_country%20from%20airports%20where%20country%20like%20%27%gu%%27%20group%20by%20country%20order%20by%20group_count_country%20desc This would involve introducing a new precedent of query string arguments that start with an _ having special meanings. While we're at it, could try adding _fields=x,y,z Tasks: - [x] Get initial version working - [ ] Refactor code to not just ""pretend to be a view"" - [ ] Get foreign key relationships expanded",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/44/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 271301468,MDU6SXNzdWUyNzEzMDE0Njg=,46,Dockerfile should build more recent SQLite with FTS5 and spatialite support,9599,simonw,closed,0,,,,,13,2017-11-05T18:16:22Z,2017-11-17T14:32:12Z,2017-11-17T14:32:12Z,OWNER,,"The SQLite bundled with Python 3 doesn't support the FTS5 search extension. It would be nice if the SQLite built by our Dockerfile could support as many modern SQLite features as possible. https://web.archive.org/web/20170212034155/http://charlesleifer.com/blog/using-the-sqlite-json1-and-fts5-extensions-with-python/ has instructions on building a more recent SQLite and the pysqlite package. Our Dockerfile could carry out an updated version of this process.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/46/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 271831408,MDU6SXNzdWUyNzE4MzE0MDg=,47,Create neat example database,9599,simonw,closed,0,,,,,5,2017-11-07T13:29:38Z,2017-11-14T03:08:13Z,2017-11-14T03:08:13Z,OWNER,,How about data from open elections eg https://github.com/openelections/openelections-data-ca?files=1,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/47/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 272391665,MDU6SXNzdWUyNzIzOTE2NjU=,48,Switch to ujson,9599,simonw,closed,0,,,,,4,2017-11-08T23:50:29Z,2019-06-24T06:57:54Z,2019-06-24T06:57:43Z,OWNER,,"ujson is already a dependency of Sanic, and should be quite a bit faster.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/48/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273026602,MDU6SXNzdWUyNzMwMjY2MDI=,52,Solution for temporarily uploading DB so it can be built by docker,9599,simonw,closed,0,,,,,2,2017-11-10T18:55:25Z,2017-12-10T03:02:57Z,2017-12-10T03:02:57Z,OWNER,,For the `datasette publish` command I ideally need a way of uploading the specified DB to somewhere temporary on the internet so that when the Dockerfile is built by the final hosting location it can download that database as part of the build process.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/52/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273127443,MDU6SXNzdWUyNzMxMjc0NDM=,56,Easy way to block search engine crawling in robots.txt,9599,simonw,closed,0,,,,,1,2017-11-11T07:46:07Z,2018-05-28T20:50:25Z,2018-05-28T20:50:24Z,OWNER,,For people who don't want their datasets to be crawled by search engines.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/56/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273127694,MDU6SXNzdWUyNzMxMjc2OTQ=,57,Ship a Docker image of the whole thing,9599,simonw,closed,0,,,,,7,2017-11-11T07:51:28Z,2018-06-28T04:01:51Z,2018-06-28T04:01:38Z,OWNER,,"The generated Docker images can then just inherit from that. This will speed up deploys as no need to `pip install` anything. - [x] Ship that image to Docker Hub - [ ] Update the generated Dockerfile to use it",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/57/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273157085,MDU6SXNzdWUyNzMxNTcwODU=,59,datasette publish hyper,9599,simonw,closed,0,,,,,4,2017-11-11T16:27:26Z,2019-05-13T19:01:00Z,2019-05-13T19:00:44Z,OWNER,,"This is a bit tricky, because unlike Now there doesn't seem to be a way to tell Hyper to ""build this Dockerfile and deploy the resulting image"". They expect you to build a container and publish it to a registry instead. https://docs.hyper.sh/Reference/CLI/load.html allows you to publish an image directly from a tarball, but that still leaves the challenge of creating that image. The nice thing about the Now integration is that you don't need to have Docker installed on your local machine.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/59/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273181020,MDU6SXNzdWUyNzMxODEwMjA=,64,Support for ?field__isnull=1 or similar,9599,simonw,closed,0,,,,,1,2017-11-11T22:26:52Z,2017-11-17T14:38:21Z,2017-11-17T14:38:21Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/64/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273296178,MDU6SXNzdWUyNzMyOTYxNzg=,73,_nocache=1 query string option for use with sort-by-random,9599,simonw,closed,0,,,,,2,2017-11-13T02:57:10Z,2018-05-28T17:25:15Z,2018-05-28T17:25:15Z,OWNER,,The one place where we wouldn’t want cdching is if we have something which uses sort by random to return random items. We can offer a _nocache=1 querystring argument to support this.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/73/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273569068,MDU6SXNzdWUyNzM1NjkwNjg=,79,Add more detailed API documentation to the README,9599,simonw,closed,0,,,,,3,2017-11-13T20:36:21Z,2018-05-28T17:24:48Z,2018-05-28T17:24:48Z,OWNER,,"Need to document: - [ ] The ?column__gt=4 style filter arguments for tables - [ ] The ?sql= API, and how named parameters work - [ ] How API pagination works - [ ] How redirects and cache headers work",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/79/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273626815,MDU6SXNzdWUyNzM2MjY4MTU=,83,Individual row view is broken,9599,simonw,closed,0,,,,,0,2017-11-14T00:29:11Z,2017-11-14T00:45:34Z,2017-11-14T00:45:34Z,OWNER,,"https://parlgov.datasettes.com/parlgov-25f9855/viewcalc_parliament_composition/18 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/83/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273660425,MDU6SXNzdWUyNzM2NjA0MjU=,84,datasette package --metadata does not work with a relative path,9599,simonw,closed,0,,,,,0,2017-11-14T04:00:50Z,2017-11-15T05:18:35Z,2017-11-15T05:18:35Z,OWNER,," $ datasette package ~/parlgov-db/parlgov.db --metadata=~/parlgov-db/parlgov.json Usage: datasette package [OPTIONS] FILES... Error: Invalid value for ""-m"" / ""--metadata"": Could not open file: ~/parlgov-db/parlgov.json: No such file or directory simonw-07542:~ simonw$ cd ~/parlgov-db/ simonw-07542:parlgov-db simonw$ datasette package ~/parlgov-db/parlgov.db --metadata=parlgov.json Sending build context to Docker daemon 4.46MB Step 1/7 : FROM python:3 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/84/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273709194,MDU6SXNzdWUyNzM3MDkxOTQ=,87,Configure Travis to release new tags to PyPI,9599,simonw,closed,0,,,,,1,2017-11-14T08:44:08Z,2018-07-10T17:49:13Z,2018-07-10T17:49:12Z,OWNER,,https://docs.travis-ci.com/user/deployment/pypi/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/87/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273846123,MDU6SXNzdWUyNzM4NDYxMjM=,90,datasette publish heroku,9599,simonw,closed,0,,,,,8,2017-11-14T16:01:39Z,2017-12-10T03:06:34Z,2017-12-10T03:05:48Z,OWNER,,"Heroku has Docker container support so this should not be too hard: https://devcenter.heroku.com/articles/container-registry-and-runtime See also #59 This should work exactly like the existing “datasette publish now....” command except it would be “datasette publish heroku...”",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/90/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273878873,MDU6SXNzdWUyNzM4Nzg4NzM=,91,"Option to serve databases from a different prefix, serve regular content elsewhere",9599,simonw,closed,0,,,,,1,2017-11-14T17:32:46Z,2017-12-10T03:07:58Z,2017-12-10T03:07:53Z,OWNER,,"It would be useful if the databases themselves could be served from a prefix e.g. datasette serve mydb.db --path-prefix=db Now my database is at `http://localhost:8001/db/mydb-23423` This would free up the rest of the URL namespace for other things. Maybe we could have an option to serve static content from a known folder e.g. datasette serve mydb.db --path-prefix=db --root-content=~/my-project/static Now a hit to `http://localhost:8001/news/` serves content from `~/my-project/static/news/index.html` This would make it trivial to package up entire HTML/CSS/JS apps with one or more underlying SQLite databases. Running without `--cors` would be fine here because any JS apps would be hosted on the same origin.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/91/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273895344,MDU6SXNzdWUyNzM4OTUzNDQ=,92,Add --license --license_url --source --source_url --title arguments to datasette publish,9599,simonw,closed,0,,,,,0,2017-11-14T18:27:07Z,2017-11-15T05:04:41Z,2017-11-15T05:04:41Z,OWNER,,"I keep on using the `echo '{""source"": ""...""}' | datasette publish now --metadata=-` pattern, which suggests it makes sense for us to support these as optional arguments. https://gist.github.com/simonw/9f8bf23b37a42d7628c4dcc4bba10253",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/92/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 273998513,MDU6SXNzdWUyNzM5OTg1MTM=,95,Allow shorter time limits to be set using a ?_sql_time_limit_ms =20 query string limit,9599,simonw,closed,0,,,,,1,2017-11-15T01:02:16Z,2017-11-15T02:56:13Z,2017-11-15T02:56:13Z,OWNER,,This cannot be greater than the configured time limit.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/95/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274001453,MDU6SXNzdWUyNzQwMDE0NTM=,96,UI for editing named parameters,9599,simonw,closed,0,,,,,3,2017-11-15T01:19:21Z,2017-11-16T01:45:51Z,2017-11-16T01:33:38Z,OWNER,,"On any page displaying a custom query that includes named parameters, we should show HTML form fields for editing those parameters. Eg the breed parameter on https://australian-dogs.now.sh/australian-dogs-3ba9628?sql=select+name%2C+count%28*%29+as+n+from+%28%0D%0A%0D%0Aselect+upper%28%22Animal+name%22%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2013%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2014%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28Animal_Name%29+as+name+from+%5BAdelaide-City-Council-dog-registrations-2015%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22AnimalName%22%29+as+name+from+%5BCity-of-Port-Adelaide-Enfield-Dog_Registrations_2016%5D+where+AnimalBreed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5BMitcham-dog-registrations-2015%5D+where+Breed+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22DOG_NAME%22%29+as+name+from+%5Bburnside-dog-registrations-2015%5D+where+DOG_BREED+like+%3Abreed%0D%0A%0D%0Aunion+all+%0D%0A%0D%0Aselect+upper%28%22Animal_Name%22%29+as+name+from+%5Bcity-of-playford-2015-dog-registration%5D+where+Breed_Description+like+%3Abreed%0D%0A%0D%0Aunion+all%0D%0A%0D%0Aselect+upper%28%22Animal+Name%22%29+as+name+from+%5Bcity-of-prospect-dog-registration-details-2016%5D+where%22Breed+Description%22+like+%3Abreed%0D%0A%0D%0A%29+group+by+name+order+by+n+desc%3B&breed=pug",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/96/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274022950,MDU6SXNzdWUyNzQwMjI5NTA=,97,Link to JSON for the list of tables ,9599,simonw,closed,0,,,,,3,2017-11-15T03:29:05Z,2018-05-29T18:51:35Z,2018-05-28T20:57:21Z,OWNER,,https://twitter.com/yschimke/status/930606210855854080,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/97/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274023417,MDU6SXNzdWUyNzQwMjM0MTc=,98,Default to 127.0.0.1 not 0.0.0.0,9599,simonw,closed,0,,,,,0,2017-11-15T03:31:55Z,2017-11-15T05:08:54Z,2017-11-15T05:08:54Z,OWNER,,https://twitter.com/yschimke/status/930606210855854080,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/98/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274023625,MDU6SXNzdWUyNzQwMjM2MjU=,99,Start a change log,9599,simonw,closed,0,,,,,0,2017-11-15T03:33:21Z,2017-11-16T15:12:46Z,2017-11-16T15:12:45Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/99/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274264175,MDU6SXNzdWUyNzQyNjQxNzU=,102,datasette publish elasticbeanstalk,9599,simonw,closed,0,,,,,1,2017-11-15T18:48:31Z,2021-01-04T20:13:20Z,2021-01-04T20:13:19Z,OWNER,,"It looks like Elastic Beanstalk is the most convenient way to deploy a docker container to AWS without first deploying a cluster. https://aws.amazon.com/blogs/devops/dockerizing-a-python-web-app/ looks helpful. We would need to automate the deployment with Boto: http://boto3.readthedocs.io/en/latest/reference/services/elasticbeanstalk.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/102/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274265878,MDU6SXNzdWUyNzQyNjU4Nzg=,103,datasette publish appengine,9599,simonw,closed,0,,,,,1,2017-11-15T18:54:18Z,2021-01-04T20:05:14Z,2021-01-04T20:05:14Z,OWNER,,"Similar approach to Heroku, discussed in #90 Looks like this could be pretty easy: https://cloud.google.com/appengine/docs/flexible/python/quickstart",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/103/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274314940,MDU6SXNzdWUyNzQzMTQ5NDA=,105,Consider data-package as a format for metadata,9599,simonw,closed,0,,,,,4,2017-11-15T21:43:34Z,2017-11-20T19:50:53Z,2017-11-20T19:50:53Z,OWNER,,http://frictionlessdata.io/specs/data-package/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/105/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274315193,MDU6SXNzdWUyNzQzMTUxOTM=,106,Document how pagination works,9599,simonw,closed,0,,,,,1,2017-11-15T21:44:32Z,2019-06-24T06:42:33Z,2019-06-24T06:42:33Z,OWNER,,I made a start at that in this comment: https://news.ycombinator.com/item?id=15691926,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/106/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274374317,MDU6SXNzdWUyNzQzNzQzMTc=,108,"Include version in python code, output in template",9599,simonw,closed,0,,,,,0,2017-11-16T02:32:40Z,2017-11-16T15:30:04Z,2017-11-16T15:30:04Z,OWNER,,It would be useful if I could tell which version of datasette was running on a site. Embed version number and include it in maybe a tooltip on the “powered by datasette” link,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/108/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274378301,MDU6SXNzdWUyNzQzNzgzMDE=,109,Set up readthedocs,9599,simonw,closed,0,,,,,1,2017-11-16T02:58:01Z,2017-11-16T16:53:26Z,2017-11-16T16:13:56Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/109/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274578142,MDU6SXNzdWUyNzQ1NzgxNDI=,110,Add --load-extension option to datasette for loading extra SQLite extensions,9599,simonw,closed,0,,,,,2,2017-11-16T16:26:19Z,2017-11-16T18:38:30Z,2017-11-16T16:58:50Z,OWNER,,"This would allow users with extra SQLite extensions installed (like spatialite) to load them at runtime. Inspired by this comment: https://github.com/simonw/datasette/issues/46#issuecomment-344810525",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/110/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274617240,MDU6SXNzdWUyNzQ2MTcyNDA=,112,Allow --load-extension to be set via environment variables,9599,simonw,closed,0,,,,,1,2017-11-16T18:28:31Z,2017-11-17T14:19:23Z,2017-11-17T14:17:27Z,OWNER,,"This will make it easier to package up datasette in a Docker container with a bunch of pre-compiled extensions without the user having to remember to include all of the options every time. Click has a mechanism for this: http://click.pocoo.org/5/options/#multiple-values-from-environment-values",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/112/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 274884209,MDU6SXNzdWUyNzQ4ODQyMDk=,116,Add documentation section about SQLite extensions,9599,simonw,closed,0,,,,,1,2017-11-17T14:36:30Z,2018-05-28T17:23:42Z,2018-05-28T17:23:41Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/116/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275048699,MDExOlB1bGxSZXF1ZXN0MTUzNDMyMDQ1,118,Foreign key information on row and table pages,9599,simonw,closed,0,,,,,0,2017-11-18T03:13:27Z,2017-11-18T03:15:57Z,2017-11-18T03:15:50Z,OWNER,simonw/datasette/pulls/118,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/118/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 275082158,MDU6SXNzdWUyNzUwODIxNTg=,119,"Build an ""export this data to google sheets"" plugin",9599,simonw,closed,0,,,,,1,2017-11-18T14:14:51Z,2020-06-04T18:46:40Z,2020-06-04T18:46:39Z,OWNER,,"Inspired by https://github.com/kren1/tosheets It should be a plug-in because I'd like to keep all interactions with proprietary / non-open-source software encapsulated in plugins rather than shipped as part of core.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/119/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275087397,MDU6SXNzdWUyNzUwODczOTc=,120,Plugin that adds an authentication layer of some sort,9599,simonw,closed,0,,,,,4,2017-11-18T15:39:13Z,2020-03-16T18:48:06Z,2020-03-16T18:48:06Z,OWNER,,"Would allow people who want to host private data to do so. .sh ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/120/reactions"", ""total_count"": 7, ""+1"": 5, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 2, ""rocket"": 0, ""eyes"": 0}",,completed 275089535,MDU6SXNzdWUyNzUwODk1MzU=,121,?_json=foo&_json=bar query string argument ,9599,simonw,closed,0,,,,,4,2017-11-18T16:09:55Z,2018-05-31T13:48:12Z,2018-05-28T18:11:51Z,OWNER,,"Causes the specified columns in the output to be treated as JSON, and returned deserialized in the .json or .jsono response. This will be particularly powerful when combined with https://sqlite.org/json1.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/121/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275092453,MDU6SXNzdWUyNzUwOTI0NTM=,122,"Redesign JSON output, ditch jsono, offer variants controlled by parameter instead",9599,simonw,closed,0,,,,,5,2017-11-18T16:52:28Z,2018-04-08T14:54:09Z,2018-04-08T14:54:09Z,OWNER,,"I want to support three variants for the rows output: * a list of lists, with a columns key saying what they are * a list of dictionaries * a single dictionary where the keys are the primary keys of the rows and the values are the row dictionaries themselves I also want to make the various bits of metadata opt-in - so you don't get the SQL statement unless you ask for it. These output options should be controlled by query string arguments. I will set the .jsono URL to redirect to .json with the corresponding options. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/122/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275125805,MDU6SXNzdWUyNzUxMjU4MDU=,124,Option to open readonly but not immutable,9599,simonw,closed,0,,,,,5,2017-11-19T02:11:03Z,2019-06-24T06:43:46Z,2019-06-24T06:43:46Z,OWNER,,Immutable assumes no other process can modify the file. An option to open reqdonly instead would enable other processes to update the file in place.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/124/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275135393,MDU6SXNzdWUyNzUxMzUzOTM=,125,Plot rows on a map with Leaflet and Leaflet.markercluster,9599,simonw,closed,0,,,,,2,2017-11-19T06:05:05Z,2018-04-26T15:14:31Z,2018-04-26T15:14:31Z,OWNER,,"https://github.com/Leaflet/Leaflet.markercluster would allow us to paginate-load in an enormous set of rows with latitude/longitude points, e.g. https://australian-dunnies.now.sh/ Here's a demo of it loading 50,000 markers: https://leaflet.github.io/Leaflet.markercluster/example/marker-clustering-realworld.50000.html - and it looks like it's easy to support progress bars for if we were iteratively loading 1,000 markers at a time using datasette pagination.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/125/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275164558,MDU6SXNzdWUyNzUxNjQ1NTg=,129,Hide FTS-created tables by default on the database index page,9599,simonw,closed,0,,,,,2,2017-11-19T14:50:42Z,2017-11-22T20:22:02Z,2017-11-22T20:19:04Z,OWNER,,"SQLite databases that use FTS include a number of automatically generated tables, e.g.: https://sf-trees-search.now.sh/sf-trees-search-a899b92 Of these, only the `Street_Tree_List` table is actually relevant to the user. We can detect which tables are FTS tables by first finding the virtual tables: sqlite> .headers on sqlite> select * from sqlite_master where rootpage = 0; type|name|tbl_name|rootpage|sql table|Search|Search|0|CREATE VIRTUAL TABLE ""Street_Tree_List_fts"" USING FTS4 (""qAddress"", ""qCaretaker"", ""qSpecies"") Then parsing the above to figure out which ones are USING FTS? - then assume that any table which starts with that `Street_Tree_List_fts` prefix was created to support search: sqlite> select * from sqlite_master where type='table' and tbl_name like 'Street_Tree_List_fts%'; type|name|tbl_name|rootpage|sql table|Search_content|Search_content|10355|CREATE TABLE 'Street_Tree_List_fts_content'(docid INTEGER PRIMARY KEY, 'c0qAddress', 'c1qCaretaker', 'c2qSpecies') table|Search_segments|Search_segments|10356|CREATE TABLE 'Street_Tree_List_fts_segments'(blockid INTEGER PRIMARY KEY, block BLOB) table|Search_segdir|Search_segdir|10357|CREATE TABLE 'Street_Tree_List_fts_segdir'(level INTEGER,idx INTEGER,start_block INTEGER,leaves_end_block INTEGER,end_block INTEGER,root BLOB,PRIMARY KEY(level, idx)) table|Search_docsize|Search_docsize|10359|CREATE TABLE 'Street_Tree_List_fts_docsize'(docid INTEGER PRIMARY KEY, size BLOB) table|Search_stat|Search_stat|10360|CREATE TABLE 'Street_Tree_List_fts_stat'(id INTEGER PRIMARY KEY, value BLOB) We won't hide these completely - instead, we'll default the database index view to not showing them with a message that says ""5 hidden tables"" and support ?_hidden=1 to display them.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/129/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275166078,MDU6SXNzdWUyNzUxNjYwNzg=,130,"Rename ""datasette build"" to ""datasette inspect""",9599,simonw,closed,0,,,,,0,2017-11-19T15:08:02Z,2017-12-07T16:57:58Z,2017-12-07T16:57:58Z,OWNER,,"This command introspects the databases and writes out a JSON summary. I think I'd like to use `datasette build` for something more interesting, potentially duplicating functionality from https://github.com/simonw/csvs-to-sqlite Since the internal method that does this is called `ds.inspect()` that seems like a reasonable replacement name for the command.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/130/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275166669,MDU6SXNzdWUyNzUxNjY2Njk=,131,UI support for running FTS searches,9599,simonw,closed,0,,,,,3,2017-11-19T15:16:20Z,2017-11-19T17:18:05Z,2017-11-19T17:00:12Z,OWNER,,"Here's an example query that searches all FTS indexed columns in a table: https://sf-trees-search.now.sh/sf-trees-search-a899b92?sql=select+*+from+Street_Tree_List+where+rowid+in+%28select+rowid+from+Street_Tree_List_fts+where+Street_Tree_List_fts+match+%27grove+london+dpw%27%29%0D%0A And here's a query that searches a specific column: https://sf-trees-search.now.sh/sf-trees-search-a899b92?sql=select+*+from+Street_Tree_List+where+rowid+in+%28select+rowid+from+Street_Tree_List_fts+where+qSpecies+match+%27london%27%29%0D%0A If we detect that a table has FTS enabled (which we can do by looking for it as a content table reference in another FTS table's create definition) we should add a search box to the table page which constructs this query - maybe using `?_search=XXX` in the query string? To support search against specified columns, we can do `?_search__ qSpecies=London`. - not necessary, see comment below. - [x] Detect if a table has a FTS index defined against it as a content= parameter - [x] Decide what to do if there is more than one FTS index (maybe just pick the first one?) - [x] Add the `?_search=` query string argument - [x] Add the UI",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/131/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275228834,MDU6SXNzdWUyNzUyMjg4MzQ=,136,"""Reformat SQL"" button next to SQL editor textarea",9599,simonw,closed,0,,,,,0,2017-11-20T03:42:19Z,2019-10-14T03:46:13Z,2019-10-14T03:46:13Z,OWNER,,"Can use this: https://github.com/zeroturnaround/sql-formatter https://zeroturnaround.github.io/sql-formatter/ ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/136/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275476839,MDU6SXNzdWUyNzU0NzY4Mzk=,138,"Per-database and per-table metadata, probably using data-package",9599,simonw,closed,0,,,,,1,2017-11-20T19:50:10Z,2017-12-10T03:08:36Z,2017-12-10T03:08:26Z,OWNER,,"Ability to annotate databases and tables with extra metadata describing their purpose, providing source and licensing information and describing individual columns. http://frictionlessdata.io/specs/data-package/ looks like a great format for this, see #105 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/138/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275493851,MDU6SXNzdWUyNzU0OTM4NTE=,139,Build a visualization plugin for Vega,9599,simonw,closed,0,,,,,2,2017-11-20T20:47:41Z,2018-07-10T17:48:18Z,2018-07-10T17:48:18Z,OWNER,,"https://vega.github.io/vega/examples/population-pyramid/ for example looks pretty easy to hook up to Datasette. Depends on #14 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/139/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275917760,MDU6SXNzdWUyNzU5MTc3NjA=,142,Show extra instructions with the interrupted,9599,simonw,closed,0,,,,,3,2017-11-22T01:44:29Z,2018-05-28T21:25:06Z,2018-05-28T21:24:35Z,OWNER,,"When you are using Datasette locally for ad-hoc analysis it can be frustrating to hit the time limit. If you start it with the correct command line arguments you can disable that time limit. So how about we tell you how to do that anytime you hit the interrupted error provided you are accessing it from localhost.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/142/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 275939188,MDU6SXNzdWUyNzU5MzkxODg=,143,"Mechanism for ""suggested visualizations""",9599,simonw,closed,0,,,,,1,2017-11-22T04:10:25Z,2018-07-10T17:48:34Z,2018-07-10T17:48:34Z,OWNER,," Each visualization should have a way of deciding if it might be appropriate for the current view of data. We can then offer a ""suggested visualizations"" prompt which shows previews.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/143/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 276192732,MDExOlB1bGxSZXF1ZXN0MTU0MjQ2ODE2,145,Fix pytest version conflict,9599,simonw,closed,0,,,,,0,2017-11-22T20:15:34Z,2017-11-22T20:17:54Z,2017-11-22T20:17:52Z,OWNER,simonw/datasette/pulls/145,"https://travis-ci.org/simonw/datasette/jobs/305929426 pkg_resources.VersionConflict: (pytest 3.2.1 (/home/travis/virtualenv/python3.5.3/lib/python3.5/site-packages), Requirement.parse('pytest==3.2.3'))",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/145/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 276455748,MDU6SXNzdWUyNzY0NTU3NDg=,146,datasette publish gcloud,9599,simonw,closed,0,,,,,2,2017-11-23T18:55:03Z,2019-06-24T06:48:20Z,2019-06-24T06:48:20Z,OWNER,,"See also #103 It looks like you can start a Google Cloud VM with a ""docker container"" option - and the Google Cloud Registry is easy to push containers to. So it would be feasible to have `datasette publish gcloud ...` automatically build a container, push it to GCR, then start a new VM instance with it: https://cloud.google.com/container-registry/docs/pushing-and-pulling ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/146/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 276704327,MDU6SXNzdWUyNzY3MDQzMjc=,150,_group_count= feature improvements,9599,simonw,closed,0,,,,,3,2017-11-24T22:06:18Z,2018-05-28T16:41:28Z,2018-05-28T16:41:28Z,OWNER,,"- [ ] The ""apply filters"" form should keep you on the _group_count= page - [ ] Foreign key references should be expand - [ ] Page title should reflect the view you are on",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/150/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 276718605,MDU6SXNzdWUyNzY3MTg2MDU=,151,Set up a pattern portfolio,9599,simonw,closed,0,,,,,2,2017-11-25T02:09:49Z,2020-07-02T00:13:24Z,2020-05-03T03:13:16Z,OWNER,,"https://www.slideshare.net/nataliedowne/practical-maintainable-css/75 This will be a single page that demonstrates all of the different CSS styles and classes available to Datasette.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/151/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 279547886,MDU6SXNzdWUyNzk1NDc4ODY=,163,Document the querystring argument for setting a different time limit,9599,simonw,closed,0,,,,,2,2017-12-05T22:05:08Z,2021-03-23T02:44:33Z,2017-12-06T15:06:57Z,OWNER,,"http://datasette.readthedocs.io/en/latest/sql_queries.html#query-limits Need to explain why this is useful too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/163/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 280662866,MDExOlB1bGxSZXF1ZXN0MTU3MzY1ODEx,168,Upgrade to Sanic 0.7.0,9599,simonw,closed,0,,,,,1,2017-12-09T01:25:08Z,2017-12-09T03:00:34Z,2017-12-09T03:00:34Z,OWNER,simonw/datasette/pulls/168,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/168/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 280896290,MDU6SXNzdWUyODA4OTYyOTA=,172,Show size of .db file next to download link,9599,simonw,closed,0,,,,,1,2017-12-11T05:12:46Z,2019-02-06T05:09:06Z,2019-02-06T05:00:36Z,OWNER,,"Size in bytes should be calculated by datasette inspect. Template should display it in KB or MB or GB",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/172/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 281197863,MDU6SXNzdWUyODExOTc4NjM=,174,License/Source in footer should inherit from top level,9599,simonw,closed,0,,,,,1,2017-12-11T23:01:35Z,2018-08-11T17:46:51Z,2018-08-11T17:46:51Z,OWNER,,"The footer on https://vice-police-shootings.now.sh/vice-bc7c892/ViceNews_FullOISData does not show license and source information... but that Datasette has that information, it's just defined at the top level: https://vice-police-shootings.now.sh/ The footer for a row/table page should fall back on information for the database, and if there is none for the database it should fall back on the top-level metadata instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/174/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 309471814,MDU6SXNzdWUzMDk0NzE4MTQ=,189,Ability to sort (and paginate) by column,9599,simonw,closed,0,9599,simonw,,,31,2018-03-28T18:04:51Z,2018-04-15T18:54:22Z,2018-04-09T05:16:02Z,OWNER,,"As requested in https://github.com/simonw/datasette/issues/185#issuecomment-376614973 I've previously avoided this for performance reasons: sort-by-column on a column without an index is likely to perform badly for hundreds of thousands of rows. That's not a good enough reason to avoid the feature entirely though. A few options: * Allow sort-by-column by default, give users the option to disable it for specific tables/columns * Disallow sort-by-column by default, give users option (probably in `metadata.json`) to enable it for specific tables/columns * Automatically detect if a column either has an index on it OR a table has less than X rows in it We already have the mechanism in place to cut off SQL queries that take more than X seconds, so if someone DOES try to sort by a column that's too expensive it won't actually hurt anything - but it would be nice to not show people a ""sort"" option which is guaranteed to throw a timeout error. The vast majority of datasette usage that I've seen so far is on smaller datasets where the performance penalties of sort-by-column are extremely unlikely to show up. ---- Still left to do: - [x] UI that shows which sort order is currently being applied (in HTML and in JSON) - [x] UI for applying a sort order (with rel=nofollow to avoid Google crawling it) - [x] Sort column names should be escaped correctly in generated SQL - [x] Validation that the selected sort order is a valid column - [x] Throw error if user attempts to apply _sort AND _sort_desc at the same time - [x] Ability to disable sorting (or sort only for specific columns) in metadata.json - [x] Fix ""201 rows where sorted by sortable_with_nulls "" bug ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/189/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 309558826,MDU6SXNzdWUzMDk1NTg4MjY=,190,Keyset pagination doesn't work correctly for compound primary keys,9599,simonw,closed,0,,,,,7,2018-03-28T22:45:06Z,2018-03-30T06:31:15Z,2018-03-30T06:26:28Z,OWNER,,"Consider https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key ![2018-03-28 at 3 47 pm](https://user-images.githubusercontent.com/9599/38060388-56da86dc-329f-11e8-9f20-5576153ad55c.png) The next= link is to `d,v`: https://datasette-issue-190-compound-pks.now.sh/compound-pks-9aafe8f/compound_primary_key?_next=d%2Cv But that page starts with: ![2018-03-28 at 3 48 pm](https://user-images.githubusercontent.com/9599/38060402-6b0f5984-329f-11e8-85b8-44a666c4ee71.png) The next key in the sequence should be `d,w`. Also we should return the full a-z of the ones that start with the letter e - in this example we only return `e-w`, `e-x`, `e-y` and `e-z`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/190/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 310533258,MDU6SXNzdWUzMTA1MzMyNTg=,191,Figure out how to bundle a more up-to-date SQLite,9599,simonw,closed,0,,,,,6,2018-04-02T16:33:25Z,2018-07-10T17:46:13Z,2018-07-10T17:46:13Z,OWNER,,"The version of SQLite that ships with Python 3 is a bit limited - it doesn't support row values for example https://www.sqlite.org/rowvalue.html Figure out how to bundle a more recent SQLite engine with datasette. We need to figure out two cases: * Bundling a recent version in a Dockerfile build. I expect this to be quite easy. * Making a more recent version available to people hacking around in Mac OS X. I have no idea how to start on this. I want it working on Mac OS X too because I don't want to force Docker as a dependency for anyone who just want to hack around with Datasette a little and run the test suite.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/191/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 310850458,MDExOlB1bGxSZXF1ZXN0MTc5MTA4OTYx,192,New ?_shape=objects/object/lists param for JSON API,9599,simonw,closed,0,,,,,0,2018-04-03T14:02:58Z,2018-04-03T14:53:00Z,2018-04-03T14:52:55Z,OWNER,simonw/datasette/pulls/192,Refs #122,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/192/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 310882100,MDU6SXNzdWUzMTA4ODIxMDA=,193,Cleaner mechanism for handling custom errors,9599,simonw,closed,0,,,,,3,2018-04-03T15:19:13Z,2018-04-13T18:18:59Z,2018-04-13T18:18:59Z,OWNER,,"This code is pretty messy: https://github.com/simonw/datasette/blob/0abd3abacb309a2bd5913a7a2df4e9256585b1bb/datasette/app.py#L245-L265 Instead, it would be nice if I could raise an exception that would be converted into the appropriate JSON or HTML error message, with a corresponding HTTP code.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/193/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 312312125,MDU6SXNzdWUzMTIzMTIxMjU=,194,Rename table_rows and filtered_table_rows to have _count suffix,9599,simonw,closed,0,,,,,2,2018-04-08T14:53:37Z,2018-04-09T05:25:22Z,2018-04-09T05:25:22Z,OWNER,,"These fields represent counts of items: ""table_rows"": 131, ""filtered_table_rows"": 8, But the names make it sound like they might be arrays full of rows. Adding a `_count` suffix would make this more clear: ""table_rows_count"": 131, ""filtered_table_rows_count"": 8, ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/194/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 312313496,MDU6SXNzdWUzMTIzMTM0OTY=,195,"Run pks_for_table in inspect, executing once at build time rather than constantly",9599,simonw,closed,0,,,,,3,2018-04-08T15:12:40Z,2018-04-10T00:54:43Z,2018-04-10T00:54:43Z,OWNER,,"Right now several Datasette views call the `await self.pks_for_table(...)` method to figure out what primary keys are set for a specific table. This executes a `PRAGMA table_info` SQL query. It would be faster and more efficient to execute this query for each table as part of the `inspect()` method.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/195/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 312355154,MDExOlB1bGxSZXF1ZXN0MTgwMTg4Mzk3,196,_sort= and _sort_desc= parameters to table view,9599,simonw,closed,0,,,,,0,2018-04-09T00:07:21Z,2018-04-09T05:10:29Z,2018-04-09T05:10:23Z,OWNER,simonw/datasette/pulls/196,See #189 ,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/196/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 312620566,MDU6SXNzdWUzMTI2MjA1NjY=,199,Ability to apply sort on mobile in portrait mode,9599,simonw,closed,0,,,,,4,2018-04-09T17:35:04Z,2018-04-10T00:37:53Z,2018-04-10T00:34:38Z,OWNER,,"Missed this in #189... on mobile in portrait mode we hide the column headers, which means you can't click them to sort! You can sort in landscape mode at least. Need to come up with an alternative sort UI for portrait on mobile.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/199/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 313512748,MDU6SXNzdWUzMTM1MTI3NDg=,201,Support explain select / explain query plan select,9599,simonw,closed,0,,,,,1,2018-04-11T22:41:26Z,2018-04-13T21:17:14Z,2018-04-12T21:32:52Z,OWNER,,See https://www.sqlite.org/eqp.html and https://www.sqlite.org/lang_explain.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/201/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 314469126,MDExOlB1bGxSZXF1ZXN0MTgxNzMxOTU2,210,"Start of the plugin system, based on pluggy",9599,simonw,closed,0,,,,,0,2018-04-16T00:51:30Z,2018-04-16T00:56:16Z,2018-04-16T00:56:16Z,OWNER,simonw/datasette/pulls/210,Refs #14,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/210/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 314471743,MDU6SXNzdWUzMTQ0NzE3NDM=,211,Load plugins from a `--plugins-dir=plugins/` directory,9599,simonw,closed,0,,,,,6,2018-04-16T01:17:43Z,2018-04-16T05:22:02Z,2018-04-16T05:22:02Z,OWNER,,"In #14 and 33c7c53ff87c2 I've added working support for setuptools entry_points plugins. These can be installed from PyPI using `pip install ...`. I imagine some projects will benefit from being able to add plugins without first publishing them to PyPI. Datasette already supports [loading custom templates](http://datasette.readthedocs.io/en/latest/custom_templates.html#custom-templates) like so: datasette serve --template-dir=mytemplates/ mydb.db I propose an additional option, `--plugins-dir=` which specifies a directory full of `blah.py` files which will be loaded into Datasette when the application server starts. datasette serve --plugins-dir=myplugins/ mydb.db This will also need to be supported by `datasette publish` as those Python files should be copied up as part of the deployment.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/211/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 314504812,MDExOlB1bGxSZXF1ZXN0MTgxNzU1MjIw,212,New --plugins-dir=plugins/ option,9599,simonw,closed,0,,,,,0,2018-04-16T05:19:28Z,2018-04-16T05:22:18Z,2018-04-16T05:22:01Z,OWNER,simonw/datasette/pulls/212,Refs #211,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/212/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 314506033,MDU6SXNzdWUzMTQ1MDYwMzM=,213,Documentation for plugins system,9599,simonw,closed,0,,,,,0,2018-04-16T05:27:07Z,2018-04-16T15:12:48Z,2018-04-16T15:12:48Z,OWNER,,"Documentation for #14 - how to write plugins, how to ship plugins to PyPI and how to use the `--plugins-dir` option added in #211 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/213/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 314506446,MDU6SXNzdWUzMTQ1MDY0NDY=,214,Ability for plugins to define extra JavaScript and CSS,9599,simonw,closed,0,,,,,6,2018-04-16T05:29:34Z,2020-09-30T20:36:11Z,2018-04-18T03:13:03Z,OWNER,,"This can hook in to the existing `extra_css_urls` and `extra_js_urls` mechanism: https://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L304-L305 The plugins should be able to bundle their own assets though, so it will also have to integrate with the `/static/` static mounts mechanism somehow: https://github.com/simonw/datasette/blob/b2955d9065ea019500c7d072bcd9d49d1967f051/datasette/app.py#L1255-L1257 Refs #14",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/214/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 314725342,MDU6SXNzdWUzMTQ3MjUzNDI=,217,Plugin support for datasette publish,9599,simonw,closed,0,,,,,1,2018-04-16T16:17:14Z,2018-07-26T05:33:39Z,2018-07-26T05:16:00Z,OWNER,,"It should be possible to support additional deployment options by writing a plugin (see #59). As part of this, rewrite the Heroku and Now publishers to be implemented as plugins (they will still ship with datasette by default). Maybe `datasette package` should be changed to being part of publish instead, `datasette publish docker` perhaps? Refs #14",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/217/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 314847571,MDU6SXNzdWUzMTQ4NDc1NzE=,220,Investigate syntactic sugar for plugins,9599,simonw,closed,0,,,,,2,2018-04-16T23:01:39Z,2020-06-11T21:50:06Z,2020-06-11T21:49:55Z,OWNER,,"Suggested by @andrewhayward on Twitter: https://twitter.com/arhayward/status/986015118965268480?s=21 > Have you considered a basic abstraction on top of that, for standard hook features? ``` @sql_function random_integer(a,b): return random.randint(a,b) @template_filter uppercase(str): return str.upper() ``` Maybe `from datasette.plugins import template_filter`? Would have to work out how to get this to play well with pluggy",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/220/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315142414,MDU6SXNzdWUzMTUxNDI0MTQ=,221,Allow plugins to add new cli sub commands ,9599,simonw,closed,0,,,,,3,2018-04-17T16:40:13Z,2021-01-04T20:12:14Z,2021-01-04T20:12:14Z,OWNER,,I could then test this out by having https://github.com/simonw/csvs-to-sqlite register itself as a plugin,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/221/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315316214,MDExOlB1bGxSZXF1ZXN0MTgyMzU3NjEz,222,Fix for plugins in Python 3.5,9599,simonw,closed,0,,,,,0,2018-04-18T03:21:01Z,2018-04-18T04:26:50Z,2018-04-18T03:24:21Z,OWNER,simonw/datasette/pulls/222,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/222/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 315327860,MDU6SXNzdWUzMTUzMjc4NjA=,223,datasette publish --install=name-of-plugin,9599,simonw,closed,0,,,,,3,2018-04-18T04:33:59Z,2018-04-18T14:56:17Z,2018-04-18T14:56:17Z,OWNER,,Mechanism for causing datasette publish and datasette package to install one or more additional plugins using `pip install` - refs #14 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/223/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315517578,MDU6SXNzdWUzMTU1MTc1Nzg=,224,Ability for plugins to bundle templates,9599,simonw,closed,0,,,,,1,2018-04-18T14:57:53Z,2018-04-19T05:50:36Z,2018-04-19T05:50:36Z,OWNER,,"Plugins should be able to bundle templates. The Datasette template loader should then consult those plugins first when loading a template. Jinja2 has a `PackageLoader` class that can help with this: http://jinja.pocoo.org/docs/2.10/api/#jinja2.PackageLoader Refs #14",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/224/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315548495,MDU6SXNzdWUzMTU1NDg0OTU=,225,/-/(inspect|metadata|plugins)(.json)? introspection,9599,simonw,closed,0,,,,,0,2018-04-18T16:14:58Z,2018-04-19T05:25:33Z,2018-04-19T05:25:33Z,OWNER,,"3 pages (and accompanying .json endpoints) for viewing: * the metadata.json that datasette was loaded with * the output of ds.inspect() * a list of installed plugins, detected by pluggy",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/225/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315738696,MDU6SXNzdWUzMTU3Mzg2OTY=,226,Unit tests for installable plugins,9599,simonw,closed,0,,,,,2,2018-04-19T06:05:32Z,2020-11-24T19:52:51Z,2020-11-24T19:52:46Z,OWNER,,"I'd like more thorough unit test coverage of the plugins mechanism - in particular for installable plugins. I think I can do this while still having the code live in the same repo, by creating a subdirectory in tests/example_plugin with its own setup.py and then running `python setup.py install` as part of the test runner. I imagine I will need to bump the version number every time I change the plugin in case someone runs the test again in the same virtual environment. If that doesn't work I can instead ship a datasette-plugins-tests two to PyPI and add that as a tests_require dependency. Refs #14",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/226/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 315960272,MDU6SXNzdWUzMTU5NjAyNzI=,227,prepare_context() plugin hook,9599,simonw,closed,0,,,,,8,2018-04-19T16:55:26Z,2020-03-24T22:19:54Z,2020-03-24T22:19:54Z,OWNER,,This would be called with the context dictionary before each template is rendered. It would have the opportunity to modify that context.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/227/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 316031566,MDU6SXNzdWUzMTYwMzE1NjY=,228,"If spatialite detected, mark idx_XXX_Geometry tables as hidden",9599,simonw,closed,0,,,,,1,2018-04-19T20:37:24Z,2018-04-26T03:25:39Z,2018-04-26T03:25:39Z,OWNER,,"https://timezones-api.now.sh/timezones-faf26d0 ![2018-04-19 at 1 36 pm](https://user-images.githubusercontent.com/9599/39016906-a5acbb3e-43d6-11e8-9a31-814ff1d0022e.png) Need to update this logic: https://github.com/simonw/datasette/blob/e2750c7cc0585adaa8c866be611089e62961ee35/datasette/app.py#L1276-L1288",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/228/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 316123256,MDU6SXNzdWUzMTYxMjMyNTY=,229,Table view should support ?_size=400 parameter,9599,simonw,closed,0,,,,,1,2018-04-20T04:23:18Z,2018-04-26T04:49:46Z,2018-04-26T04:48:32Z,OWNER,,Allows callers to request more rows at once. The limit will still be `max_returned_rows` (defaults to 1000).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/229/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 316128955,MDU6SXNzdWUzMTYxMjg5NTU=,230,Setting page size AND max returned rows to 1000 doesn't seem to work,9599,simonw,closed,0,,,,,1,2018-04-20T05:05:11Z,2018-04-26T04:04:25Z,2018-04-26T04:04:25Z,OWNER,,"It appears that if the two settings are the same Datasette fails to return any results, probably because of the trick where we try to fetch 1001 rows so we know if there's a next page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/230/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 316323336,MDU6SXNzdWUzMTYzMjMzMzY=,231,metadata.json support for plugin configuration options,9599,simonw,closed,0,,,,,4,2018-04-20T15:58:47Z,2019-05-13T18:56:21Z,2019-05-13T18:56:21Z,OWNER,,"My [datasette-cluster-map](https://github.com/simonw/datasette-cluster-map) plugin currently works by detecting `latitude` and `longitude` columns. I'd like to be able to configure it to look for different column names. One way to do this could be to support optional plugin configuration as part of `metadata.json`. Something like this: { ""title"": ""Polar Bear Ear Tags, 2009-2011"", ""source"": ""USGS Alaska Science Center, Polar Bear Research Program"", ""source_url"": ""https://alaska.usgs.gov/products/data.php?dataid=130"", ""plugins"": { ""datasette_cluster_map"": { ""latitude_columns"": [ ""latitude"", ""Capture Latitude"" ], ""longitude_columns"": [ ""longitude"", ""Capture Longitude"" ] } } } These settings should be supported at the root level or at the individual database or table level. They could also be exposed in the https://datasette-cluster-map-demo.now.sh/-/plugins debug tool. Refs #14",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 316444720,MDU6SXNzdWUzMTY0NDQ3MjA=,233,Option to expose expanded foreign keys in JSON/CSV,9599,simonw,closed,0,,,,,11,2018-04-21T00:18:25Z,2018-06-16T22:26:21Z,2018-06-16T22:20:14Z,OWNER,,"https://datasette-cluster-map-demo.datasettes.com/sf-trees-02c8ef1/Street_Tree_List?qCareAssistant=1 ![f36b87c0-478e-4d55-9a5f-ad37df0b47cb](https://user-images.githubusercontent.com/9599/39078411-bb3e4f88-44be-11e8-9d0c-d22324793c77.png) It would be nice if the info bubbles there could expose more than just the IDs, and if the title showed the expanded name of the selected qCareAssistant.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/233/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 316526433,MDU6SXNzdWUzMTY1MjY0MzM=,234,label_column option in metadata.json,9599,simonw,closed,0,,,,,3,2018-04-21T21:19:08Z,2018-04-22T20:47:12Z,2018-04-22T20:47:12Z,OWNER,,"Currently the column used for displaying a foreign key relationship is automatically detected by `inspect()` by looking for tables that have a primary key column and one other column. This doesn't work for tables with more than two columns. Let's allow the table section in `metadata.json` to optionally define a `label_column` which, if present, will be used for those displays.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/234/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 317475156,MDU6SXNzdWUzMTc0NzUxNTY=,237,Support for ?_search_colname=blah searches,9599,simonw,closed,0,,,,,2,2018-04-25T04:29:53Z,2018-05-05T22:56:42Z,2018-05-05T22:33:23Z,OWNER,,"Right now the `_search=` argument searches across all fields in a full-text index, for example: https://san-francisco.datasettes.com/sf-film-locations-84594a7/Film_Locations_in_San_Francisco?_search=justin SQLite FTS also supports searches within a specified field, for example: https://san-francisco.datasettes.com/sf-film-locations-84594a7?sql=select+rowid%2C+*+from+Film_Locations_in_San_Francisco+where+rowid+in+%28select+rowid+from+%5BFilm_Locations_in_San_Francisco_fts%5D+where+%5BLocations%5D+match+%3Asearch%29+order+by+rowid+limit+101&search=justin ``` select rowid, * from Film_Locations_in_San_Francisco where rowid in ( select rowid from [Film_Locations_in_San_Francisco_fts] where [Locations] match :search ) order by rowid limit 101 ``` The `_search=` parameter could be extended to support this using `_search_colname=`. This should also be able to support columns with spaces and special characters in their names, something like this: `_search_Column%20With%20Spaces=foo` ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/237/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 317714268,MDU6SXNzdWUzMTc3MTQyNjg=,238,External metadata.json,9599,simonw,closed,0,,,,,3,2018-04-25T17:02:30Z,2019-06-24T06:52:55Z,2019-06-24T06:52:45Z,OWNER,,"A frustration I'm having with https://register-of-members-interests.datasettes.com/ is that I keep coming up with new canned queries but I don't want to redeploy the whole thing just to add them to `metadata.json` Maybe Datasette could optionally take a `--metadata-url` option which causes it to load from a URL instead and occasionally check for updates.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/238/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 317760361,MDU6SXNzdWUzMTc3NjAzNjE=,239,Support for hidden tables in metadata.json,9599,simonw,closed,0,,,,,2,2018-04-25T19:21:17Z,2018-04-26T03:45:12Z,2018-04-26T03:43:10Z,OWNER,,"Since we already have a hidden feature, let's expose it more to our users ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/239/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 317900587,MDU6SXNzdWUzMTc5MDA1ODc=,240,FTS table detection should be part of .inspect(),9599,simonw,closed,0,,,,,0,2018-04-26T06:58:10Z,2018-04-29T00:04:44Z,2018-04-29T00:04:44Z,OWNER,,"The code that detects if specific tables have a corresponding FTS column is currently called from TableView - it should instead be handled as part of `.inspect()`. This will make it easier to build other features that need to behave differently depending on whether a table can be searched, e.g. an autocomplete widget for selecting filters from foreign key tables. Current code: https://github.com/simonw/datasette/blob/f188ceaa2a3a5b2eab83425ad0f00cb0d364e24a/datasette/app.py#L728-L733",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/240/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 318692953,MDU6SXNzdWUzMTg2OTI5NTM=,242,Rename ?_sql_time_limit_ms= to ?_timelimit=,9599,simonw,closed,0,,,,,0,2018-04-29T06:11:35Z,2018-05-02T00:20:42Z,2018-05-02T00:20:42Z,OWNER,,It's a bit of a mouthful at the moment.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/242/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 318737808,MDU6SXNzdWUzMTg3Mzc4MDg=,243,--spatialite option for datasette publish commands,9599,simonw,closed,0,,,,,2,2018-04-29T18:19:32Z,2018-05-31T14:17:53Z,2018-05-31T14:17:53Z,OWNER,,Performs the necessary incantations to install Spatialite on Zeit Now or Heroku and sets the corresponding environment variable to ensure the module is correctly loaded by datasette serve.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/243/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 318738000,MDU6SXNzdWUzMTg3MzgwMDA=,244,/-/versions page,9599,simonw,closed,0,,,,,1,2018-04-29T18:22:15Z,2018-05-03T14:13:49Z,2018-05-03T14:09:53Z,OWNER,,"Displays the current version of: * datasette * Python * SQLite * Spatialite (if available) Installed plugin versions should be shown on /-/plugins",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/244/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 319358200,MDU6SXNzdWUzMTkzNTgyMDA=,245,?_shape=array option,9599,simonw,closed,0,,,,,1,2018-05-01T23:11:07Z,2018-05-03T14:14:33Z,2018-05-02T00:12:20Z,OWNER,,"Some tools (`pandas.DataFrame(...)` for example) are happiest when you give them a raw array of JSON objects. `?_shape=array` should do just that While I'm at it, rename the default `?_shape=lists` to instead be called `?shape=arrays` And validate that `_shape` is a valid option And have `?_shape=object` return the object at the root level rather than nested in `.rows` to better match the behavior of `?_shape=array`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/245/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 319371036,MDExOlB1bGxSZXF1ZXN0MTg1MzA3NDA3,246,?_shape=array and _timelimit=,9599,simonw,closed,0,,,,,0,2018-05-02T00:18:54Z,2018-05-02T00:20:41Z,2018-05-02T00:20:40Z,OWNER,simonw/datasette/pulls/246,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/246/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 319954545,MDU6SXNzdWUzMTk5NTQ1NDU=,248,/-/plugins should show version of each installed plugin,9599,simonw,closed,0,,,,,2,2018-05-03T14:50:45Z,2018-05-04T18:25:40Z,2018-05-04T18:05:04Z,OWNER,,"Refs #244 https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules ``` >>> import pkg_resources >>> pkg_resources.get_distribution('datasette_cluster_map').version '0.4' ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/248/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 320090329,MDU6SXNzdWUzMjAwOTAzMjk=,249,?_size=max argument ,9599,simonw,closed,0,,,,,1,2018-05-03T21:42:04Z,2018-05-04T18:26:30Z,2018-05-04T18:05:04Z,OWNER,,"For plugins that want to load the most data allowable, having `?_size=max` would be useful.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 320592643,MDU6SXNzdWUzMjA1OTI2NDM=,251,"Explore ""distinct values for column"" in inspect()",9599,simonw,closed,0,,,,,4,2018-05-06T13:27:24Z,2018-05-14T22:47:55Z,2018-05-14T22:47:55Z,OWNER,,"A lot of datasets have columns which have a small number of possible values in them - this one for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-2628db9?sql=select+distinct+category+from+%5Binconvenient-sequel%2Fratings%5D%3B Detecting these could be interesting as part of `.inspect()`, since it would allow for various UI enhancements like autocomplete / select box filters for those columns. The problem is detecting them efficiently. `.inspect()` shouldn't spend 5 minutes churning through columns on giant tables trying to determine if they have a small collection of unique values.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/251/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 321624016,MDU6SXNzdWUzMjE2MjQwMTY=,252,/-/versions should report the FTS version supported by SQLite,9599,simonw,closed,0,,,,,0,2018-05-09T15:43:47Z,2018-05-11T13:19:52Z,2018-05-11T13:19:52Z,OWNER,,I can copy this function from `csvs-to-sqlite`: https://github.com/simonw/csvs-to-sqlite/blob/dccbf65b37bc9eed50e9edb80a42f257e93edb1f/csvs_to_sqlite/utils.py#L283-L293,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/252/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 321631020,MDU6SXNzdWUzMjE2MzEwMjA=,253,Documentation explaining how to use SQLite FTS with Datasette,9599,simonw,closed,0,,,,,1,2018-05-09T16:02:08Z,2018-05-12T12:09:02Z,2018-05-12T12:06:51Z,OWNER,,"In particular how to work with https://www.sqlite.org/fts3.html#_external_content_fts4_tables_ - which Datasette can automatically detect and use to add a search UI to your page. Examples of basic search setup like this: ``` CREATE VIRTUAL TABLE ""interests_fts"" USING FTS4 (name, content=""interests""); INSERT INTO ""interests_fts"" (rowid, name) SELECT rowid, name FROM interests; ``` And complex join-based search setup like this: ``` CREATE VIRTUAL TABLE ""interests_fts"" USING FTS4 (name, category, member, content=""interests""); INSERT INTO ""interests_fts"" (rowid, name, category, member) SELECT interests.rowid, interests.name, interest_categories.name, members.name FROM interests JOIN interest_categories ON interests.category_id = interest_categories.id JOIN members ON interests.member_id = members.id; ``` Also mention how `csvs-to-sqlite` can be used to do this easily. This will benefit from #252 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/253/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 322477187,MDU6SXNzdWUzMjI0NzcxODc=,255,Facets,9599,simonw,closed,0,,,,,16,2018-05-12T03:00:07Z,2019-05-29T21:39:12Z,2018-05-16T15:32:12Z,OWNER,,"Ability to display facets and facet counts on the table view. Facets can be specified in the URL with `?_facet=column&_facet=othercolumn` or the default facets for a table can be set using a new `""facets"": [...]` property in `metadata.json` - [x] Implement `?_facet=` - [x] Implement `metadata.json` `facets` key - [x] Design for how facets should be presented - [x] Facets should be able to toggle off as well as on - [x] Expand labels for facets that are foreign keys - [x] Suggest potential facets (if we can do so within a tight time limit) - [x] Documentation",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/255/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 322551723,MDU6SXNzdWUzMjI1NTE3MjM=,256,Break up app.py into separate view modules,9599,simonw,closed,0,,,,,1,2018-05-12T23:56:33Z,2018-05-14T03:05:37Z,2018-05-14T03:05:37Z,OWNER,,"`views/table.py` and `views/database.py` and `views/utils.py` as a starting point. Likewise, create `tests/test_views_table.py` and `tests/test_views_database.py` - these will contain both HTML and API test for those views.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/256/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 322591993,MDExOlB1bGxSZXF1ZXN0MTg3NjY4ODkw,257,Refactor views,9599,simonw,closed,0,,,,,5,2018-05-13T13:00:50Z,2018-05-14T03:04:25Z,2018-05-14T03:04:24Z,OWNER,simonw/datasette/pulls/257,"* Split out view classes from main `app.py` * Run [black](https://github.com/ambv/black) against resulting code to apply opinionated source code formatting * Run [isort](https://github.com/timothycrosley/isort) to re-order my imports Refs #256 ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/257/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 322787470,MDU6SXNzdWUzMjI3ODc0NzA=,259,inspect() should detect many-to-many relationships,9599,simonw,closed,0,,,,,6,2018-05-14T12:03:58Z,2019-05-23T03:55:37Z,2019-05-23T03:55:37Z,OWNER,,"Relates to #255 - in particular supporting facets across M2M relationships. It should be possible for `.inspect()` to notice when a table has two foreign keys to two different tables, and assume that this means there is a M2M relationship between those tables. When rendering a table with a m2m relationship we could display the first X associated records as a comma separated list of hyperlinks in a new column on the table view, with a column name derived from the table on the other side. Since SQLite doesn't have RANK or an equivalent of https://www.xaprb.com/blog/2006/12/02/how-to-number-rows-in-mysql/ this would be implemented as N+1 queries (one query per cell that we want to display an m2m summary). This should be OK in SQLite: https://sqlite.org/np1queryprob.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/259/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 323459939,MDExOlB1bGxSZXF1ZXN0MTg4MzEyNDEx,261,Facets improvements plus suggested facets,9599,simonw,closed,0,,,,,0,2018-05-16T03:52:39Z,2018-05-16T15:27:26Z,2018-05-16T15:27:25Z,OWNER,simonw/datasette/pulls/261,Refs #255,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/261/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 323671577,MDU6SXNzdWUzMjM2NzE1Nzc=,263,Facets should not execute for ?shape=array|object,9599,simonw,closed,0,,,,,3,2018-05-16T15:26:13Z,2021-06-02T02:54:34Z,2021-06-02T02:54:34Z,OWNER,,Split off from #255 - there's no point executing the facet SQL for the `?_shape=array` and `?_shape=object` API responses.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/263/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 323673899,MDU6SXNzdWUzMjM2NzM4OTk=,264,Make it possible to customize various facet settings,9599,simonw,closed,0,,,,,1,2018-05-16T15:31:34Z,2018-05-18T06:18:00Z,2018-05-18T05:11:52Z,OWNER,,"The new Facets implementation from #255 includes several hard-coded settings which should be made configurable somehow: Number of rows to return in a facet (maybe this should also be an option that can be set via quersytring argument, e.g. `?_facet=qSpecies:40`): https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L539 Time limit for executing a facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L559-L562 Maximum unique values returned in order for a column to be suggested as a facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L646-L647 Time limit for calculating if a column should be a suggested facet: https://github.com/simonw/datasette/blob/9959a9e4deec8e3e178f919e8b494214d5faa7fd/datasette/views/table.py#L664-L667 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/264/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 323677499,MDU6SXNzdWUzMjM2Nzc0OTk=,265,Add links to example Datasette instances to appropiate places in docs,9599,simonw,closed,0,,,,,5,2018-05-16T15:40:20Z,2018-06-18T15:52:15Z,2018-06-18T15:52:15Z,OWNER,,"Links to working examples would really help, especially on these pages: * http://datasette.readthedocs.io/en/latest/json_api.html * http://datasette.readthedocs.io/en/latest/sql_queries.html * http://datasette.readthedocs.io/en/latest/facets.html * http://datasette.readthedocs.io/en/latest/full_text_search.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/265/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 323681589,MDU6SXNzdWUzMjM2ODE1ODk=,266,Export to CSV,9599,simonw,closed,0,,,,,27,2018-05-16T15:50:24Z,2021-06-17T18:14:24Z,2018-06-18T06:05:25Z,OWNER,,Datasette needs to be able to export data to CSV.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 323716411,MDU6SXNzdWUzMjM3MTY0MTE=,267,"Documentation for URL hashing, redirects and cache policy",9599,simonw,closed,0,,,,,3,2018-05-16T17:29:01Z,2019-06-24T06:41:02Z,2019-06-24T06:41:02Z,OWNER,,See my comments on #258 for a starting point,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/267/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 323726888,MDU6SXNzdWUzMjM3MjY4ODg=,269,"If a facet fails due to timing out, let the user know somehow",9599,simonw,closed,0,,,,,0,2018-05-16T18:01:47Z,2018-05-18T06:11:46Z,2018-05-18T06:11:46Z,OWNER,,Refs #255 - right now facets fail silently if the user requested them but they take longer than 200ms to calculate - see also #264,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/269/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 323830051,MDU6SXNzdWUzMjM4MzAwNTE=,270,--limit= CLI option for setting limits,9599,simonw,closed,0,,,,,1,2018-05-17T00:14:24Z,2018-05-18T06:19:31Z,2018-05-18T06:16:39Z,OWNER,,"#264 calls for four new datasette limit options, on top of the two existing ones: * `--max_returned_rows` * `--sql_time_limit_ms` These are already clogging up `datasette serve --help` a bit. How about this syntax instead? datasette --limit max_returned_rows:100 \ --limit facet_timeout_ms:500 demo.db Then we can add as many new user over-rideable limits as we like without clogging up `--help` too much - though it would be good to have a way of optionally listings their documentation as well.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/270/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 324162476,MDU6SXNzdWUzMjQxNjI0NzY=,271,Mechanism for automatically picking up changes when on-disk .db file changes,9599,simonw,closed,0,,,,,4,2018-05-17T19:53:15Z,2019-01-10T21:35:18Z,2019-01-10T21:35:18Z,OWNER,,"It would be useful if Datasette could spot when a SQLite database file changes on disk and restart itself (hence re-running .inspect() and picking up the new content hash). Ideally this could happen in an atomic way so no requests get dropped during the switch-over. This may not play well with SQLite opening databases in immutable mode. Research required.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/271/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 324451322,MDU6SXNzdWUzMjQ0NTEzMjI=,273,Figure out a way to have /-/version return current git commit hash,9599,simonw,closed,0,,,,,2,2018-05-18T15:16:56Z,2018-05-22T19:35:22Z,2018-05-22T19:35:22Z,OWNER,,"https://fivethirtyeight.datasettes.com/-/versions reports Datasette version `0.21` This isn't actually correct. The deploy script for that site actually deploys current master using `https://github.com/simonw/datasette/archive/master.zip`: https://github.com/simonw/fivethirtyeight-datasette/blob/66b4b0dfedd7237bc8c02d3e26d905bca7b84069/Dockerfile#L9 Ideally this would show the current commit hash, but I'm not at all sure if it's possible to derive that from `pip install https://github.com/simonw/datasette/archive/master.zip`. Is there another mechanism that could be used to reliably `pip install` current master but still provide access to the most recent commit hash?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/273/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 324652142,MDU6SXNzdWUzMjQ2NTIxNDI=,274,"Rename --limit to --config, add --help-config",9599,simonw,closed,0,,,,,2,2018-05-19T18:57:42Z,2018-05-20T17:04:55Z,2018-05-20T17:04:11Z,OWNER,,"#270 introduced `--limit` but on further thought it should be called `--config` instead. `--page_size` should becomes `--config default_page_size:1000` Add `--help-config` to show full help showing all config settings.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/274/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 324720095,MDU6SXNzdWUzMjQ3MjAwOTU=,275,"""config"" section in metadata.json (root, database and table level)",9599,simonw,closed,0,,,,,3,2018-05-20T16:02:28Z,2023-08-23T01:28:37Z,2023-08-23T01:28:37Z,OWNER,,"Split off from #274 Metadata should an optional `""config""` section at root, table or database level. The TableView and RowView and DatabaseView and BaseView classes could all have a `.config(""key"")` method which knows how to resolve the hierarchy of configs. This will allow individual tables (or databases) to set their own config settings for things like `sql_time_limit_ms`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/275/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 325294102,MDU6SXNzdWUzMjUyOTQxMDI=,278,Build smallest possible Docker image with Datasette plus recent SQLite (with json1) plus Spatialite 4.4.0,9599,simonw,closed,0,,,,,3,2018-05-22T13:28:40Z,2018-05-23T17:43:36Z,2018-05-23T17:43:36Z,OWNER,,"A Dockerfile that does the following: * Bundles Datasette master * Python 3.6 most recent version (or 3.7 if it has been released) * SQLite 3.23.1 (or most recent release) such that ""import sqite3"" in Python gets that version. Ideally with the json1 module baked in by default, but having it loadable as an optional module is fine too * SpatiaLite 4.4.0-RC0 (or most recent version) such that it can be loaded as an optional module * Uses multi-stage builds to stay as small as possible Note that the current ""release"" of SpatiaLite is 4.3.0 which is missing key features like https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN - 4.4.0 probably needs to be compiled from source. I don't know the best way to get a current SQLite version bundled for Python 3. Maybe https://github.com/coleifer/pysqlite3 ?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/278/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 325705981,MDU6SXNzdWUzMjU3MDU5ODE=,282,Faceting breaks pagination,9599,simonw,closed,0,,,,,1,2018-05-23T13:29:47Z,2018-05-23T13:53:39Z,2018-05-23T13:42:07Z,OWNER,,"e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id#facet-lg_id - click the ""next page"" link: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/nba-elo%2Fnbaallelo?_facet=lg_id&_next=100 Invalid SQL: near ""and"": syntax error",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/282/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 325958506,MDU6SXNzdWUzMjU5NTg1MDY=,283,Support cross-database joins,9599,simonw,closed,0,,,,,26,2018-05-24T04:18:39Z,2021-06-06T09:40:18Z,2021-02-18T22:16:46Z,OWNER,,"SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases. Since Datasette supports more than one database, this would make a pretty neat feature.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/283/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326182814,MDU6SXNzdWUzMjYxODI4MTQ=,284,Ability to enable/disable specific features via --config,9599,simonw,closed,0,,,,,5,2018-05-24T15:47:56Z,2018-05-25T06:05:02Z,2018-05-25T05:51:09Z,OWNER,,"`--config` settings from #274 can currently only be integers. I'd like them to be available as boooeans too. Then we can use them to have that are turned on by default but can be turned off. First features to get this treatment: - [x] `allow_sql` - whether or not the `?sql=` parameter is allowed and form is displayed - [X] `allow_facet` - is `?_facet=` allowed or do we only run facets defined in `metadata.json` - [X] `allow_download` - do we let users download the full SQLite database file? - [X] `suggest_facets` - do we attempt to calculate suggested facets? Refs #275 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/284/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326189744,MDU6SXNzdWUzMjYxODk3NDQ=,285,num_threads and cache_max_age should be --config options,9599,simonw,closed,0,,,,,2,2018-05-24T16:04:51Z,2018-05-27T00:53:35Z,2018-05-27T00:43:33Z,OWNER,,"https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/app.py#L106 And https://github.com/simonw/datasette/blob/58b5a37dbbf13868a46bcbb284509434e66eca25/datasette/views/base.py#L325 Refs #275 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/285/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326617744,MDU6SXNzdWUzMjY2MTc3NDQ=,287,?_shape=arrayfirst,9599,simonw,closed,0,,,,,1,2018-05-25T18:11:03Z,2018-05-27T00:32:53Z,2018-05-27T00:32:29Z,OWNER,,Return an array of single items (the first item in each row returned from the SQL query).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326767626,MDU6SXNzdWUzMjY3Njc2MjY=,288,Support multiple filters of the same type,9599,simonw,closed,0,,,,,3,2018-05-26T21:13:12Z,2019-04-15T23:45:04Z,2019-04-15T23:44:26Z,OWNER,,"This should work for example: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/biopics%2Fbiopics?year_release__not=2014&year_release__not=2015",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/288/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326768188,MDU6SXNzdWUzMjY3NjgxODg=,289,?_ttl= parameter to control caching,9599,simonw,closed,0,,,,,3,2018-05-26T21:22:55Z,2018-05-26T22:22:47Z,2018-05-26T22:17:48Z,OWNER,,"This would allow clients to specify the max-age caching header that should be returned with the query. Most important this will allow caching to be completely urned off for specific queries using `?_ttl=0`. Sending 0 should cause a `Cache-Control: no-cache` header to be returned.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/289/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326783670,MDU6SXNzdWUzMjY3ODM2NzA=,291,Avoid plugins accidentally loading dependencies twice,9599,simonw,closed,0,,,,,3,2018-05-27T03:15:21Z,2020-09-30T20:36:12Z,2018-05-28T20:42:02Z,OWNER,,Plugins that include JavaScript files risk loading the same code twice. In particular: I want to build a second plugin that uses the Leaflet mapping library (the first was [datasette-cluster-map](https://pypi.org/project/datasette-cluster-map/)). But I don't want the two plugins to load duplicate copies of Leaflet.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/291/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 326800219,MDU6SXNzdWUzMjY4MDAyMTk=,292,Mechanism for customizing the SQL used to select specific columns in the table view,9599,simonw,closed,0,,,,,15,2018-05-27T09:05:52Z,2021-05-27T04:25:01Z,2021-05-27T04:25:01Z,OWNER,,"Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow. We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice. I think there are two features here: * the ability to request a subset of columns on the table view * the ability to override the SQL for a specific column and/or add extra columns - `AsGeoJSON(Geometry)` for example Both features should be available via both querystring arguments and in `metadata.json` The querystring argument for custom SQL should only work if `allow_sql` config is turned on. Refs #276",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/292/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 327383759,MDU6SXNzdWUzMjczODM3NTk=,295,Extract unit tests for inspect out to test_inspect.py,9599,simonw,closed,0,,,,,2,2018-05-29T15:55:04Z,2019-05-11T21:40:32Z,2019-05-11T21:40:32Z,OWNER,,"Right now they are bundled up as API unit tests for a relatively unimportant endpoint. They should be their own thing. Blocks #294",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/295/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 327420945,MDU6SXNzdWUzMjc0MjA5NDU=,297,datasette publish Dockerfile should use python:3.6-slim-stretch,9599,simonw,closed,0,,,,,1,2018-05-29T17:40:08Z,2018-05-31T14:44:37Z,2018-05-31T14:44:37Z,OWNER,,"Right now the Dockerfile generated by `datasette package` and `datasette publish` uses this: https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/utils.py#L269 This appears to result in a SQLite version of `3.8.7.1` - https://parlgov.datasettes.com/-/versions ``` ""sqlite"": { ""extensions"": {}, ""fts_versions"": [ ""FTS4"", ""FTS3"" ], ""version"": ""3.8.7.1"" } ``` Meanwhile, https://fivethirtyeight.datasettes.com/-/versions is deployed with this Dockerfile https://github.com/simonw/fivethirtyeight-datasette/blob/0849901cae06e957fe04892cd4033bdcd1fcf966/Dockerfile which uses `FROM python:3.6-slim-stretch` and results in the following version report: ``` ""sqlite"": { ""extensions"": { ""json1"": null }, ""fts_versions"": [ ""FTS5"", ""FTS4"", ""FTS3"" ], ""version"": ""3.16.2"" } ``` So not only do we get a more recent SQLite (including https://www.sqlite.org/rowvalue.html added in 3.15) but we also get `FTS5` and `json1` as well. Refs #191 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/297/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 327459829,MDU6SXNzdWUzMjc0NTk4Mjk=,298,URLify URLs in results from custom SQL statements / views,9599,simonw,closed,0,,,,,2,2018-05-29T19:41:07Z,2018-07-24T04:53:20Z,2018-07-24T03:56:50Z,OWNER,,"Consider this custom query: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3?sql=select+user%2C+%28%27https%3A%2F%2Ftwitter.com%2F%27+%7C%7C+user%29+as+user_url%2C+created_at%2C+text%2C+url+from+%5Btwitter-ratio%2Fsenators%5D+limit+10%3B ```select user, ('https://twitter.com/' || user) as user_url, created_at, text, url from [twitter-ratio/senators] limit 10;``` ![2018-05-29 at 12 38 pm](https://user-images.githubusercontent.com/9599/40681177-44a36d5c-633d-11e8-935b-c49dad7ac682.png) It would be nice if these URLs were turned into links, as happens on the table view page: https://fivethirtyeight.datasettes.com/fivethirtyeight-5de27e3/twitter-ratio%2Fsenators ![2018-05-29 at 12 39 pm](https://user-images.githubusercontent.com/9599/40681206-5c69c47c-633d-11e8-9f3a-08899f8659b8.png) This currently does not happen because the table view render logic takes a different path through `display_columns_and_rows()` which includes this bit: https://github.com/simonw/datasette/blob/b0a95da96386ddf99816911e08df86178ffa9a89/datasette/views/table.py#L195-L202",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/298/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 327461381,MDU6SXNzdWUzMjc0NjEzODE=,299,Documentation covering ALL datasette URLs,9599,simonw,closed,0,,,,,1,2018-05-29T19:46:15Z,2018-07-28T04:24:05Z,2018-07-28T04:22:30Z,OWNER,,"Relates to #296. We need a single page of the docs listing all of the URL patterns Datasette responds to, also detailing which templates are used to render them and linking to examples of the JSON they output when called with `.json`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/299/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 327541975,MDU6SXNzdWUzMjc1NDE5NzU=,300,Hide sort select box on larger screens,9599,simonw,closed,0,,,,,0,2018-05-30T01:34:59Z,2018-05-31T14:43:13Z,2018-05-31T14:43:13Z,OWNER,,"I'm larger screens you can sort by clicking column headers, so no need to show the select box (which was added for the small screen layout that doesn't show headers)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/300/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 328172521,MDU6SXNzdWUzMjgxNzI1MjE=,303,Support table names ending with .json or .csv,9599,simonw,closed,0,,,,,4,2018-05-31T14:53:23Z,2018-06-15T06:55:50Z,2018-06-15T06:55:50Z,OWNER,,"This is needed for #266 - if a table name ends with `.json` or `.csv` right now our URL pattern matching will do the wrong thing. We should be smarter about this. This does mean we will have some URLs that look like this: http://localhost:8001/dbname/weird.json - returning HTML, not JSON http://localhost:8001/dbname/weird.json.json - returning JSON http://localhost:8001/dbname/weird.json.csv - returning CSV ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/303/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 328229224,MDU6SXNzdWUzMjgyMjkyMjQ=,304,Ability to configure SQLite cache_size,9599,simonw,closed,0,,,,,3,2018-05-31T17:28:07Z,2018-06-04T16:13:32Z,2018-06-04T16:03:19Z,OWNER,,"See https://www.sqlite.org/pragma.html#pragma_cache_size Let's call the config setting `cache_size_kb` to emphasize that we're using the negative option. Note this warning: perhaps we should raise an error if you try to use this setting against a SQLite version prior to 3.7.10 > If the argument N is positive then the suggested cache size is set to N. If the argument N is negative, then the number of cache pages is adjusted to use approximately abs(N*1024) bytes of memory. Backwards compatibility note: The behavior of cache_size with a negative N was different in prior to version 3.7.10 (2012-01-16). In version 3.7.9 and earlier, the number of pages in the cache was set to the absolute value of N.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/304/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 329147284,MDU6SXNzdWUzMjkxNDcyODQ=,305,Add contributor guidelines to docs,9599,simonw,closed,0,,,,,2,2018-06-04T17:25:30Z,2019-06-24T06:40:19Z,2019-06-24T06:40:19Z,OWNER,,https://channels.readthedocs.io/en/latest/contributing.html is a nice example of this done well.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/305/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 329661905,MDU6SXNzdWUzMjk2NjE5MDU=,306,Custom URL routing with independent tests,9599,simonw,closed,0,,,,,5,2018-06-05T23:40:08Z,2018-06-07T15:29:28Z,2018-06-07T15:29:28Z,OWNER,,"The more I think about #303 the more I feel that Datasette's URL routing needs go beyond Django-style regex matching. If we go custom, tests should live in `test_routing.py`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/306/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 330323860,MDExOlB1bGxSZXF1ZXN0MTkzMzYxMzQx,307,"Initial sketch of custom URL routing, refs #306",9599,simonw,closed,0,,,,,1,2018-06-07T15:26:48Z,2018-06-07T15:29:54Z,2018-06-07T15:29:41Z,OWNER,simonw/datasette/pulls/307,See #306 for background on this.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/307/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 332830309,MDU6SXNzdWUzMzI4MzAzMDk=,310,datasette publish now is broken in master,9599,simonw,closed,0,,,,,0,2018-06-15T16:01:14Z,2018-06-16T16:29:50Z,2018-06-16T16:29:50Z,OWNER,,"``` > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2 > unable to execute 'gcc': No such file or directory > error: command 'gcc' failed with exit status 1 > > ---------------------------------------- > Command ""/usr/local/bin/python -u -c ""import setuptools, tokenize;__file__='/tmp/pip-install-s73273rj/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))"" install --record /tmp/pip-record-yha7dxqq/install-record.txt --single-version-externally-managed --compile"" failed with error code 1 in /tmp/pip-install-s73273rj/httptools/ ``` Turns out the `python-slim` base image I introduced in b18e4515855c3f1eeca3dfcccdbb6df05869084a doesn't include gcc: https://github.com/docker-library/python/issues/60#issuecomment-134322383",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/310/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 332998752,MDExOlB1bGxSZXF1ZXN0MTk1MzM5MTEx,311,"?_labels=1 to expand foreign keys (in csv and json), refs #233",9599,simonw,closed,0,,,,,2,2018-06-16T16:31:12Z,2018-06-16T22:20:31Z,2018-06-16T22:20:31Z,OWNER,simonw/datasette/pulls/311,"Output looks something like this: { ""rowid"": 233, ""TreeID"": 121240, ""qLegalStatus"": { ""value"" 2, ""label"": ""Private"" } ""qSpecies"": { ""value"": 16, ""label"": ""Sycamore"" } ""qAddress"": ""91 Commonwealth Ave"", ... }",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/311/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 333000163,MDU6SXNzdWUzMzMwMDAxNjM=,312,"HTML, CSV and JSON views should support ?_col=&_col=",9599,simonw,closed,0,,,,,1,2018-06-16T16:53:35Z,2021-06-17T18:14:24Z,2018-06-16T17:00:12Z,OWNER,,To support whitelisting columns to display.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/312/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 333086005,MDU6SXNzdWUzMzMwODYwMDU=,313,Deploy demo of Datasette on every commit that passes tests,9599,simonw,closed,0,,,,,6,2018-06-17T19:19:12Z,2018-06-17T21:52:58Z,2018-06-17T21:52:58Z,OWNER,,We can use Travis CI and Zeit Now to ensure there is always a live demo of current master. We can ship archived demos for releases as well.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/313/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 333120982,MDExOlB1bGxSZXF1ZXN0MTk1NDEzMjQx,315,Streaming mode for downloading all rows as a CSV,9599,simonw,closed,0,,,,,0,2018-06-18T03:06:59Z,2018-06-18T03:29:13Z,2018-06-18T03:21:02Z,OWNER,simonw/datasette/pulls/315,Refs #266,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/315/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 334169932,MDU6SXNzdWUzMzQxNjk5MzI=,320,Need unit tests covering the different states for the advanced export box,9599,simonw,closed,0,,,,,1,2018-06-20T17:03:40Z,2018-07-24T04:53:20Z,2018-07-24T03:38:40Z,OWNER,,"There are quite a few variants of this box: ![2018-06-20 at 10 02 am](https://user-images.githubusercontent.com/9599/41673229-1d423adc-7471-11e8-99d4-4251f7d03aa5.png) Test coverage should exercise all of them, since the logic is a little unclear. https://github.com/simonw/datasette/blob/fdfbbbb9ee0d02fd4d43dfc42382252fa2287d6d/datasette/templates/table.html#L140-L159",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/320/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 334698969,MDU6SXNzdWUzMzQ2OTg5Njk=,323,Speed up Travis CI builds,9599,simonw,closed,0,,,,,1,2018-06-21T23:55:27Z,2018-07-10T15:03:37Z,2018-07-10T15:03:36Z,OWNER,,"They've got a bit slow. Part of this is the Zeit Now deploy, but the build-and-test cycle is taking at least a couple of minutes. ![2018-06-21 at 4 54 pm](https://user-images.githubusercontent.com/9599/41751010-e48c823e-7573-11e8-88f3-7aa8a7e53917.png) ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/323/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 334731076,MDExOlB1bGxSZXF1ZXN0MTk2NjI4MzA0,324,Speed up Travis by reusing pip wheel cache across builds,9599,simonw,closed,0,,,,,0,2018-06-22T03:20:08Z,2018-06-24T01:03:47Z,2018-06-24T01:03:47Z,OWNER,simonw/datasette/pulls/324,From https://atchai.com/blog/faster-ci/ - refs #323 ,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/324/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 335064777,MDU6SXNzdWUzMzUwNjQ3Nzc=,325,Error on row page if table has slashes in the name and ends in .csv,9599,simonw,closed,0,,,,,1,2018-06-23T03:43:42Z,2018-07-09T17:28:27Z,2018-07-08T05:21:59Z,OWNER,,"https://v0-23-1.datasette.io/fixtures-e14e080/table%252Fwith%252Fslashes.csv/3 > no such table: table%252Fwith%252Fslashes.csv From clicking the row link on https://v0-23-1.datasette.io/fixtures-e14e080/table%2Fwith%2Fslashes.csv",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/325/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 335141434,MDU6SXNzdWUzMzUxNDE0MzQ=,326,CSV should respect --cors and return cors headers,9599,simonw,closed,0,,,,,1,2018-06-24T00:44:07Z,2021-06-17T18:14:24Z,2018-06-24T00:59:45Z,OWNER,,Otherwise tools like Vega can't load data via CSV.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/326/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 336464733,MDU6SXNzdWUzMzY0NjQ3MzM=,328,"Installation instructions, including how to use the docker image",9599,simonw,closed,0,,,,,4,2018-06-28T03:59:33Z,2023-09-05T14:10:39Z,2018-06-28T04:02:10Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/328/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 336465018,MDU6SXNzdWUzMzY0NjUwMTg=,329,Travis should push tagged images to Docker Hub for each release,9599,simonw,closed,0,,,,,7,2018-06-28T04:01:31Z,2018-11-05T06:54:10Z,2018-11-05T06:53:28Z,OWNER,,https://sebest.github.io/post/using-travis-ci-to-build-docker-images/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/329/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 337141108,MDU6SXNzdWUzMzcxNDExMDg=,332,Sanely handle Infinity/-Infinity values in JSON using ?_json_infinity=1,9599,simonw,closed,0,,,,,12,2018-06-29T21:21:27Z,2018-07-24T04:53:20Z,2018-07-24T03:08:30Z,OWNER,,"It turns out if you load this CSV using `csvs-to-sqlite` you get an Infinity value in SQLite: ``` name,num sasha,10 terry,Inf cathy,0.5 ``` `csvs-to-sqlite infinity-bug.csv infinity-bug.db` I deployed this using: ``` datasette publish now infinity-bug.db --name=datasette-infinity-bug --install=datasette-vega ``` Datasette outputs that as `Infinity` in the JSON format, which causes JavaScript errors. Demo * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug - HTML view works * https://datasette-infinity-bug.now.sh/infinity-bug-0d0224e/infinity-bug.json?_shape=array - this outputs the following: ``` [ { ""rowid"": 1, ""name"": ""sasha"", ""num"": 10.0 }, { ""rowid"": 2, ""name"": ""terry"", ""num"": Infinity }, { ""rowid"": 3, ""name"": ""cathy"", ""num"": 0.5 } ] ``` But... in Firefox that gets rendered like this: ![2018-06-29 at 4 20 pm](https://user-images.githubusercontent.com/9599/42115408-5d30f630-7bb8-11e8-8370-c8484801c49b.png) And if you click the ""Show charting options"" button you get this error in the console: ``` SyntaxError: JSON.parse: unexpected character at line 1 column 83 of the JSON data ``` ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/332/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 338768551,MDU6SXNzdWUzMzg3Njg1NTE=,333,Datasette on Zeit Now returns http URLs for facet and next links,9599,simonw,closed,0,,,,,4,2018-07-06T00:40:49Z,2018-07-24T04:53:20Z,2018-07-24T01:51:53Z,OWNER,,"e.g. on https://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=0 ``` { ""facet_results"": { ""lg_id"": { ""name"": ""lg_id"", ""results"": [ { ""value"": ""NBA"", ""label"": ""NBA"", ""count"": 118016, ""toggle_url"": ""http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=NBA"", ""selected"": false }, { ""value"": ""ABA"", ""label"": ""ABA"", ""count"": 8298, ""toggle_url"": ""http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&lg_id=ABA"", ""selected"": false } ], ""truncated"": false } }, ""suggested_facets"": [ { ""name"": ""_iscopy"", ""toggle_url"": ""/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_facet=_iscopy"" } ], ""next_url"": ""http://fivethirtyeight.datasettes.com/fivethirtyeight-ac35616/nba-elo%2Fnbaallelo.json?_facet=lg_id&_size=1&_next=1"", } ``` `next_url` and `facet_results` both link to `http://` when they should link to `https://`. Note that suggested facets doesn't include the full URL at all, which is a consistency bug.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/333/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 339505204,MDU6SXNzdWUzMzk1MDUyMDQ=,335,Package datasette for installation using homebrew,9599,simonw,closed,0,,,,,12,2018-07-09T15:45:03Z,2020-08-11T16:54:06Z,2020-08-11T16:54:06Z,OWNER,,"https://docs.brew.sh/Python-for-Formula-Authors describes how. > Applications should be installed into a Python virtualenv environment rooted in libexec. This prevents the app’s Python modules from contaminating the system site-packages and vice versa. It recommends using https://github.com/tdsmith/homebrew-pypi-poet",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/335/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 340039409,MDU6SXNzdWUzNDAwMzk0MDk=,336,Ensure --help examples in docs are always up to date,9599,simonw,closed,0,,,,,3,2018-07-10T23:20:01Z,2018-07-24T16:01:29Z,2018-07-24T16:01:29Z,OWNER,,"Ideally I would automatically generate the --help output shown in our docs, but I don't think I can get that working with readthedocs. Instead, I'm going to add a unit test that checks that those extracts in the documentation match the current output of the --help command.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/336/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 340065374,MDU6SXNzdWUzNDAwNjUzNzQ=,337,Documentation for datasette publish and datasette package,9599,simonw,closed,0,,,,,1,2018-07-11T02:04:06Z,2018-07-11T02:07:32Z,2018-07-11T02:05:56Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/337/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 340282796,MDU6SXNzdWUzNDAyODI3OTY=,338,Only load vegaEmbed if charting tools are enabled,9599,simonw,closed,0,,,,,1,2018-07-11T15:02:14Z,2018-07-11T15:21:47Z,2018-07-11T15:21:47Z,OWNER,,"vegaEmbed is a LOT of code (it bundles d3) Inspired by this tweet: https://twitter.com/thelarkinn/status/1017053567641948162 - it would be great if we loaded that code on demand the first time the ""Show chart options"" button was clicked, or when the page loads with #g. options in the URL. Even better: avoid the overhead if loading React unless the chart options need to be displayed. This would be a pretty major refactoring though.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/338/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 340730961,MDU6SXNzdWUzNDA3MzA5NjE=,340,Embrace black,9599,simonw,closed,0,,,,,1,2018-07-12T17:32:29Z,2019-06-24T06:50:27Z,2019-06-24T06:50:26Z,OWNER,,"Run [black](https://github.com/ambv/black) against everything. Then set up CI to fail if code doesn't conform to black's style. Here's how Starlette does this: * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/.travis.yml#L14 * https://github.com/encode/starlette/blob/e3d090b3597167f7b3a4f76e4bb3c0d3e94be61a/scripts/lint - essentially runs `black starlette tests --check` And here's an example of a test run that failed: https://travis-ci.org/encode/starlette/jobs/403172478",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/340/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 340733753,MDExOlB1bGxSZXF1ZXN0MjAxMDc1NTMy,341,Bump aiohttp to fix compatibility with Python 3.7,9599,simonw,closed,0,,,,,0,2018-07-12T17:41:24Z,2018-07-12T18:07:38Z,2018-07-12T18:07:38Z,OWNER,simonw/datasette/pulls/341,Tests failed here: https://travis-ci.org/simonw/datasette/jobs/403223333,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/341/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 344656114,MDU6SXNzdWUzNDQ2NTYxMTQ=,348,"Unit tests for ""datasette publish""",9599,simonw,closed,0,,,,,1,2018-07-26T00:52:23Z,2018-07-26T05:46:10Z,2018-07-26T05:46:10Z,OWNER,,"The datasette publish family of commands all work by shelling out to heroku/now/docker from subprocess import call, check_output So in tests I should be able to mock those calls: @mock.patch('subprocess.call') ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/348/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 344695978,MDExOlB1bGxSZXF1ZXN0MjA0MDI5MTQy,349,"publish_subcommand hook + default plugins mechanism, used for publish heroku/now",9599,simonw,closed,0,,,,,1,2018-07-26T05:03:22Z,2018-07-26T05:28:54Z,2018-07-26T05:16:00Z,OWNER,simonw/datasette/pulls/349,"This change introduces a new plugin hook, publish_subcommand, which can be used to implement new subcommands for the ""datasette publish"" command family. I've used this new hook to refactor out the ""publish now"" and ""publish heroku"" implementations into separate modules. I've also added unit tests for these two publishers, mocking the subprocess.call and subprocess.check_output functions. As part of this, I introduced a mechanism for loading default plugins. These are defined in the new ""default_plugins"" list inside datasette/app.py Closes #217 (Plugin support for ""datasette publish"") Closes #348 (Unit tests for ""datasette publish"") Refs #14, #59, #102, #103, #146, #236, #347",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/349/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 344701755,MDU6SXNzdWUzNDQ3MDE3NTU=,350,Don't list default plugins on /-/plugins,9599,simonw,closed,0,,,,,2,2018-07-26T05:38:00Z,2018-08-28T17:13:50Z,2018-08-28T16:48:19Z,OWNER,,"https://dbbe707.datasette.io/-/plugins is showing ""datasette.publish.now"" and ""datasette.publish.heroku""",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/350/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 345469355,MDU6SXNzdWUzNDU0NjkzNTU=,351,Automatically create a GitHub release linking to release notes for every tagged release,9599,simonw,closed,0,,,,,1,2018-07-28T18:31:12Z,2020-05-28T18:56:16Z,2020-05-28T18:56:15Z,OWNER,,"Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release The release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/351/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 345821500,MDU6SXNzdWUzNDU4MjE1MDA=,352,render_cell(value) plugin hook,9599,simonw,closed,0,,,,,4,2018-07-30T15:56:20Z,2020-02-10T16:18:58Z,2018-08-05T00:14:57Z,OWNER,,To allow plugins to customize how values matching a specific pattern are displayed in the HTML table view.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/352/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 345821778,MDExOlB1bGxSZXF1ZXN0MjA0ODUxNTEx,353,render_cell(value) plugin hook,9599,simonw,closed,0,,,,,0,2018-07-30T15:57:08Z,2018-08-05T00:14:57Z,2018-08-05T00:14:57Z,OWNER,simonw/datasette/pulls/353,Closes #352.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/353/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 346028655,MDU6SXNzdWUzNDYwMjg2NTU=,356,Ability to display facet counts for many-to-many relationships,9599,simonw,closed,0,,,,,4,2018-07-31T04:14:26Z,2019-05-29T21:39:12Z,2019-05-25T16:30:09Z,OWNER,,Parent: #354,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/356/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 347058326,MDExOlB1bGxSZXF1ZXN0MjA1NzcwOTk2,1,Make .indexes compatible with older SQLite versions,9599,simonw,closed,0,,,,,0,2018-08-02T15:17:05Z,2018-08-02T15:17:30Z,2018-08-02T15:17:30Z,OWNER,simonw/sqlite-utils/pulls/1,Older SQLite versions return a different set of columns from the PRAGMA we are using.,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 348534997,MDExOlB1bGxSZXF1ZXN0MjA2ODYzODAz,358,"Bump versions of pytest, pluggy and beautifulsoup4",9599,simonw,closed,0,,,,,0,2018-08-08T00:44:38Z,2018-08-08T01:11:13Z,2018-08-08T01:11:13Z,OWNER,simonw/datasette/pulls/358,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/358/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 349827640,MDU6SXNzdWUzNDk4Mjc2NDA=,359,Faceted browse against a JSON list of tags,9599,simonw,closed,0,,,,,6,2018-08-12T17:01:14Z,2019-05-29T21:39:12Z,2019-05-03T00:21:44Z,OWNER,,"If a table has a `[""foo"", ""bar"", ""baz""]` JSON column allow that to be faceted against. - [x] Support `?column__arraycontains=x` filter queries - [x] Support `?_facet_array=column` faceting",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/359/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 349850687,MDU6SXNzdWUzNDk4NTA2ODc=,2,Mechanism for adding foreign keys to an existing table,9599,simonw,closed,0,,,,,1,2018-08-12T22:50:56Z,2019-02-24T21:34:41Z,2019-02-24T21:34:41Z,OWNER,,"SQLite does not have ALTER TABLE support for adding new foreign keys... but it turns out it's possible to make these changes without having to duplicate the entire table by carefully running `UPDATE sqlite_master SET sql=... WHERE type='table' AND name='X';` Here's how Django does it: https://github.com/django/django/blob/d3449faaa915a08c275b35de01e66a7ef6bdb2dc/django/db/backends/sqlite3/schema.py#L103-L125 And here's the official documentation about this: https://sqlite.org/lang_altertable.html#otheralter (scroll to the very bottom of the page)",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 351017129,MDU6SXNzdWUzNTEwMTcxMjk=,360,Use pysqlite3 if available,9599,simonw,closed,0,,,,,3,2018-08-16T00:50:45Z,2018-08-16T01:50:42Z,2018-08-16T00:58:58Z,OWNER,,[pysqlite3](https://github.com/coleifer/pysqlite3) is a way to provide access to a more recent version of SQLite than the standard library `sqlite3` module (which tends to use the version provided with the operating system - which on e.g. the Travis CI Ubuntu build environment can be as old as 3.8.0).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/360/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 351017365,MDExOlB1bGxSZXF1ZXN0MjA4NzE5MDQz,361," Import pysqlite3 if available, closes #360 ",9599,simonw,closed,0,,,,,0,2018-08-16T00:52:21Z,2018-08-16T00:58:57Z,2018-08-16T00:58:57Z,OWNER,simonw/datasette/pulls/361,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/361/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 351845423,MDU6SXNzdWUzNTE4NDU0MjM=,3,Experiment with contentless FTS tables,9599,simonw,closed,0,,,,,1,2018-08-18T19:31:01Z,2019-07-22T20:58:55Z,2019-07-22T20:58:55Z,OWNER,,Could greatly reduce size of resulting database for large datasets: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 377518499,MDU6SXNzdWUzNzc1MTg0OTk=,374,Get Datasette working with Zeit Now v2's 100MB image size limit,9599,simonw,closed,0,,,,,5,2018-11-05T18:08:29Z,2018-12-19T01:35:59Z,2018-12-19T01:35:59Z,OWNER,,"Follow-on from #366 Zeit Now's v2 cloud has a 100MB size limit on Docker images, in order to support much faster wake-ups of new instances. Fitting Datasette AND the SQLite database it is hosting in here is going to be a challenge.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/374/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 382471625,MDExOlB1bGxSZXF1ZXN0MjMyMTcyMTA2,389,Bump dependency versions,9599,simonw,closed,0,,,,,2,2018-11-20T02:23:12Z,2019-11-13T19:13:41Z,2019-11-13T19:13:41Z,OWNER,simonw/datasette/pulls/389,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/389/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 394751072,MDExOlB1bGxSZXF1ZXN0MjQxNDE4NDQz,392,Fix some regex DeprecationWarnings,9599,simonw,closed,0,,,,,0,2018-12-29T02:10:28Z,2018-12-29T02:22:28Z,2018-12-29T02:22:28Z,OWNER,simonw/datasette/pulls/392,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/392/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 396215043,MDU6SXNzdWUzOTYyMTUwNDM=,395,Find a cleaner pattern for fixtures with arguments,9599,simonw,closed,0,,,,,1,2019-01-06T00:31:22Z,2020-06-07T21:23:22Z,2020-06-07T21:23:22Z,OWNER,,"A lot of Datasette tests look like this: https://github.com/simonw/datasette/blob/b65d97792a53f78cb14b226231063209d22c4602/tests/test_api.py#L438-L444 The loop here isn't actually expected to loop - it's there because the `make_app_client` function yields a value and then cleans it up afterwards. This pattern works, but it is a little confusing. It would be nice to replace it with something less strange looking. The answer may be to switch to the ""factories as fixtures"" pattern described here: https://docs.pytest.org/en/latest/fixture.html#factories-as-fixtures In particular some variant of this example: ``` @pytest.fixture def make_customer_record(): created_records = [] def _make_customer_record(name): record = models.Customer(name=name, orders=[]) created_records.append(record) return record yield _make_customer_record for record in created_records: record.destroy() def test_customer_records(make_customer_record): customer_1 = make_customer_record(""Lisa"") customer_2 = make_customer_record(""Mike"") customer_3 = make_customer_record(""Meredith"") ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/395/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 397098882,MDU6SXNzdWUzOTcwOTg4ODI=,396,Add pragma compile_options output to /-/versions,9599,simonw,closed,0,,,,,1,2019-01-08T21:43:54Z,2019-01-11T00:55:22Z,2019-01-11T00:44:56Z,OWNER,,"``` sqlite> pragma compile_options ...> ; BUG_COMPATIBLE_20160819 COMPILER=clang-9.0.0 DEFAULT_CACHE_SIZE=2000 DEFAULT_CKPTFULLFSYNC DEFAULT_JOURNAL_SIZE_LIMIT=32768 DEFAULT_PAGE_SIZE=4096 DEFAULT_SYNCHRONOUS=2 DEFAULT_WAL_SYNCHRONOUS=1 ENABLE_API_ARMOR ENABLE_COLUMN_METADATA ENABLE_DBSTAT_VTAB ENABLE_FTS3 ENABLE_FTS3_PARENTHESIS ENABLE_FTS3_TOKENIZER ENABLE_FTS4 ENABLE_FTS5 ENABLE_JSON1 ENABLE_LOCKING_STYLE=1 ENABLE_PREUPDATE_HOOK ENABLE_RTREE ENABLE_SESSION ENABLE_SNAPSHOT ENABLE_SQLLOG ENABLE_UNKNOWN_SQL_FUNCTION ENABLE_UPDATE_DELETE_LIMIT HAVE_ISNAN MAX_LENGTH=2147483645 MAX_MMAP_SIZE=1073741824 MAX_VARIABLE_NUMBER=500000 OMIT_AUTORESET OMIT_LOAD_EXTENSION STMTJRNL_SPILL=131072 THREADSAFE=2 USE_URI sqlite> ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/396/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 398089089,MDU6SXNzdWUzOTgwODkwODk=,399,/-/versions for official Docker image returns wrong Datasette version,9599,simonw,closed,0,,,,,2,2019-01-11T01:19:58Z,2019-01-13T23:31:59Z,2019-01-13T23:10:45Z,OWNER,,"``` docker run -p 8001:8001 datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 ``` http://0.0.0.0:8001/-/versions returns this: ``` { ""datasette"": { ""version"": ""0+unknown"" }, ... ``` This is because the Docker image is built by copying in the Datasette source code, which confuses versioneer. Maybe the Docker image should install the code using a wheel or similar? ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/399/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 403028630,MDExOlB1bGxSZXF1ZXN0MjQ3NTc2OTQy,4,Fts5,9599,simonw,closed,0,,,,,0,2019-01-25T06:54:05Z,2019-01-25T06:54:33Z,2019-01-25T06:54:33Z,OWNER,simonw/sqlite-utils/pulls/4,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 403396009,MDExOlB1bGxSZXF1ZXN0MjQ3ODYxNDE5,5,Run Travis tests against Python 3.8-dev,9599,simonw,closed,0,,,,,0,2019-01-26T02:30:55Z,2019-01-26T02:37:54Z,2019-01-26T02:37:54Z,OWNER,simonw/sqlite-utils/pulls/5,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 403499298,MDExOlB1bGxSZXF1ZXN0MjQ3OTIzMzQ3,404,Experiment: run Jinja in async mode,9599,simonw,closed,0,,,,,3,2019-01-27T00:28:44Z,2019-11-12T05:02:18Z,2019-11-12T05:02:13Z,OWNER,simonw/datasette/pulls/404,"See http://jinja.pocoo.org/docs/2.10/api/#async-support Tests all pass. Have not checked performance difference yet. Creating pull request to run tests in Travis. This is not ready to merge - I'm not yet sure if this is a good idea.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/404/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 403617881,MDU6SXNzdWU0MDM2MTc4ODE=,405,.json?_nl=on option for exporting newline-delimited JSON,9599,simonw,closed,0,,,,,2,2019-01-28T01:10:45Z,2019-01-28T01:49:00Z,2019-01-28T01:48:37Z,OWNER,,"The neat thing about newline-delimited JSON is that you don't have to read an entire array (of potentially thousands of objects) into memory in order to parse it - you can parse things a line at a time instead. It will look like this: `https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on` ``` {""pk"": 1, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Mission""} {""pk"": 2, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Dogpatch""} {""pk"": 3, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""SOMA""} {""pk"": 4, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Tenderloin""} {""pk"": 5, ""planet_int"": 1, ""on_earth"": 1, ""state"": ""CA"", ""city_id"": 1, ""neighborhood"": ""Bernal Heights""} ``` I added this as part of the `sqlite-utils json` CLI command is this commit - I think Datasette should offer it as well: https://github.com/simonw/sqlite-utils/commit/5466c9745dfef858286146ea158ffd5a71391d10 It can be offered alongside `_stream=on` (which currently only works for CSV, but it could work for JSON as well thanks to this trick).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/405/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 403624090,MDU6SXNzdWU0MDM2MjQwOTA=,6,"""sqlite-utils insert"" should support newline-delimited JSON",9599,simonw,closed,0,,,,,1,2019-01-28T02:00:02Z,2019-01-28T02:17:45Z,2019-01-28T02:17:45Z,OWNER,,"We can already export newline delimited JSON. We should learn to import it as well. The neat thing about importing it is that you can import GBs of data without having to read the whole lot into memory in order to decode the wrapping JSON array. Datasette can export it now: https://github.com/simonw/datasette/issues/405 Demo: https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on It should be possible to do this: $ curl ""https://latest.datasette.io/fixtures/facetable.json?_shape=array&_nl=on"" \ | sqlite-utils insert data.db facetable - --nl ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 403625674,MDU6SXNzdWU0MDM2MjU2NzQ=,7,.insert_all() should accept a generator and process it efficiently,9599,simonw,closed,0,,,,,3,2019-01-28T02:11:58Z,2019-01-28T06:26:53Z,2019-01-28T06:26:53Z,OWNER,,"Right now you have to load every record into memory before passing the list to `.insert_all()` and friends. If you want to process millions of rows, this is inefficient. Python has generators - we should use them! The only catch here is that part of the magic of `sqlite-utils` is that it guesses the column types and creates the table for you. This code will need to be updated to notice if the table needs creating and, if it does, create it using the first X (where x=1,000 but can be customized) records. If a record outside of those first 1,000 has a rogue column, we can crash with an error. This will free us up to make the `--nl` option added in #6 much more efficient.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 406055201,MDU6SXNzdWU0MDYwNTUyMDE=,406,Support nullable foreign keys in _labels mode,9599,simonw,closed,0,9599,simonw,,,2,2019-02-03T05:34:20Z,2019-11-02T22:39:28Z,2019-11-02T22:30:27Z,OWNER,,"Currently if there's a null in a foreign key we get ""None"" displayed in the inflated view: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/406/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 407073223,MDExOlB1bGxSZXF1ZXN0MjUwNjI4Mjc1,407,Heroku --include-vcs-ignore,9599,simonw,closed,0,,,,,1,2019-02-06T04:06:20Z,2019-02-06T04:31:30Z,2019-02-06T04:15:47Z,OWNER,simonw/datasette/pulls/407,"Should mean `datasette publish heroku` can work under Travis, unlike this failure: https://travis-ci.org/simonw/fivethirtyeight-datasette/builds/488047550 ``` 2.25s$ datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette tar: unrecognized option '--exclude-vcs-ignores' Try 'tar --help' or 'tar --usage' for more information. ▸ Command failed: tar cz -C /tmp/tmpuaxm7i8f --exclude-vcs-ignores --exclude ▸ .git --exclude .gitmodules . > ▸ /tmp/f49440e0-1bf3-4d3f-9eb0-fbc2967d1fd4.tar.gz ▸ tar: unrecognized option '--exclude-vcs-ignores' ▸ Try 'tar --help' or 'tar --usage' for more information. ▸ The command ""datasette publish heroku fivethirtyeight.db -m metadata.json -n fivethirtyeight-datasette"" exited with 0. ``` The fix for that issue is to call the heroku command like this: heroku builds:create -a app_name --include-vcs-ignore ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/407/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 413740684,MDU6SXNzdWU0MTM3NDA2ODQ=,11,Detect numpy types when creating tables,9599,simonw,closed,0,,,,,2,2019-02-23T21:09:35Z,2019-02-24T04:02:20Z,2019-02-24T04:02:20Z,OWNER,,Inspired by #8,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/11/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 413778585,MDExOlB1bGxSZXF1ZXN0MjU1NjU4MTEy,12,"Support for numpy types, closes #11",9599,simonw,closed,0,,,,,0,2019-02-24T03:57:32Z,2019-02-24T04:02:20Z,2019-02-24T04:02:20Z,OWNER,simonw/sqlite-utils/pulls/12,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 413779210,MDU6SXNzdWU0MTM3NzkyMTA=,13,Ability to automatically create IDs from content hash of row,9599,simonw,closed,0,,,,,1,2019-02-24T04:07:08Z,2019-02-24T04:36:48Z,2019-02-24T04:36:48Z,OWNER,,"Sometimes when you are importing data the underlying source provides records without IDs that can be uniquely identified by their contents. A utility mechanism for calculating a sha1 hash of the contents and using that as a unique ID would be useful.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 413842611,MDU6SXNzdWU0MTM4NDI2MTE=,14,Utilities for adding indexes,9599,simonw,closed,0,,,,,3,2019-02-24T16:57:28Z,2019-02-24T19:11:28Z,2019-02-24T19:11:28Z,OWNER,,"Both in the Python API and the CLI tool. For the CLI tool this should work: $ sqlite-utils create-index mydb.db mytable col1 col2 This will create a compound index across col1 and col2. The name of the index will be automatically chosen unless you use the `--name=...` option. Support a `--unique` option too.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 413857257,MDU6SXNzdWU0MTM4NTcyNTc=,15,Ability to add columns to tables,9599,simonw,closed,0,,,,,0,2019-02-24T19:20:51Z,2019-02-24T20:04:40Z,2019-02-24T20:04:40Z,OWNER,,"Makes sense to do this before foreign keys in #2 Python: db[""table""].add_column(""new_column"", int) CLI: $ sqlite-utils add-column table new_column INTEGER ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/15/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 413867537,MDU6SXNzdWU0MTM4Njc1Mzc=,16,add_column() should support REFERENCES {other_table}({other_column}),9599,simonw,closed,0,,,,,4,2019-02-24T21:00:45Z,2019-05-29T05:17:59Z,2019-05-29T04:56:18Z,OWNER,,Related to #2 ,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/16/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 413868452,MDU6SXNzdWU0MTM4Njg0NTI=,17,Improve and document foreign_keys=... argument to insert/create/etc,9599,simonw,closed,0,,,,,7,2019-02-24T21:09:11Z,2019-02-24T23:45:48Z,2019-02-24T23:45:48Z,OWNER,,"The `foreign_keys=` argument to `table.insert_all()` and friends can be used to specify foreign key relationships that should be created. It is not yet documented. It also requires you to specify the SQLite type of each column, even though this can be detected by introspecting the referenced table: cols = [c for c in self.db[other_table].columns if c.name == other_column] cols[0].type Relates to #2 ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 421348146,MDExOlB1bGxSZXF1ZXN0MjYxNDE4Mjg1,416,URL hashing now optional: turn on with --config hash_urls:1 (#418),9599,simonw,closed,0,,,,,8,2019-03-15T04:26:06Z,2019-03-17T22:55:04Z,2019-03-17T22:55:04Z,OWNER,simonw/datasette/pulls/416,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/416/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 423316403,MDU6SXNzdWU0MjMzMTY0MDM=,422,Figure out what to do about table counts in a mutable world,9599,simonw,closed,0,,,,,4,2019-03-20T15:27:15Z,2019-05-02T05:43:11Z,2019-05-02T05:43:11Z,OWNER,,"In moving away from the existing static inspect method (see #420 and #419) the biggest thing lost is full table row counts. These can be expensive against large tables, but currently Datasette runs the `count (*) from x` query once at inspection time and then reuses it for every page. We can run those counts with a timelimit, but this means that for larger tables we won't be able to show a count at all, which is disappointing. Is there a way we can find an approximate or lower bound count for a table?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/422/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 430103450,MDU6SXNzdWU0MzAxMDM0NTA=,425,Submitting SQL on hide page is broken,9599,simonw,closed,0,,,,,2,2019-04-07T04:21:31Z,2019-04-12T05:12:13Z,2019-04-12T05:00:53Z,OWNER,,Clicking the submit button here doesn't work correctly: https://3a208a4.datasette.io/fixtures?sql=select+%2A+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101&_hide_sql=1,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 431756352,MDExOlB1bGxSZXF1ZXN0MjY5MzY0OTI0,426,Upgrade to Jinja2==2.10.1,9599,simonw,closed,0,,,,,1,2019-04-10T23:03:08Z,2019-04-22T21:23:22Z,2019-04-10T23:13:31Z,OWNER,simonw/datasette/pulls/426,"https://nvd.nist.gov/vuln/detail/CVE-2019-10906 This is only a security issue of concern if evaluating templates from untrusted sources, which isn't something I would ever expect a Datasette user to do.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/426/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 431800286,MDU6SXNzdWU0MzE4MDAyODY=,427,"New design for facet abstraction, including querystring and metadata.json",9599,simonw,closed,0,,,,,10,2019-04-11T02:24:15Z,2019-05-29T21:39:12Z,2019-05-03T00:11:29Z,OWNER,,"I need a better design for query strings for facets (and for how facets are enabled in `metadata.json`). Think of all of the potential kinds of facets: * `?_facet_array=tags` where tags is a JSON array of values * `_facet_date=datetimecol` - faceted by date part of a datetime * `_facet_bins=numeric_column` - can I do some kind of fancy binning here? Might need to take an argument * `?_facet_bins=numeric_column:5` - could be a way to take an argument. We’ll ignore columns with a : in their name. * `?_facet_json=jsoncol:jsonpath` - could use a JSON path to extract out something to facet on? * `?_facet_percentile=numericcolumn` - could this work? * `?_facet_function=column:sqlfunctionname` - maybe this could be interesting? Would allow for e.g. facet by soundex * `?_facet_prefix=column:prefix` - facet by terms but only if they start with a specific prefix * `?_facet_substring=column:3,6` - facet by a substr(column, 3, 6) Maybe bundling JSON in querystrings is a way to do options? `?_facet_distance={""latitude_column"":""x"",...}` Could detect values starting with `{` - and if for some weird reason you have a column starting with that character you can pass this instead: `?_facet_percentile={""column"": ""{value}""}` This could even be the mechanism that allows us to extend regular facets to support additional options like adding a sum or max to each one. Problem: it’s not obvious what the name associated with these facets should be. What if one column is faceted multiple times using multiple facet variants? Maybe just number them? name1=… name2=… etc? Other option is to use Solr style querystring syntax for notation. Solr does this: `?f.price.facet.range.gap=100&f.age.facet.range.gap=10` So how about this: `?_facet_range=age&_facet_range.span=5` Related: #359",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/427/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 432371762,MDU6SXNzdWU0MzIzNzE3NjI=,428,Make ?_fts_table=x and ?_fts_pk=y available as URL parameters on table view,9599,simonw,closed,0,,,,,2,2019-04-12T03:30:55Z,2019-04-12T04:30:29Z,2019-04-12T04:21:25Z,OWNER,,"These can currently only be set using `metadata.json`: https://datasette.readthedocs.io/en/0.27/full_text_search.html#configuring-full-text-search-for-a-table-or-view There's no reason not to support these as URL parameters as well. That way it would be easy to use FTS search against a view without having to use `metadata.json`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/428/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 432636432,MDU6SXNzdWU0MzI2MzY0MzI=,429,?_where=sql-fragment parameter for table views,9599,simonw,closed,0,,,,,7,2019-04-12T15:58:51Z,2019-04-15T10:48:01Z,2019-04-13T01:37:25Z,OWNER,,"Only available if arbitrary SQL is enabled (the default). `?_where=id in (1,2,3)&_where=id in (select tag_id from tags)` Allows any table (or view) page to have arbitrary additional `extra_where` clauses defined using the URL! This would be extremely useful for building JavaScript applications against the Datasette API that only need on extra tiny bit of SQL but still want to benefit from other table view features like faceting. Would be nice if this could take `:named` parameters and have them filled in via querystring as well.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/429/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 432792459,MDExOlB1bGxSZXF1ZXN0MjcwMTkxMDg0,430,"?_where= parameter on table views, closes #429",9599,simonw,closed,0,,,,,0,2019-04-13T01:15:09Z,2019-04-13T01:37:23Z,2019-04-13T01:37:23Z,OWNER,simonw/datasette/pulls/430,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/430/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 432893491,MDExOlB1bGxSZXF1ZXN0MjcwMjUxMDIx,432,"Refactor facets to a class and new plugin, refs #427",9599,simonw,closed,0,,,,,4,2019-04-13T20:04:45Z,2019-05-03T00:04:24Z,2019-05-03T00:04:24Z,OWNER,simonw/datasette/pulls/432,WIP for #427,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/432/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 433297989,MDU6SXNzdWU0MzMyOTc5ODk=,433,"?column__in=value1,value2,value3 filter",9599,simonw,closed,0,,,,,0,2019-04-15T13:58:24Z,2019-04-15T23:00:20Z,2019-04-15T23:00:20Z,OWNER,,"Support for the SQL `where column in (...)` construct, inspired by the new design for facet configuration in #427 `?column__in=value1,value2,value3` will map to `where column in (""value1"", ""value2"", ""value3"")` If comma separation won't work (because the values themselves contain commas) you can do this instead: `?column__in=[""value1"",""value2"",""value3,with-comma""]` See also #288",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/433/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 439836586,MDExOlB1bGxSZXF1ZXN0Mjc1NTU4NjEy,445,"Extract facet code out into a new plugin hook, closes #427",9599,simonw,closed,0,,,,,0,2019-05-03T00:02:41Z,2019-05-03T18:17:18Z,2019-05-03T00:11:27Z,OWNER,simonw/datasette/pulls/445,"Datasette previously only supported one type of faceting: exact column value counting. With this change, faceting logic is extracted out into one or more separate classes which can implement other patterns of faceting - this is discussed in #427, but potential upcoming facet types include facet-by-date, facet-by-JSON-array, facet-by-many-2-many and more. A new plugin hook, register_facet_classes, can be used by plugins to add in additional facet classes. Each class must implement two methods: suggest(), which scans columns in the table to decide if they might be worth suggesting for faceting, and facet_results(), which executes the facet operation and returns results ready to be displayed in the UI.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/445/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 440159137,MDExOlB1bGxSZXF1ZXN0Mjc1ODAxNDYz,447,Use dist: xenial and python: 3.7 on Travis,9599,simonw,closed,0,,,,,1,2019-05-03T18:07:07Z,2019-05-03T18:17:05Z,2019-05-03T18:16:53Z,OWNER,simonw/datasette/pulls/447,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/447/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 440237422,MDExOlB1bGxSZXF1ZXN0Mjc1ODYxNTU5,449,Apply black to everything,9599,simonw,closed,0,,,,,0,2019-05-03T21:57:26Z,2019-05-04T02:17:14Z,2019-05-04T02:15:15Z,OWNER,simonw/datasette/pulls/449,"I've been hesitating on this for literally months, because I'm not at all excited about the giant diff that will result. But I've been using black on many of my other projects (most actively [sqlite-utils](https://github.com/simonw/sqlite-utils)) and the productivity boost is undeniable: I don't have to spend a single second thinking about code formatting any more! So it's worth swallowing the one-off pain and moving on in a new, black-enabled world.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/449/reactions"", ""total_count"": 4, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 4, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 440332621,MDU6SXNzdWU0NDAzMzI2MjE=,453,Error pages do not return CORS header with --cors,9599,simonw,closed,0,,,,,1,2019-05-04T15:07:44Z,2019-05-05T12:24:24Z,2019-05-05T12:11:33Z,OWNER,,"This is very confusing. It means that if you send invalid SQL you will get back a CORS error, because the resulting 400 page cannot be accessed via JavaScript.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/453/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 440437037,MDU6SXNzdWU0NDA0MzcwMzc=,454,Plugin for allowing CORS from specified hosts,9599,simonw,closed,0,9599,simonw,,,5,2019-05-05T12:05:02Z,2019-10-03T23:59:57Z,2019-10-03T23:59:56Z,OWNER,,"It would be useful if Datasette could be configured to allow CORS requests from one or more origins, as opposed to only allowing either none or `""*""`. This is slightly tricky because the `Access-Control-Allow-Origin: https://foo.example` header is only allowed to return one value per request - and according to https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS ""The Access-Control-Allow-Origin header should contain the value that was sent in the request's Origin header."" This means the application code needs to have a whitelist of allowed hosts and code that dynamically changes the outgoing `Access-Control-Allow-Origin` header based on the `Origin` header from the incoming request.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/454/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 442330564,MDU6SXNzdWU0NDIzMzA1NjQ=,457,"Ability to ""publish cloudrun"" with no user input",9599,simonw,closed,0,,,,,2,2019-05-09T16:42:51Z,2019-05-09T19:41:31Z,2019-05-09T16:45:08Z,OWNER,,"If you attempt to deploy a new version of a cloudrun deployment, the script currently pauses and asks for user input for the service name like this: ```77d4d7de-3dfc-4acc-9a23-efe16230f318 2019-05-09T15:01:48+00:00 52S gs://datasette-222320_cloudbuild/source/1557414063.1-3a82df8096e9434b93511b0588d8d155.tgz gcr.io/datasette-222320/sf-trees (+1 more) SUCCESS Service name: (sf-trees): USER INPUT REQUIRED HERE Deploying container to Cloud Run service [sf-trees] in project [datasette-222320] region [us-central1] ✓ Deploying... Done. ✓ Creating Revision... ✓ Routing traffic... ✓ Setting IAM Policy... ``` This is incompatible with running under CI.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/457/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 443038584,MDU6SXNzdWU0NDMwMzg1ODQ=,465,Decide what to do about /-/inspect,9599,simonw,closed,0,,,,,4,2019-05-11T21:39:46Z,2019-06-28T16:34:33Z,2019-06-28T16:34:33Z,OWNER,,"It's not clear to me what this endpoint should do now as a result of #419 - it's still useful to be able to introspect databases for tools like datasette-registry, but since we aren't pre-calculating introspection data any more I need to rethink the approach. For one thing, this endpoint may need to be paginated. Or maybe it should be split up into separate endpoints for each connected database? Those should probably be paginated too seeing as fivethirtyeight has 400+ tables.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/465/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 444749373,MDU6SXNzdWU0NDQ3NDkzNzM=,469,publish commands should use new -i option,9599,simonw,closed,0,,,,,1,2019-05-16T04:31:40Z,2019-05-19T22:53:41Z,2019-05-19T22:53:41Z,OWNER,,"I can make this change only after releasing 0.28 - if I make the change earlier than that `publish heroku` etc will break because they will install the latest release of Datasette which will not understand the `-i` option. This is a one-line fix: replace this: https://github.com/simonw/datasette/blob/2ad9d15cd6901654e6801e2faa29e6fc08bae5fa/datasette/utils.py#L489 With this: (need to do it for other publishers too though) ``` quoted_files = "" "".join( [""-i {}"".format(shlex.quote(file_name)) for file_name in file_names] ) ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/469/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 445850934,MDU6SXNzdWU0NDU4NTA5MzQ=,473,Plugin hook: filters_from_request,9599,simonw,closed,0,,,,,13,2019-05-19T18:44:33Z,2021-12-17T23:11:30Z,2021-12-17T19:02:17Z,OWNER,,"I meant to add this as part of the facets plugin mechanism but didn't quite get to it. Original idea was to allow plugins to register extra filters, as seen in `datasette/filters.py`: https://github.com/simonw/datasette/blob/260085838887ee343f4d3b177c422e7aef5ade9d/datasette/filters.py#L83-L98",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/473/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 446429421,MDU6SXNzdWU0NDY0Mjk0MjE=,481,Facet by date,9599,simonw,closed,0,,,,,1,2019-05-21T05:55:54Z,2019-05-29T21:39:12Z,2019-05-21T06:09:49Z,OWNER,,Ability to facet on datetime fields by their date.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/481/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 448391492,MDU6SXNzdWU0NDgzOTE0OTI=,21,Option to ignore inserts if primary key exists already,9599,simonw,closed,0,,,,,3,2019-05-25T00:17:12Z,2019-05-29T05:09:01Z,2019-05-29T04:18:26Z,OWNER,,"> I've just noticed that SQLite lets you IGNORE inserts that collide with a pre-existing key. This can be quite handy if you have a dataset that keeps changing in part, and you don't want to upsert and replace pre-existing PK rows but you do want to ignore collisions to existing PK rows. > > Do `sqlite_utils` support such (cavalier!) behaviour? _Originally posted by @psychemedia in https://github.com/simonw/sqlite-utils/issues/18#issuecomment-480621924_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/21/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 448664792,MDU6SXNzdWU0NDg2NjQ3OTI=,487,Refactor database methods off Datasette class,9599,simonw,closed,0,,,,,1,2019-05-27T04:52:41Z,2019-05-27T20:05:34Z,2019-05-27T05:08:01Z,OWNER,,"Methods like this one: https://github.com/simonw/datasette/blob/182a3017c24e3fa3af60e4ac0c91c7e48f8736fd/datasette/app.py#L497-L503 Should live on the `ConnectedDatabase` class instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/487/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 448668204,MDU6SXNzdWU0NDg2NjgyMDQ=,488,Move detect_primary_keys to Database class method,9599,simonw,closed,0,,,,,0,2019-05-27T05:11:51Z,2019-05-27T20:05:34Z,2019-05-27T18:29:02Z,OWNER,,"e.g. https://github.com/simonw/datasette/blob/026c84db30bd0a75ecde146a80a5d142078dc299/datasette/views/table.py#L73-L75 Should be ``` pks = await db.primary_keys(table) ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/488/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 448977444,MDU6SXNzdWU0NDg5Nzc0NDQ=,489,Pagination breaks when combined with expanded foreign keys,9599,simonw,closed,0,,,,,1,2019-05-27T19:56:56Z,2019-05-28T02:48:57Z,2019-05-28T02:23:27Z,OWNER,,"Consider https://edb3662.datasette.io/fixtures/roadside_attraction_characteristics?_sort=attraction_id&_size=2 The ""Next page"" link goes here, which returns 0 rows: https://edb3662.datasette.io/fixtures/roadside_attraction_characteristics?_size=2&_next=%257B%2527value%2527%253A%2B2%252C%2B%2527label%2527%253A%2B%2527Winchester%2BMystery%2BHouse%2527%257D%2C2&_sort=attraction_id That's because if you double-url-decode that `_next` link you get this: `_next={'value': 2, 'label': 'Winchester Mystery House'},2` It should be `_next=2,2`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/489/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 448978907,MDU6SXNzdWU0NDg5Nzg5MDc=,490,Rename InterruptedError exception class,9599,simonw,closed,0,,,,,1,2019-05-27T20:04:25Z,2019-05-28T00:16:45Z,2019-05-28T00:16:45Z,OWNER,,"https://github.com/simonw/datasette/blob/edb36629e7356f70f42b9d37fea5dfe9cc3c364a/datasette/utils.py#L49-L50 Python has a built-in exception called this, so we should call ours something else.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/490/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 449565204,MDU6SXNzdWU0NDk1NjUyMDQ=,23,Syntactic sugar for creating m2m records,9599,simonw,closed,0,,,,,10,2019-05-29T02:17:48Z,2019-08-04T03:54:58Z,2019-08-04T03:37:34Z,OWNER,,Python library only. What would be a syntactically pleasant way of creating a m2m record?,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/23/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 449848803,MDU6SXNzdWU0NDk4NDg4MDM=,25,"Allow .insert(..., foreign_keys=()) to auto-detect table and primary key",9599,simonw,closed,0,,,,,4,2019-05-29T14:39:22Z,2019-06-13T05:32:32Z,2019-06-13T05:32:32Z,OWNER,,"The `foreign_keys=` argument currently takes a list of triples: ```python db[""usages""].insert_all( usages_to_insert, foreign_keys=( (""line_id"", ""lines"", ""id""), (""definition_id"", ""definitions"", ""id""), ), ) ``` As of #16 we have a mechanism for detecting the primary key column (the third item in this triple) - we should use that here too, so foreign keys can be optionally defined as a list of pairs.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 449931899,MDU6SXNzdWU0NDk5MzE4OTk=,494,--reload should only trigger for -i databases,9599,simonw,closed,0,9599,simonw,,,1,2019-05-29T17:28:43Z,2020-02-24T19:45:05Z,2020-02-24T19:45:05Z,OWNER,,Right now it's triggering any time a mutable database changes.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/494/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 451261628,MDExOlB1bGxSZXF1ZXN0Mjg0MzkwMTk3,497,Upgrade pytest to 4.6.1,9599,simonw,closed,0,,,,,0,2019-06-03T01:45:34Z,2019-06-03T02:06:32Z,2019-06-03T02:06:27Z,OWNER,simonw/datasette/pulls/497,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/497/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 452901999,MDExOlB1bGxSZXF1ZXN0Mjg1Njk4MzEw,501,Test against Python 3.8-dev using Travis,9599,simonw,closed,0,,,,,3,2019-06-06T08:37:53Z,2019-11-11T03:23:29Z,2019-11-11T03:23:29Z,OWNER,simonw/datasette/pulls/501,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/501/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 453639196,MDU6SXNzdWU0NTM2MzkxOTY=,504,Remove TableView ?_group_count= feature,9599,simonw,closed,0,9599,simonw,,,0,2019-06-07T18:25:18Z,2019-11-06T05:13:10Z,2019-11-06T05:13:10Z,OWNER,,"This feature really doesn't warrant continuing to exist. For reference: #150 and #44 Don't forget to remove it from the docs: https://github.com/simonw/datasette/blob/172da009d890aa029cff7138b4dcfd4f60948525/docs/json_api.rst#L322-L324",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/504/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 453846217,MDU6SXNzdWU0NTM4NDYyMTc=,506,Option to display binary data,9599,simonw,closed,0,,,,,10,2019-06-08T23:44:12Z,2019-06-11T15:48:27Z,2019-06-09T16:07:39Z,OWNER,,"In #442 we suppressed rendering of binary data: It turns out there is one use-case where displaying binary data is useful: when you're poking around looking at random SQLite databases you find in `~/Library` trying to figure out what they are for. So, a mechanism for opting in to ugly display of binary data again would be useful.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/506/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 455496504,MDU6SXNzdWU0NTU0OTY1MDQ=,27,sqlite-utils create-table command,9599,simonw,closed,0,,,,,8,2019-06-13T01:43:30Z,2020-05-03T15:26:15Z,2020-05-03T15:26:15Z,OWNER,,"Spun off from #24 - it would be useful if CLI users could create new tables (with explicit column types, not null rules and defaults) without having to insert an example record. - [x] Get it working - [x] Support `--pk` - [x] Support `--not-null` - [x] Support `--default` - [x] Support `--fk colname othertable othercol` - [x] Support `--replace` and `--ignore` - [x] Documentation",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 455965174,MDU6SXNzdWU0NTU5NjUxNzQ=,508,Ability to set default sort order for a table or view in metadata.json,9599,simonw,closed,0,9599,simonw,,,1,2019-06-13T21:40:51Z,2020-05-28T18:53:03Z,2020-05-28T18:53:02Z,OWNER,,"It can go here in the documentation: https://datasette.readthedocs.io/en/stable/metadata.html#setting-which-columns-can-be-used-for-sorting Also need to fix this sentence which is no longer true: > By default, database views in Datasette do not support sorting",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/508/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 455996809,MDU6SXNzdWU0NTU5OTY4MDk=,28,"Rearrange the docs by area, not CLI vs Python",9599,simonw,closed,0,,,,,1,2019-06-13T23:33:35Z,2019-07-15T02:37:20Z,2019-07-15T02:37:20Z,OWNER,,"The docs for eg inserting data should live on the same page, rather than being split across the API and CLI pages.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/28/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 458941203,MDU6SXNzdWU0NTg5NDEyMDM=,29,Prevent accidental add-foreign-key with invalid column,9599,simonw,closed,0,,,,,0,2019-06-20T23:57:24Z,2019-06-20T23:58:26Z,2019-06-20T23:58:26Z,OWNER,,"You can corrupt your database by running: $ sqlite-utils add-foreign-key my.db table non_existent_column other_table other_column ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/29/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 459537047,MDU6SXNzdWU0NTk1MzcwNDc=,517,"Add unit test for ""static"" mechanism in plugins",9599,simonw,closed,0,,,,,1,2019-06-23T05:03:31Z,2021-01-04T20:15:19Z,2021-01-04T20:15:19Z,OWNER,,"Split out from #272 - this is actually quite tricky. Here's the relevant code: https://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/utils.py#L602-L614",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/517/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 459598080,MDU6SXNzdWU0NTk1OTgwODA=,520,asgi_wrapper plugin hook,9599,simonw,closed,0,9599,simonw,,,3,2019-06-23T17:16:45Z,2019-07-03T04:40:34Z,2019-07-03T04:06:28Z,OWNER,,"After #272 we can finally add this hook. It will allow plugins to wrap their own ASGI middleware around Datasette. Potential use-cases include: * adding authentication * custom CORS headers (see #454) * maybe gzip support? * possibly defining entirely new routes, though that may be better handled by a separate hook",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/520/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 459621683,MDU6SXNzdWU0NTk2MjE2ODM=,521,Easier way of creating custom row templates,9599,simonw,closed,0,,,,,6,2019-06-23T21:49:27Z,2019-07-03T03:23:56Z,2019-07-03T03:23:56Z,OWNER,,"I was messing around with a custom `_rows_and_columns.html` template and ended up with this: ```html {% for row in display_rows %}
{% for cell in row %} {% if cell.column == ""First_Name"" %}

{{ cell.value }} {% elif cell.column == ""Last_Name"" %} {{ cell.value }}

{% elif cell.column == ""Short_Description"" %}

{{ cell.column }}: {{ cell.value }}

{% else %} {{ cell.column }}: {{ cell.value }}    {% endif %} {% endfor %}

{% endfor %} ``` This is nasty. I'd like to be able to do something like this instead: ``` {% for row in display_rows %}

{{ row[""First_Name""] }} {{ row[""Last_Name""] }}

... ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/521/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 459689615,MDExOlB1bGxSZXF1ZXN0MjkwOTcxMjk1,524,"Sort commits using isort, refs #516",9599,simonw,closed,0,,,,,1,2019-06-24T05:04:48Z,2023-08-23T01:31:08Z,2023-08-23T01:31:08Z,OWNER,simonw/datasette/pulls/524,Also added a lint unit test to ensure they stay sorted. #516,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/524/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 459714943,MDU6SXNzdWU0NTk3MTQ5NDM=,525,Add section on sqite-utils enable-fts to the search documentation,9599,simonw,closed,0,9599,simonw,,,2,2019-06-24T06:39:16Z,2019-06-24T16:36:35Z,2019-06-24T16:29:43Z,OWNER,,"https://datasette.readthedocs.io/en/stable/full_text_search.html already has a section about csvs-to-sqlite, sqlite-utils is even more relevant.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/525/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 460540321,MDU6SXNzdWU0NjA1NDAzMjE=,530,Extract codemirror SQL editor out into a plugin,9599,simonw,closed,0,,,,,1,2019-06-25T17:07:51Z,2020-10-01T00:42:08Z,2020-10-01T00:42:08Z,OWNER,,"Right now codemirror (used for the SQL editor on https://latest.datasette.io/fixtures?sql=select+*+from+%5B123_starts_with_digits%5D ) is the only JavaScript in Datasette. It's also the only vendored dependency. I'd like to move it out to a plugin. But... ideally I would like that plugin to be part of the default ""pip install datasette"" experience. I don't know what the best pattern for optional dependencies is. I don't want to have to tell people to run `pip install datasette[full]`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/530/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 461215118,MDU6SXNzdWU0NjEyMTUxMTg=,30,Option to open database in read-only mode,9599,simonw,closed,0,,,,,1,2019-06-26T22:50:38Z,2020-05-11T19:17:17Z,2020-05-11T19:17:17Z,OWNER,,Would this make it 100% safe to run reads against a database file that is being written to by another process?,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/30/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 461237618,MDU6SXNzdWU0NjEyMzc2MTg=,31,Mechanism for adding multiple foreign key constraints at once,9599,simonw,closed,0,,,,,0,2019-06-27T00:04:30Z,2019-06-29T06:27:40Z,2019-06-29T06:27:40Z,OWNER,,"Needed by [db-to-sqlite](https://github.com/simonw/db-to-sqlite). It currently works by collecting all of the foreign key relationships it can find and then applying them at the end of the process. The problem is, the `add_foreign_key()` method looks like this: https://github.com/simonw/sqlite-utils/blob/86bd2bba689e25f09551d611ccfbee1e069e5b66/sqlite_utils/db.py#L498-L516 That means it's doing a full `VACUUM` for every single relationship it sets up - and if you have hundreds of foreign key relationships in your database this can take hours. I think the right solution is to have a `.add_foreign_keys(list_of_args)` method which does the bulk operation and then a single `VACUUM`. `.add_foreign_key(...)` can then call the bulk action with a single list item.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 462094937,MDExOlB1bGxSZXF1ZXN0MjkyODc5MjA0,32,db.add_foreign_keys() method,9599,simonw,closed,0,,,,,1,2019-06-28T15:40:33Z,2019-06-29T06:27:39Z,2019-06-29T06:27:39Z,OWNER,simonw/sqlite-utils/pulls/32,"Refs #31. Still TODO: - [x] Unit tests - [x] Documentation",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/32/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 462423839,MDU6SXNzdWU0NjI0MjM4Mzk=,33,index_foreign_keys / index-foreign-keys utilities,9599,simonw,closed,0,,,,,2,2019-06-30T16:42:03Z,2019-06-30T23:54:11Z,2019-06-30T23:50:55Z,OWNER,,"Sometimes it's good to have indices on all columns that are foreign keys, to allow for efficient reverse lookups. This would be a useful utility: $ sqlite-utils index-foreign-keys database.db ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/33/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 462423972,MDExOlB1bGxSZXF1ZXN0MjkzMTE3MTgz,34,sqlite-utils index-foreign-keys / db.index_foreign_keys(),9599,simonw,closed,0,,,,,0,2019-06-30T16:43:40Z,2019-06-30T23:50:55Z,2019-06-30T23:50:55Z,OWNER,simonw/sqlite-utils/pulls/34,"Refs #33 - [x] `sqlite-utils index-foreign-keys` command - [x] `db.index_foreign_keys()` method - [x] unit tests - [x] documentation",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/34/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 462430920,MDU6SXNzdWU0NjI0MzA5MjA=,35,table.update(...) method,9599,simonw,closed,0,,,,,2,2019-06-30T18:06:15Z,2019-07-28T15:43:52Z,2019-07-28T15:43:52Z,OWNER,,"Spun off from #23 - this method will allow a user to update a specific row. Currently the only way to do that it is to call `.upsert({full record})` with the primary key field matching an existing record - but this does not support partial updates. ```python db[""events""].update(3, {""name"": ""Renamed""}) ``` This method only works on an existing table, so there's no need for a `pk=""id""` specifier - it can detect the primary key by looking at the table. If the primary key is compound the first argument can be a tuple: ```python db[""events_venues""].update((3, 2), {""custom_label"": ""Label""}) ``` The method can be called without the second dictionary argument. Doing this selects the row specified by the primary key (throwing an error if it does not exist) and remembers it so that chained operations can be carried out - see proposal in https://github.com/simonw/sqlite-utils/issues/23#issuecomment-507055345 ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/35/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 462817589,MDU6SXNzdWU0NjI4MTc1ODk=,36,Support compound primary keys,9599,simonw,closed,0,,,,,0,2019-07-01T17:00:07Z,2019-07-15T04:28:52Z,2019-07-15T04:28:52Z,OWNER,,"This should work: ```python table = db[""dog_breeds""].insert({ ""dog_id"": 1, ""breed_id"": 2 }, pk=(""dog_id"", ""breed_id"")) ``` Needed for m2m work in #23",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/36/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 462928038,MDU6SXNzdWU0NjI5MjgwMzg=,532,Switch setup.py to using ~= for dependencies,9599,simonw,closed,0,,,,,0,2019-07-01T21:53:48Z,2019-07-03T04:32:58Z,2019-07-03T04:32:58Z,OWNER,,"`~=` means ""compatible release"" https://www.python.org/dev/peps/pep-0440/#compatible-release See also https://stackoverflow.com/questions/39590187/in-requirements-txt-what-does-tilde-equals-mean",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/532/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 463492395,MDExOlB1bGxSZXF1ZXN0MjkzOTYyNDA1,533,"Support cleaner custom templates for rows and tables, closes #521",9599,simonw,closed,0,,,,,1,2019-07-03T00:40:18Z,2019-07-03T03:23:06Z,2019-07-03T03:23:06Z,OWNER,simonw/datasette/pulls/533,"- [x] Rename `_rows_and_columns.html` to `_table.html` - [x] Unit test - [x] Documentation",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/533/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 463531894,MDExOlB1bGxSZXF1ZXN0MjkzOTkyMzgy,535,"Added asgi_wrapper plugin hook, closes #520",9599,simonw,closed,0,,,,,0,2019-07-03T03:58:00Z,2019-07-03T04:06:26Z,2019-07-03T04:06:26Z,OWNER,simonw/datasette/pulls/535,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/535/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 463534974,MDExOlB1bGxSZXF1ZXN0MjkzOTk0NDQz,536,"Switch to ~= dependencies, closes #532",9599,simonw,closed,0,,,,,0,2019-07-03T04:12:16Z,2019-07-03T04:32:55Z,2019-07-03T04:32:55Z,OWNER,simonw/datasette/pulls/536,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/536/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 463915863,MDU6SXNzdWU0NjM5MTU4NjM=,538,Mechanism for secrets in plugin configuration,9599,simonw,closed,0,,,,,3,2019-07-03T19:23:34Z,2019-07-04T05:47:54Z,2019-07-04T05:47:54Z,OWNER,,"See https://github.com/simonw/datasette-auth-github/issues/1 We need a mechanism where by plugins can tap into ""secret"" config options without exposing them in the visible metadata.json (where plugin configs currently live, see https://datasette.readthedocs.io/en/stable/plugins.html#plugin-configuration )",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/538/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 464040911,MDExOlB1bGxSZXF1ZXN0Mjk0NDAwNDQ2,539,Secret plugin configuration options,9599,simonw,closed,0,,,,,2,2019-07-04T03:21:20Z,2019-07-04T05:36:45Z,2019-07-04T05:36:45Z,OWNER,simonw/datasette/pulls/539,Refs #538 ,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/539/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 464449570,MDU6SXNzdWU0NjQ0NDk1NzA=,540,Add a universal navigation bar which can be modified by plugins,9599,simonw,closed,0,,,,,8,2019-07-05T03:50:33Z,2019-07-06T23:13:29Z,2019-07-06T23:11:35Z,OWNER,,"Needed by https://github.com/simonw/datasette-auth-github/issues/5 We already have a navigation breadcrumbs header on some pages, I can extend that to be present on every page and make it easy to modify with custom templates. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/540/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 464779810,MDU6SXNzdWU0NjQ3Nzk4MTA=,541,Plugin hook for adding extra template context variables,9599,simonw,closed,0,,,,,2,2019-07-05T21:37:05Z,2019-07-06T00:05:59Z,2019-07-06T00:05:59Z,OWNER,,"It turns out I need this for https://github.com/simonw/datasette-auth-github/issues/5 It can be modelled on the `extra_body_script` hook: https://datasette.readthedocs.io/en/stable/plugins.html#extra-body-script-template-database-table-view-name-datasette",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/541/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 464786717,MDExOlB1bGxSZXF1ZXN0Mjk0OTkyNTc4,542,extra_template_vars plugin hook,9599,simonw,closed,0,,,,,5,2019-07-05T22:19:17Z,2019-07-06T00:05:57Z,2019-07-06T00:05:56Z,OWNER,simonw/datasette/pulls/542,Refs #541,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/542/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 465815372,MDU6SXNzdWU0NjU4MTUzNzI=,37,Experiment with type hints,9599,simonw,closed,0,,,,,6,2019-07-09T14:30:34Z,2021-08-18T21:48:57Z,2021-08-18T21:48:57Z,OWNER,,"Since it's designed to be used in Jupyter or for rapid prototyping in an IDE (and it's still pretty small) `sqlite-utils` feels like a great candidate for me to finally try out Python type hints. https://veekaybee.github.io/2019/07/08/python-type-hints/ is good. It suggests the mypy docs for getting started: https://mypy.readthedocs.io/en/latest/existing_code.html plus this tutorial: https://pymbook.readthedocs.io/en/latest/typehinting.html",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/37/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 466996584,MDExOlB1bGxSZXF1ZXN0Mjk2NzM1MzIw,557,Get tests running on Windows using Travis CI,9599,simonw,closed,0,,,,,4,2019-07-11T16:36:57Z,2021-07-10T23:39:48Z,2021-07-10T23:39:48Z,OWNER,simonw/datasette/pulls/557,Refs #511,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/557/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 467623820,MDExOlB1bGxSZXF1ZXN0Mjk3MjQzMDcz,559,Bump to uvicorn 0.8.4,9599,simonw,closed,0,,,,,0,2019-07-12T22:30:29Z,2019-07-13T22:34:58Z,2019-07-13T22:34:58Z,OWNER,simonw/datasette/pulls/559,"https://github.com/encode/uvicorn/commits/0.8.4 Query strings will now be included in log files: https://github.com/encode/uvicorn/pull/384",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/559/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 467790646,MDU6SXNzdWU0Njc3OTA2NDY=,560,CodeMirror fails to load on database page,9599,simonw,closed,0,,,,,3,2019-07-14T03:31:00Z,2019-09-03T01:03:02Z,2019-07-14T03:38:59Z,OWNER,,"It's not loading on https://latest.datasette.io/fixtures But it does load on https://latest.datasette.io/fixtures?sql=select+*+from+facetable",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/560/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 467862459,MDExOlB1bGxSZXF1ZXN0Mjk3NDEyNDY0,38,table.update() method,9599,simonw,closed,0,,,,,2,2019-07-14T17:03:49Z,2019-07-28T15:43:51Z,2019-07-28T15:43:51Z,OWNER,simonw/sqlite-utils/pulls/38,"Refs #35 Still to do: - [x] Unit tests - [x] Switch to using `.get()` - [x] Better exceptions, plus unit tests for what happens if pk does not exist - [x] Documentation - [x] Ensure compound primary keys work properly - [x] `alter=True` support",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/38/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 467864071,MDU6SXNzdWU0Njc4NjQwNzE=,39,table.get(...) method,9599,simonw,closed,0,,,,,0,2019-07-14T17:20:51Z,2019-07-15T04:28:53Z,2019-07-15T04:28:53Z,OWNER,,"Utility method for fetching a record by its primary key. Accepts a single value (for primary key / rowid tables) or a list/tuple of values (for compound primary keys, refs #36). Raises a `NotFoundError` if the record cannot be found.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/39/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 467928674,MDExOlB1bGxSZXF1ZXN0Mjk3NDU5Nzk3,40,.get() method plus support for compound primary keys,9599,simonw,closed,0,,,,,1,2019-07-15T03:43:13Z,2019-07-15T04:28:57Z,2019-07-15T04:28:52Z,OWNER,simonw/sqlite-utils/pulls/40,"- [x] Tests for the `NotFoundError` exception - [x] Documentation for `.get()` method - [x] Support `--pk` multiple times to define CLI compound primary keys - [x] Documentation for compound primary keys",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/40/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 470131537,MDU6SXNzdWU0NzAxMzE1Mzc=,41,sqlite-utils insert --tsv option,9599,simonw,closed,0,,,,,0,2019-07-19T04:27:21Z,2019-07-19T04:50:47Z,2019-07-19T04:50:47Z,OWNER,,"Right now we only support ingesting CSV, but sometimes interesting data is released as TSV. https://www.washingtonpost.com/national/2019/07/18/how-download-use-dea-pain-pills-database/ for example.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/41/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470542938,MDU6SXNzdWU0NzA1NDI5Mzg=,562,Facet by array shouldn't suggest for arrays that are not arrays-of-strings,9599,simonw,closed,0,,,,,2,2019-07-19T20:51:29Z,2019-11-01T19:42:10Z,2019-11-01T19:37:55Z,OWNER,,"It's triggering for arrays that look like this at the moment: ```json [ { ""type"": ""HKWorkoutEventTypeSegment"", ""date"": ""2019-05-21 09:43:50 -0700"", ""duration"": ""12.2780519704024"", ""durationUnit"": ""min"" }, { ""type"": ""HKWorkoutEventTypeSegment"", ""date"": ""2019-05-21 09:43:50 -0700"", ""duration"": ""19.467273102204"", ""durationUnit"": ""min"" } ] ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/562/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470637068,MDU6SXNzdWU0NzA2MzcwNjg=,1,Use XML Analyser to figure out the structure of the export XML,9599,simonw,closed,0,,,,,1,2019-07-20T05:19:02Z,2019-07-20T05:20:09Z,2019-07-20T05:20:09Z,MEMBER,,https://github.com/simonw/xml_analyser,197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470637152,MDU6SXNzdWU0NzA2MzcxNTI=,2,Import workouts,9599,simonw,closed,0,,,,,1,2019-07-20T05:20:21Z,2019-07-20T06:21:41Z,2019-07-20T06:21:41Z,MEMBER,,From #1,197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470637206,MDU6SXNzdWU0NzA2MzcyMDY=,3,Import ActivitySummary,9599,simonw,closed,0,,,,,0,2019-07-20T05:21:00Z,2019-07-20T05:58:07Z,2019-07-20T05:58:07Z,MEMBER,,"From #1 ```python 'ActivitySummary': {'attr_counts': {'activeEnergyBurned': 980, 'activeEnergyBurnedGoal': 980, 'activeEnergyBurnedUnit': 980, 'appleExerciseTime': 980, 'appleExerciseTimeGoal': 980, 'appleStandHours': 980, 'appleStandHoursGoal': 980, 'dateComponents': 980}, 'child_counts': {}, 'count': 980, 'parent_counts': {'HealthData': 980}}, ```",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470640505,MDU6SXNzdWU0NzA2NDA1MDU=,4,Import Records,9599,simonw,closed,0,,,,,1,2019-07-20T06:11:20Z,2019-07-20T06:21:41Z,2019-07-20T06:21:41Z,MEMBER,,"From #1: ```python 'Record': {'attr_counts': {'creationDate': 2672233, 'device': 2665111, 'endDate': 2672233, 'sourceName': 2672233, 'sourceVersion': 2671779, 'startDate': 2672233, 'type': 2672233, 'unit': 2650012, 'value': 2672232}, 'child_counts': {'HeartRateVariabilityMetadataList': 2318, 'MetadataEntry': 287974}, 'count': 2672233, 'parent_counts': {'Correlation': 2, 'HealthData': 2672231}}, ```",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470691622,MDU6SXNzdWU0NzA2OTE2MjI=,5,Add progress bar,9599,simonw,closed,0,,,,,2,2019-07-20T16:29:07Z,2019-07-22T03:30:13Z,2019-07-22T02:49:22Z,MEMBER,,"Showing a progress bar would be nice, using Click. The easiest way to do this would probably be be to hook it up to the length of the compressed content, and update it as this code pushes more XML bytes through the parser: https://github.com/dogsheep/healthkit-to-sqlite/blob/d64299765064501f4efdd9a0b21dbdba9ec4287f/healthkit_to_sqlite/utils.py#L6-L10",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470691999,MDU6SXNzdWU0NzA2OTE5OTk=,43,.add_column() doesn't match indentation of initial creation,9599,simonw,closed,0,,,,,3,2019-07-20T16:33:10Z,2019-07-23T13:09:11Z,2019-07-23T13:09:05Z,OWNER,,"I spotted a table which was created once and then had columns added to it and the formatted SQL looks like this: ```sql CREATE TABLE [records] ( [type] TEXT, [sourceName] TEXT, [sourceVersion] TEXT, [unit] TEXT, [creationDate] TEXT, [startDate] TEXT, [endDate] TEXT, [value] TEXT, [metadata_Health Mate App Version] TEXT, [metadata_Withings User Identifier] TEXT, [metadata_Modified Date] TEXT, [metadata_Withings Link] TEXT, [metadata_HKWasUserEntered] TEXT , [device] TEXT, [metadata_HKMetadataKeyHeartRateMotionContext] TEXT, [metadata_HKDeviceManufacturerName] TEXT, [metadata_HKMetadataKeySyncVersion] TEXT, [metadata_HKMetadataKeySyncIdentifier] TEXT, [metadata_HKSwimmingStrokeStyle] TEXT, [metadata_HKVO2MaxTestType] TEXT, [metadata_HKTimeZone] TEXT, [metadata_Average HR] TEXT, [metadata_Recharge] TEXT, [metadata_Lights] TEXT, [metadata_Asleep] TEXT, [metadata_Rating] TEXT, [metadata_Energy Threshold] TEXT, [metadata_Deep Sleep] TEXT, [metadata_Nap] TEXT, [metadata_Edit Slots] TEXT, [metadata_Tags] TEXT, [metadata_Daytime HR] TEXT) ``` It would be nice if the columns that were added later matched the indentation of the initial columns.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/43/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 470856782,MDU6SXNzdWU0NzA4NTY3ODI=,6,Break up records into different tables for each type,9599,simonw,closed,0,,,,,1,2019-07-22T01:54:59Z,2019-07-22T03:28:55Z,2019-07-22T03:28:50Z,MEMBER,,"I don't think there's much benefit to having all of the different record types stored in the same enormous table. Here's what I get when I use `_facet=type`: I'm going to try splitting these up into separate tables - so `HKQuantityTypeIdentifierBodyMassIndex` becomes a table called `rBodyMassIndex` - and see if that's nicer to work with.",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 471628483,MDU6SXNzdWU0NzE2Mjg0ODM=,44,Utilities for building lookup tables,9599,simonw,closed,0,,,,,2,2019-07-23T10:59:58Z,2019-07-23T13:07:01Z,2019-07-23T13:07:01Z,OWNER,,"While building https://github.com/dogsheep/healthkit-to-sqlite I found a need for a neat mechanism for easily building lookup tables - tables where each unique value in a column is replaced by a foreign key to a separate table. csvs-to-sqlite currently creates those with its ""extract"" mechanism - but that's written as custom code against Pandas. I'd like to eventually replace Pandas with sqlite-utils there. See also #42 ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/44/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 471684708,MDExOlB1bGxSZXF1ZXN0MzAwMjg2NTM1,45,"Implemented table.lookup(...), closes #44",9599,simonw,closed,0,,,,,0,2019-07-23T13:03:30Z,2019-07-23T13:07:00Z,2019-07-23T13:07:00Z,OWNER,simonw/sqlite-utils/pulls/45,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/45/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 471780443,MDU6SXNzdWU0NzE3ODA0NDM=,46,extracts= option for insert/update/etc,9599,simonw,closed,0,,,,,3,2019-07-23T15:55:46Z,2020-03-01T16:53:40Z,2019-07-23T17:00:44Z,OWNER,,"Relates to #42 and #44. I want the ability to extract values out into lookup tables during bulk insert/upsert operations. `db.insert_all(rows, extracts=[""species""])` - creates species table for values in the species column `db.insert_all(rows, extracts={""species"": ""Species""})` - as above but the new table is called `Species`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/46/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 471797101,MDExOlB1bGxSZXF1ZXN0MzAwMzc3NTk5,47,extracts= table parameter,9599,simonw,closed,0,,,,,0,2019-07-23T16:30:29Z,2019-07-23T17:00:43Z,2019-07-23T17:00:43Z,OWNER,simonw/sqlite-utils/pulls/47,Still needs docs. Refs #46,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/47/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 471818939,MDU6SXNzdWU0NzE4MTg5Mzk=,48,"Jupyter notebook demo of the library, launchable on Binder",9599,simonw,closed,0,,,,,2,2019-07-23T17:05:05Z,2022-01-26T02:08:46Z,2022-01-26T02:08:39Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/48/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 472097220,MDU6SXNzdWU0NzIwOTcyMjA=,7,Script uses a lot of RAM,9599,simonw,closed,0,,,,,3,2019-07-24T06:11:11Z,2019-07-24T06:35:52Z,2019-07-24T06:35:52Z,MEMBER,,"I'm using an XML pull parser which should avoid the need to slurp the whole XML file into memory, but it's not working - the script still uses over 1GB of RAM when it runs according to Activity Monitor. I think this is because I'm still causing the full root element to be incrementally loaded into memory just in case I try and access it later. http://effbot.org/elementtree/iterparse.htm says I should use `elem.clear()` as I go. It also says: > The above pattern has one drawback; it does not clear the root element, so you will end up with a single element with lots of empty child elements. If your files are huge, rather than just large, this might be a problem. To work around this, you need to get your hands on the root element. So I will try that recipe and see if it helps.",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 472104705,MDExOlB1bGxSZXF1ZXN0MzAwNTgwMjIx,8,Use less RAM,9599,simonw,closed,0,,,,,0,2019-07-24T06:35:01Z,2019-07-24T06:35:52Z,2019-07-24T06:35:52Z,MEMBER,dogsheep/healthkit-to-sqlite/pulls/8,Closes #7,197882382,healthkit-to-sqlite,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 473083260,MDU6SXNzdWU0NzMwODMyNjA=,50,"""Too many SQL variables"" on large inserts",9599,simonw,closed,0,,,,,4,2019-07-25T21:43:31Z,2022-11-04T14:38:36Z,2019-07-28T11:59:33Z,OWNER,,"Reported here: https://github.com/dogsheep/healthkit-to-sqlite/issues/9 It looks like there's a default limit of 999 variables - we need to be smart about that, maybe dynamically lower the batch size based on the number of columns.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/50/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 473733752,MDExOlB1bGxSZXF1ZXN0MzAxODI0MDk3,51,"Fix for too many SQL variables, closes #50",9599,simonw,closed,0,,,,,1,2019-07-28T11:30:30Z,2019-07-28T11:59:32Z,2019-07-28T11:59:32Z,OWNER,simonw/sqlite-utils/pulls/51,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/51/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 476413293,MDU6SXNzdWU0NzY0MTMyOTM=,52,Throws error if .insert_all() / .upsert_all() called with empty list,9599,simonw,closed,0,,,,,1,2019-08-03T04:09:00Z,2019-11-07T04:32:39Z,2019-11-07T04:32:39Z,OWNER,,See also https://github.com/simonw/db-to-sqlite/issues/18,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/52/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 476436920,MDExOlB1bGxSZXF1ZXN0MzAzOTkwNjgz,53,Work in progress: m2m() method for creating many-to-many records,9599,simonw,closed,0,,,,,0,2019-08-03T10:03:56Z,2019-08-04T03:38:10Z,2019-08-04T03:37:33Z,OWNER,simonw/sqlite-utils/pulls/53,"- [x] `table.insert({""name"": ""Barry""}).m2m(""tags"", lookup={""tag"": ""Coworker""})` - [x] Explicit table name `.m2m(""humans"", ..., m2m_table=""relationships"")` - [x] Automatically use an existing m2m table if a single obvious candidate exists (a table with two foreign keys in the correct directions) - [x] Require the explicit `m2m_table=` argument if multiple candidates for the m2m table exist - [x] Documentation Refs #23",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/53/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 476573875,MDU6SXNzdWU0NzY1NzM4NzU=,567,Datasette Edit,9599,simonw,closed,0,,,,,3,2019-08-04T17:09:28Z,2020-02-25T03:40:50Z,2020-02-25T03:40:50Z,OWNER,,"Datasette started out immutable. Then it gained the ability to run against read-only databases that were being modified by other processes. It's time for the next logical progression: the option to allow Datasette (or more likely individual plugins) to write to the database! This is going to require some careful rethinking of how connection management works.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/567/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 481887482,MDExOlB1bGxSZXF1ZXN0MzA4MjkyNDQ3,55,Ability to introspect and run queries against views,9599,simonw,closed,0,,,,,1,2019-08-17T13:40:56Z,2019-08-23T12:19:42Z,2019-08-23T12:19:42Z,OWNER,simonw/sqlite-utils/pulls/55,See #54 ,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/55/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 487598042,MDU6SXNzdWU0ODc1OTgwNDI=,1,Implement code to pull checkins from the Foursquare API,9599,simonw,closed,0,,,,,0,2019-08-30T17:40:02Z,2019-08-30T18:23:24Z,2019-08-30T18:23:24Z,MEMBER,,"The tool currently only works with a pre-prepared JSON file of checkins. When called without options, it should prompt the user to paste in a Foursquare OAuth token. The `--token=` option should work too, and should be backed up by an optional environment variable.",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 487598468,MDU6SXNzdWU0ODc1OTg0Njg=,2,--save option to dump checkins to a JSON file on disk,9599,simonw,closed,0,,,,,1,2019-08-30T17:41:06Z,2019-08-31T02:40:21Z,2019-08-31T02:40:21Z,MEMBER,,"This is a complement to the `--load` option - mainly useful for development purposes. (I'll rename `--file` to `--load` as part of this issue).",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 487600595,MDU6SXNzdWU0ODc2MDA1OTU=,3,Option to fetch only checkins more recent than the current max checkin,9599,simonw,closed,0,,,,,4,2019-08-30T17:46:45Z,2019-10-16T20:41:23Z,2019-10-16T20:39:59Z,MEMBER,,"The Foursquare checkins API supports ""return every checkin occurring after this point"" - I can pass it the maximum createdAt date currently stored in the database. This will allow for quick incremental fetches via a cron.",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 487601121,MDU6SXNzdWU0ODc2MDExMjE=,4,Online tool for getting a Foursquare OAuth token,9599,simonw,closed,0,,,,,1,2019-08-30T17:48:14Z,2019-08-31T18:07:26Z,2019-08-31T18:07:26Z,MEMBER,,"I will link to this from the documentation. See also this conversation on Twitter: https://twitter.com/simonw/status/1166822603023011840 I've decided to go with ""copy and paste in a token"" rather than hooking up a local web server that can have tokens passed to it.",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 487721884,MDU6SXNzdWU0ODc3MjE4ODQ=,5,Treat Foursquare timestamps as UTC,9599,simonw,closed,0,,,,,0,2019-08-31T02:44:47Z,2019-08-31T02:50:41Z,2019-08-31T02:50:41Z,MEMBER,,"Current test failure is due to timezone differences between my laptop and Circle CI: https://circleci.com/gh/dogsheep/swarm-to-sqlite/3 ``` E Full diff: E - [{'created': '2018-07-01T04:48:19', E ? ^ E + [{'created': '2018-07-01T02:48:19', E ? ^ E 'createdAt': 1530413299, ``` The timestamps I store in `created` should always be UTC.",205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 488338516,MDU6SXNzdWU0ODgzMzg1MTY=,570,detect_fts should handle alternative table escaping,9599,simonw,closed,0,,,,,0,2019-09-02T23:43:29Z,2019-09-03T00:32:28Z,2019-09-03T00:32:28Z,OWNER,,"sqlite-utils now uses a better way of escaping table names, which has highlighted a bug in Datasette. Datasette has its own version of the `detect_fts` function - at https://github.com/simonw/datasette/blob/d224ee2c98ac39c2c6e21a0ac0c62e5c3e1ccd11/datasette/utils/__init__.py#L466-L479 - which fails to pick up FTS tables created using the new escaping pattern. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/57#issuecomment-527258212_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/570/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 488338965,MDU6SXNzdWU0ODgzMzg5NjU=,59,Ability to introspect triggers,9599,simonw,closed,0,,,,,0,2019-09-02T23:47:16Z,2019-09-03T01:52:36Z,2019-09-03T00:09:42Z,OWNER,,"Now that we're creating triggers (thanks to @amjith in #57) it would be neat if we could introspect them too. I'm thinking: `db.triggers` - lists all triggers for the database `db[""tablename""].triggers` - lists triggers for that table The underlying query for this is `select * from sqlite_master where type = 'trigger'` I'll return the trigger information in a new namedtuple, similar to how Indexes and ForeignKeys work.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/59/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 488341021,MDExOlB1bGxSZXF1ZXN0MzEzMzgzMzE3,60,db.triggers and table.triggers introspection,9599,simonw,closed,0,,,,,0,2019-09-03T00:04:32Z,2019-09-03T00:09:42Z,2019-09-03T00:09:42Z,OWNER,simonw/sqlite-utils/pulls/60,Closes #59,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/60/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 488343304,MDExOlB1bGxSZXF1ZXN0MzEzMzg0OTI2,571,detect_fts now works with alternative table escaping,9599,simonw,closed,0,,,,,0,2019-09-03T00:23:39Z,2019-09-03T00:32:28Z,2019-09-03T00:32:28Z,OWNER,simonw/datasette/pulls/571,Fixes #570,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/571/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 488833136,MDU6SXNzdWU0ODg4MzMxMzY=,1,"Imported followers should go in ""users"", relationships in ""following""",9599,simonw,closed,0,,,,,0,2019-09-03T21:27:37Z,2019-09-04T20:23:04Z,2019-09-04T20:23:04Z,MEMBER,,"Right now `twitter-to-sqlite followers` dumps everything in a `followers` table, and doesn't actually record which account they are following! It should instead save them all in a global `users` table and then set up m2m relationships in a `following` table. This also means it should create a record for the specified user in order to record both sides of each relationship.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 488833698,MDU6SXNzdWU0ODg4MzM2OTg=,2,"""twitter-to-sqlite user-timeline"" command for pulling tweets by a specific user",9599,simonw,closed,0,,,,,3,2019-09-03T21:29:12Z,2019-09-04T20:02:11Z,2019-09-04T20:02:11Z,MEMBER,,"Twitter only allows up to 3,200 tweets to be retrieved from https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html I'm going to do: $ twitter-to-sqlite tweets simonw ",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 488833975,MDU6SXNzdWU0ODg4MzM5NzU=,3,Command for running a search and saving tweets for that search,9599,simonw,closed,0,,,,,6,2019-09-03T21:29:56Z,2019-11-04T05:31:56Z,2019-11-04T05:31:16Z,MEMBER,, $ twitter-to-sqlite search dogsheep,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 488835586,MDU6SXNzdWU0ODg4MzU1ODY=,4,Command for importing data from a Twitter Export file,9599,simonw,closed,0,,,,,2,2019-09-03T21:34:13Z,2019-10-11T06:45:02Z,2019-10-11T06:45:02Z,MEMBER,,"Twitter lets you export all of your data as an archive file: https://twitter.com/settings/your_twitter_data A command for importing this data into SQLite would be extremely useful. $ twitter-to-sqlite import twitter.db path-to-archive.zip ",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 489419782,MDU6SXNzdWU0ODk0MTk3ODI=,6,Extract extended_entities into a media table,9599,simonw,closed,0,,,,,0,2019-09-04T21:59:10Z,2019-09-04T22:08:01Z,2019-09-04T22:08:01Z,MEMBER,," ",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 489429284,MDU6SXNzdWU0ODk0MjkyODQ=,572,Error running datasette publish with just --source_url,9599,simonw,closed,0,,,,,1,2019-09-04T22:19:22Z,2019-11-13T04:28:44Z,2019-11-13T04:28:44Z,OWNER,,"``` datasette publish now cleo.db \ --source_url=""https://twitter.com/cleopaws"" \ ``` Gave me this error: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/572/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 490798130,MDU6SXNzdWU0OTA3OTgxMzA=,7,users-lookup command for fetching users,9599,simonw,closed,0,,,,,0,2019-09-08T19:47:59Z,2019-09-08T20:32:13Z,2019-09-08T20:32:13Z,MEMBER,,"https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-users-lookup ``` https://api.twitter.com/1.1/users/lookup.json?user_id=783214,6253282 https://api.twitter.com/1.1/users/lookup.json?screen_name=simonw,cleopaws ``` CLI design: ``` $ twitter-to-sqlite users-lookup simonw cleopaws $ twitter-to-sqlite users-lookup 783214 6253282 --ids ```",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 490803176,MDU6SXNzdWU0OTA4MDMxNzY=,8,--sql and --attach options for feeding commands from SQL queries,9599,simonw,closed,0,,,,,4,2019-09-08T20:35:49Z,2020-03-20T23:13:01Z,2020-03-20T23:13:01Z,MEMBER,,"Say you want to fetch Twitter profiles for a list of accounts that are stored in another database: $ twitter-to-sqlite users-lookup users.db --attach attending.db \ --sql ""select Twitter from attending.attendes where Twitter is not null"" The SQL query you feed in is expected to return a list of screen names suitable for processing further by the command. Should be supported by all three of: - [x] `twitter-to-sqlite users-lookup` - [x] `twitter-to-sqlite user-timeline` - [x] `twitter-to-sqlite followers` and `friends` The `--attach` option allows other SQLite databases to be attached to the connection. Without it the SQL query will have to read from the single attached database.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 491791152,MDU6SXNzdWU0OTE3OTExNTI=,9,followers-ids and friends-ids subcommands,9599,simonw,closed,0,,,,,1,2019-09-10T16:58:15Z,2019-09-10T17:36:55Z,2019-09-10T17:36:55Z,MEMBER,,"These will import follower and friendship IDs into the following tables, using these APIs: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-followers-ids https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friends-ids",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 492297930,MDU6SXNzdWU0OTIyOTc5MzA=,10,Rethink progress bars for various commands,9599,simonw,closed,0,,,,,5,2019-09-11T15:06:47Z,2020-04-01T03:45:48Z,2020-04-01T03:45:48Z,MEMBER,,"Progress bars and the `--silent` option are implemented inconsistently across commands at the moment. This is made more challenging by the fact that for many operations the total length is not known. https://click.palletsprojects.com/en/7.x/api/#click.progressbar",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 493599818,MDU6SXNzdWU0OTM1OTk4MTg=,1,Command for fetching starred repos,9599,simonw,closed,0,,,,,0,2019-09-14T08:36:29Z,2019-09-14T21:30:48Z,2019-09-14T21:30:48Z,MEMBER,,,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 493668862,MDU6SXNzdWU0OTM2Njg4NjI=,2,Extract licenses from repos into a separate table,9599,simonw,closed,0,,,,,0,2019-09-14T21:33:41Z,2019-09-14T21:46:58Z,2019-09-14T21:46:58Z,MEMBER,," ",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 493670426,MDU6SXNzdWU0OTM2NzA0MjY=,3,Command to fetch all repos belonging to a user or organization,9599,simonw,closed,0,,,,,2,2019-09-14T21:54:21Z,2019-09-17T00:17:53Z,2019-09-17T00:17:53Z,MEMBER,,"How about this: $ github-to-sqlite repos simonw",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 493670730,MDU6SXNzdWU0OTM2NzA3MzA=,4,Command to fetch stargazers for one or more repos,9599,simonw,closed,0,,,,,8,2019-09-14T21:58:22Z,2020-05-02T21:30:27Z,2020-05-02T21:30:27Z,MEMBER,,"Maybe this: $ github-to-sqlite stargazers github.db simonw/datasette It could accept more than one repos. Maybe have options similar to `--sql` in [twitter-to-sqlite](https://github.com/dogsheep/twitter-to-sqlite) so you can e.g. fetch all stargazers for all of the repos you have fetched into the database already (or all of the repos belonging to owner X)",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 493671014,MDU6SXNzdWU0OTM2NzEwMTQ=,5,"Add ""incomplete"" boolean to users table for incomplete profiles",9599,simonw,closed,0,,,,,2,2019-09-14T22:01:50Z,2020-03-23T19:23:31Z,2020-03-23T19:23:30Z,MEMBER,,"User profiles that are fetched from e.g. stargazers (#4) are incomplete - they have a login but they don't have name, company etc. Add a `incomplete` boolean flag to the `users` table to record this. Then later I can add a `backfill-users` command which loops through and fetches missing data for those incomplete profiles.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 502355384,MDU6SXNzdWU1MDIzNTUzODQ=,580,Testing utilities should be available to plugins,9599,simonw,closed,0,,,,,5,2019-10-03T23:58:26Z,2020-02-28T07:58:46Z,2020-02-28T07:58:46Z,OWNER,,"I'm trying to write a plugin at the moment ([datasette-atom](https://github.com/simonw/datasette-atom)) which needs to run unit tests against a full in-memory Datasette instance, in the same way that the Datasette test suite itself works. I got it working by creating copies of the [TestClient and TestResponse classes](https://github.com/simonw/datasette/blob/a314b761866d250c16f1ff6dd682010cf4181eb4/tests/fixtures.py#L22-L96) within the plugin itself: https://github.com/simonw/datasette-atom/commit/c0e3bd9556d7b31f253a8bf666d42205cd24f4fc#diff-33337525d2d877f7cc7f33737bfd2d7b I had to do this because those classes are in the `tests/` directory within Datasette, so they don't get included in the package that ships to PyPI. It would be better if these classes were included in the main package in a way that made it easy for plugins to reuse them to write their own tests.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/580/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503045221,MDU6SXNzdWU1MDMwNDUyMjE=,11,Commands for recording real-time tweets from the streaming API,9599,simonw,closed,0,,,,,1,2019-10-06T03:09:30Z,2019-10-06T04:54:17Z,2019-10-06T04:48:31Z,MEMBER,,"https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter We can support tracking keywords and following specific users.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/11/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503053800,MDU6SXNzdWU1MDMwNTM4MDA=,12,"Extract ""source"" into a separate lookup table",9599,simonw,closed,0,,,,,3,2019-10-06T05:17:23Z,2019-10-17T15:49:24Z,2019-10-17T15:49:24Z,MEMBER,,"It's pretty bulky and ugly at the moment: ",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503085013,MDU6SXNzdWU1MDMwODUwMTM=,13,statuses-lookup command,9599,simonw,closed,0,,,,,1,2019-10-06T11:00:20Z,2019-10-07T00:33:49Z,2019-10-07T00:31:44Z,MEMBER,,"For bulk retrieving tweets by their ID. https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-lookup Rate limit is 900/15 minutes (1 call per second) but each call can pull up to 100 IDs, so we can pull 6,000 per minute. Should support `--SQL` and `--attach` #8 ",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503128914,MDU6SXNzdWU1MDMxMjg5MTQ=,583,"Enable ""explain"" and ""explain query plan"" for CTEs",9599,simonw,closed,0,,,,,1,2019-10-06T17:00:10Z,2019-10-06T17:24:07Z,2019-10-06T17:24:07Z,OWNER,,"This currently throws an error: https://latest.datasette.io/fixtures?sql=explain+WITH+RECURSIVE%0D%0A++xaxis%28x%29+AS+%28VALUES%28-2.0%29+UNION+ALL+SELECT+x%2B0.05+FROM+xaxis+WHERE+x%3C1.2%29%2C%0D%0A++yaxis%28y%29+AS+%28VALUES%28-1.0%29+UNION+ALL+SELECT+y%2B0.1+FROM+yaxis+WHERE+y%3C1.0%29%2C%0D%0A++m%28iter%2C+cx%2C+cy%2C+x%2C+y%29+AS+%28%0D%0A++++SELECT+0%2C+x%2C+y%2C+0.0%2C+0.0+FROM+xaxis%2C+yaxis%0D%0A++++UNION+ALL%0D%0A++++SELECT+iter%2B1%2C+cx%2C+cy%2C+x*x-y*y+%2B+cx%2C+2.0*x*y+%2B+cy+FROM+m+%0D%0A+++++WHERE+%28x*x+%2B+y*y%29+%3C+4.0+AND+iter%3C28%0D%0A++%29%2C%0D%0A++m2%28iter%2C+cx%2C+cy%29+AS+%28%0D%0A++++SELECT+max%28iter%29%2C+cx%2C+cy+FROM+m+GROUP+BY+cx%2C+cy%0D%0A++%29%2C%0D%0A++a%28t%29+AS+%28%0D%0A++++SELECT+group_concat%28+substr%28%27+.%2B*%23%27%2C+1%2Bmin%28iter%2F7%2C4%29%2C+1%29%2C+%27%27%29+%0D%0A++++FROM+m2+GROUP+BY+cy%0D%0A++%29%0D%0ASELECT+group_concat%28rtrim%28t%29%2Cx%270a%27%29+FROM+a%3B",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/583/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503190241,MDU6SXNzdWU1MDMxOTAyNDE=,584,Codec error in some CSV exports,9599,simonw,closed,0,,,,,2,2019-10-07T01:15:34Z,2021-06-17T18:13:20Z,2019-10-18T05:23:16Z,OWNER,,"Got this exploring my Swarm checkins: ![448DBFC4-71F8-4846-83C0-BEA511B2157A](https://user-images.githubusercontent.com/9599/66279259-3af53480-e865-11e9-9651-04fd2d895392.jpeg) `/swarm/stickers.csv?stickerType=messageOnly&_size=max`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/584/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503217375,MDU6SXNzdWU1MDMyMTczNzU=,585,"Databases on index page should display in order they were passed to ""datasette serve""?",9599,simonw,closed,0,,,,,1,2019-10-07T03:42:39Z,2019-10-14T03:52:34Z,2019-10-14T03:52:34Z,OWNER,,"If you run this: datasette serve -h 127.0.0.1 -p 8000 -m phone-locations.db healthkit.db locations.db genome.db Then the index page for that Datasette instance should show the databases in the order they were specified on the command-line. Mind you when we add pagination to that page in #468 we may want to do something different here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/585/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503218205,MDU6SXNzdWU1MDMyMTgyMDU=,586,Enable browser caching for plugin statics with datasette-auth,9599,simonw,closed,0,,,,,2,2019-10-07T03:47:14Z,2019-10-07T15:46:04Z,2019-10-07T15:46:03Z,OWNER,,"An authenticated Datasette I run is seeing delays on every page load. On looking at the network inspector it turns out it's because datasette-vega is nearly 1MB and a `cache-control: private` is preventing it from being cached! This may well turn out to be a bug in `datasette-auth-github` but it's still worth tracking here because caching of static assets from plugins is very important. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/586/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503233021,MDU6SXNzdWU1MDMyMzMwMjE=,1,Use better pagination (and implement progress bar),9599,simonw,closed,0,,,,,4,2019-10-07T04:58:11Z,2020-03-27T22:13:57Z,2020-03-27T22:13:57Z,MEMBER,,"Right now we attempt to load everything at once - which caps out at 5,000 items and is really slow. We can do better by implementing pagination using count and offset.",213286752,pocket-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503234169,MDU6SXNzdWU1MDMyMzQxNjk=,2,Track and use the 'since' value,9599,simonw,closed,0,,,,,3,2019-10-07T05:02:59Z,2020-03-27T22:22:30Z,2020-03-27T22:22:30Z,MEMBER,,"Pocket says: > Whenever possible, you should use the since parameter, or count and and offset parameters when retrieving a user's list. After retrieving the list, you should store the current time (which is provided along with the list response) and pass that in the next request for the list. This way the server only needs to return a small set (changes since that time) instead of the user's entire list every time. At the bottom of https://getpocket.com/developer/docs/v3/retrieve",213286752,pocket-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 503244410,MDU6SXNzdWU1MDMyNDQ0MTA=,14,"When importing favorites, record which user favorited them",9599,simonw,closed,0,,,,,0,2019-10-07T05:45:11Z,2019-10-14T03:30:25Z,2019-10-14T03:30:25Z,MEMBER,,"This code currently just dumps them into the `tweets` table without recording who it was who had favorited them. https://github.com/dogsheep/twitter-to-sqlite/blob/436a170d74ec70903d1b4ca430c2c6b6435cdfcc/twitter_to_sqlite/cli.py#L152-L157",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 504805857,MDU6SXNzdWU1MDQ4MDU4NTc=,587,Use --platform=managed for publish cloudrun,9599,simonw,closed,0,,,,,0,2019-10-09T18:02:16Z,2019-10-17T21:51:57Z,2019-10-17T21:51:57Z,OWNER,,"Running `datasette publish cloudrun` now shows this message: > Please choose a target platform: > [1] Cloud Run (fully managed) > [2] Cloud Run on GKE > [3] a Kubernetes cluster > [4] cancel >Please enter your numeric choice: 1 > > To specify the platform yourself, pass `--platform managed`. Or, to make this the default target platform, run `gcloud config set run/platform managed`. May as well set that as a default.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/587/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 505666744,MDExOlB1bGxSZXF1ZXN0MzI3MDUxNjcz,15,"twitter-to-sqlite import command, refs #4",9599,simonw,closed,0,,,,,0,2019-10-11T06:37:14Z,2019-10-11T06:45:01Z,2019-10-11T06:45:01Z,MEMBER,dogsheep/twitter-to-sqlite/pulls/15,,206156866,twitter-to-sqlite,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/15/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 505674949,MDU6SXNzdWU1MDU2NzQ5NDk=,17,import command should empty all archive-* tables first,9599,simonw,closed,0,,,,,2,2019-10-11T06:58:43Z,2019-10-11T15:40:08Z,2019-10-11T15:40:08Z,MEMBER,,Can have a CLI option for NOT doing that.,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 505928530,MDU6SXNzdWU1MDU5Mjg1MzA=,18,Command to import home-timeline,9599,simonw,closed,0,,,,,4,2019-10-11T15:47:54Z,2019-10-11T16:51:33Z,2019-10-11T16:51:12Z,MEMBER,,"Feature request: https://twitter.com/johankj/status/1182563563136868352 > Would it be possible to save all tweets in my timeline from the last X days? I would love to see how big a percentage some users are of my daily timeline as a metric on whether I should unfollow them/move them to a list.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 506087267,MDU6SXNzdWU1MDYwODcyNjc=,19,since_id support for home-timeline,9599,simonw,closed,0,,,,,3,2019-10-11T22:48:24Z,2019-10-16T19:13:06Z,2019-10-16T19:12:46Z,MEMBER,,Currently every time you run `home-timeline` we pull all 800 available tweets. We should offer to support `since_id` (which can be provided or can be pulled directly from the database) in order to work more efficiently if this command is executed e.g. on a cron.,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 506268945,MDU6SXNzdWU1MDYyNjg5NDU=,20,--since support for various commands for refresh-by-cron,9599,simonw,closed,0,,,,,3,2019-10-13T03:40:46Z,2019-10-21T03:32:04Z,2019-10-16T19:26:11Z,MEMBER,,"I want to run a cron that updates my Twitter database every X minutes. It should be able to retrieve the following without needing to paginate through everything: - [x] Tweets I have tweeted - [x] My home timeline (see #19) - [x] Tweets I have favourited It would be nice if this could be standardized across all commands as a `--since` option.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 506276893,MDU6SXNzdWU1MDYyNzY4OTM=,7,issue-comments command for importing issue comments,9599,simonw,closed,0,,,,,1,2019-10-13T05:23:58Z,2019-10-14T14:44:12Z,2019-10-13T05:24:30Z,MEMBER,,Using this API: https://developer.github.com/v3/issues/comments/,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 506432572,MDU6SXNzdWU1MDY0MzI1NzI=,21,Fix & escapes in tweet text,9599,simonw,closed,0,,,,,1,2019-10-14T03:37:28Z,2019-10-15T18:48:16Z,2019-10-15T18:48:16Z,MEMBER,," Shouldn't be storing `&` here.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/21/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 508024032,MDU6SXNzdWU1MDgwMjQwMzI=,22,Ability to import from uncompressed archive or from specific files,9599,simonw,closed,0,,,,,0,2019-10-16T18:31:57Z,2019-10-16T18:53:36Z,2019-10-16T18:53:36Z,MEMBER,,"Currently you can only import like this: $ twitter-to-sqlite import path-to-twitter.zip It would be useful if you could import from a folder that was decompressed from that zip: $ twitter-to-sqlite import path-to-twitter/ AND from individual files within that folder - since that would allow you to e.g. selectively import certain files: $ twitter-to-sqlite import path-to-twitter/favorites.js path-to-twitter/tweets.js",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/22/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 508070977,MDU6SXNzdWU1MDgwNzA5Nzc=,597,If you have databases called foo.db and foo-bar.db you cannot visit /foo-bar,9599,simonw,closed,0,,,,,5,2019-10-16T20:07:41Z,2019-10-18T22:51:08Z,2019-10-18T22:51:08Z,OWNER,,"Weird bug I just came across. It appears that if you have one database called `foo.db` and another called `foo-bar.db` any attempts to visit `/foo-bar` will redirect to `/foo`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/597/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 508190730,MDU6SXNzdWU1MDgxOTA3MzA=,23,Extremely simple migration system,9599,simonw,closed,0,,,,,2,2019-10-17T02:13:57Z,2019-10-17T16:57:17Z,2019-10-17T16:57:17Z,MEMBER,,"Needed for #12. This is going to be an incredibly simple version of the Django migration system. * A `migrations` table, keeping track of which migrations were applied (and when) * A `migrate()` function which applies any pending migrations * A `MIGRATIONS` constant which is a list of functions to be applied The function names will be detected and used as the names of the migrations. Every time you run the CLI tool it will call the `migrate()` function before doing anything else. Needs to take into account that there might be no tables at all. As such, migration functions should sanity check that the tables they are going to work on actually exist.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/23/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 508553387,MDExOlB1bGxSZXF1ZXN0MzI5MzI0MzY4,24,Tweet source extraction and new migration system,9599,simonw,closed,0,,,,,0,2019-10-17T15:24:56Z,2019-10-17T15:49:29Z,2019-10-17T15:49:24Z,MEMBER,dogsheep/twitter-to-sqlite/pulls/24,Closes #12 and #23,206156866,twitter-to-sqlite,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/24/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 508578780,MDU6SXNzdWU1MDg1Nzg3ODA=,25,Ensure migrations don't accidentally create foreign key twice,9599,simonw,closed,0,,,,,2,2019-10-17T16:08:50Z,2019-10-17T16:56:47Z,2019-10-17T16:56:47Z,MEMBER,,"Is it possible for these lines to run against a database table that already has these foreign keys? https://github.com/dogsheep/twitter-to-sqlite/blob/c9295233f219c446fa2085cace987067488a31b9/twitter_to_sqlite/migrations.py#L21-L22",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 509267608,MDExOlB1bGxSZXF1ZXN0MzI5ODkwMzIw,599,Fix for /foo v.s. /foo-bar issue in #597,9599,simonw,closed,0,,,,,0,2019-10-18T19:22:55Z,2019-10-18T22:51:07Z,2019-10-18T22:51:07Z,OWNER,simonw/datasette/pulls/599,Refs #597,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/599/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 509339999,MDU6SXNzdWU1MDkzMzk5OTk=,600,Don't auto-format SQL on first page load,9599,simonw,closed,0,,,,,0,2019-10-18T22:36:10Z,2019-10-18T23:56:46Z,2019-10-18T23:56:46Z,OWNER,,"I've gone back and forth on this a bit, but I've decided I'm not keen on the way Datasette now automatically formats SQL when a query (or canned query) page first loads. I like having an optional ""Format SQL"" button, but applying formatting automatically means that if the user has carefully formatted their SQL to a specific style their formatting will be automatically over-ridden.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/600/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 509340359,MDExOlB1bGxSZXF1ZXN0MzI5OTQ3MTgw,601,Don't auto-format SQL on page load,9599,simonw,closed,0,,,,,5,2019-10-18T22:37:39Z,2019-10-20T02:29:49Z,2019-10-18T23:56:45Z,OWNER,simonw/datasette/pulls/601,Refs #600,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/601/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 509693773,MDU6SXNzdWU1MDk2OTM3NzM=,604,_where= parameter is not persisted in hidden form fields,9599,simonw,closed,0,,,,,3,2019-10-21T02:14:10Z,2019-10-30T19:12:38Z,2019-10-30T18:49:44Z,OWNER,,"e.g. on this page: https://v0-30.datasette.io/fixtures/roadside_attractions?_where=name%20like%20%27%museum%%27 Click the ""Apply"" button and the `_where=` parameter will be dropped.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/604/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 512218858,MDU6SXNzdWU1MTIyMTg4NTg=,606,/-/plugins shows incorrect name for plugins,9599,simonw,closed,0,,,,,3,2019-10-24T22:53:25Z,2019-11-01T05:41:04Z,2019-11-01T05:40:07Z,OWNER,,"https://fivethirtyeight.datasettes.com/-/plugins ```json [ { ""name"": ""datasette_jellyfish"", ""static"": false, ""templates"": false, ""version"": ""0.3"" }, { ""name"": ""datasette_vega"", ""static"": true, ""templates"": false, ""version"": ""0.6.2"" } ] ``` These should be shown as `datasette-jellyfish` and `datasette-vega` since those are the names on PyPI.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 513008936,MDU6SXNzdWU1MTMwMDg5MzY=,608,"Improve UI of ""datasette publish cloudrun"" to reduce chances of accidentally over-writing a service",9599,simonw,closed,0,,,,,6,2019-10-27T19:21:28Z,2019-11-08T02:51:36Z,2019-11-08T02:48:46Z,OWNER,,"The concept of a ""service"" in Cloud Run is crucial: if you deploy to the same service, you will over-write what you deployed there last! As such, I'd like to make service a required positional argument for `publish cloudrun`: datasette publish cloudrun my-service one.db two.db three.db ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/608/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 513074501,MDU6SXNzdWU1MTMwNzQ1MDE=,26,Command for importing mentions timeline,9599,simonw,closed,0,,,,,1,2019-10-28T03:14:27Z,2019-10-30T02:36:13Z,2019-10-30T02:20:47Z,MEMBER,,"https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-mentions_timeline Almost identical to home-timeline #18 but it uses `https://api.twitter.com/1.1/statuses/mentions_timeline.json` instead.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/26/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 514459062,MDU6SXNzdWU1MTQ0NTkwNjI=,27,retweets-of-me command,9599,simonw,closed,0,,,,,4,2019-10-30T07:43:01Z,2019-11-03T01:12:58Z,2019-11-03T01:12:58Z,MEMBER,,https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-retweets_of_me,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 514899195,MDExOlB1bGxSZXF1ZXN0MzM0NDQ4MjU4,609,Update to latest black,9599,simonw,closed,0,,,,,0,2019-10-30T18:42:35Z,2019-10-30T18:49:01Z,2019-10-30T18:49:01Z,OWNER,simonw/datasette/pulls/609,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/609/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 515658861,MDU6SXNzdWU1MTU2NTg4NjE=,28,Add indexes to followers table,9599,simonw,closed,0,,,,,1,2019-10-31T18:40:22Z,2019-11-09T20:15:42Z,2019-11-09T20:11:48Z,MEMBER,,`select follower_id from following where followed_id = 12497` takes over a second for me at the moment.,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/28/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 516310670,MDU6SXNzdWU1MTYzMTA2NzA=,610,Don't suggest array facet if column is only [] empty arrays,9599,simonw,closed,0,,,,,0,2019-11-01T19:42:02Z,2019-11-01T21:46:08Z,2019-11-01T21:46:08Z,OWNER,,Follow on from #562,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/610/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 516370822,MDU6SXNzdWU1MTYzNzA4MjI=,611,Static assets no longer loading for installed plugins,9599,simonw,closed,0,,,,,3,2019-11-01T22:07:00Z,2019-11-01T22:15:55Z,2019-11-01T22:15:55Z,OWNER,,"Caused by fix I made in #606 e.g. `/-/static-plugins/datasette_leaflet_geojson/datasette-leaflet-geojson.js` is a 404, but view-`/-/static-plugins/datasette-leaflet-geojson/datasette-leaflet-geojson.js` works correctly.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/611/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 516748849,MDU6SXNzdWU1MTY3NDg4NDk=,612,CSV export is broken for tables with null foreign keys,9599,simonw,closed,0,,,,,2,2019-11-02T22:52:47Z,2021-06-17T18:13:20Z,2019-11-02T23:12:53Z,OWNER,,"Following on from #406 - this CSV export appears to be broken: https://14da705.datasette.io/fixtures/foreign_key_references.csv?_labels=on&_size=max ```csv pk,foreign_key_with_label,foreign_key_with_label_label,foreign_key_with_no_label,foreign_key_with_no_label_label 1,1,hello,1,1 2,, ``` That second row should have 5 values, but it only has 4.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/612/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 516763727,MDExOlB1bGxSZXF1ZXN0MzM1OTgwMjQ2,8,"stargazers command, refs #4",9599,simonw,closed,0,,,,,5,2019-11-03T00:37:36Z,2020-05-02T20:00:27Z,2020-05-02T20:00:26Z,MEMBER,dogsheep/github-to-sqlite/pulls/8,Needs tests. Refs #4.,207052882,github-to-sqlite,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 516769276,MDU6SXNzdWU1MTY3NjkyNzY=,9,Commands do not work without an auth.json file,9599,simonw,closed,0,,,,,0,2019-11-03T01:54:28Z,2019-11-11T05:30:48Z,2019-11-11T05:30:48Z,MEMBER,,"`auth.json` is meant to be optional. If it's not provided, the tool should make heavily rate-limited unauthenticated requests. ``` $ github-to-sqlite repos .data/repos.db simonw Usage: github-to-sqlite repos [OPTIONS] DB_PATH [USERNAME] Try ""github-to-sqlite repos --help"" for help. Error: Invalid value for ""-a"" / ""--auth"": File ""auth.json"" does not exist. ```",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 516950748,MDU6SXNzdWU1MTY5NTA3NDg=,614,"Add ""not in"" filter - ?pk__notin=x,y,z",9599,simonw,closed,0,,,,,1,2019-11-04T04:07:17Z,2019-11-04T04:31:58Z,2019-11-04T04:12:00Z,OWNER,,"We have a `__in` filter at the moment: https://latest.datasette.io/fixtures/facetable?pk__in=1,2,3 Today I found myself needing the inverse, a `?pk__notin=` filter, which isn't currently supported.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/614/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 516967682,MDU6SXNzdWU1MTY5Njc2ODI=,10,Add this repos_starred view,9599,simonw,closed,0,,,,,3,2019-11-04T05:44:38Z,2020-05-02T16:37:36Z,2020-05-02T16:37:36Z,MEMBER,,"```sql create view repos_starred as select stars.starred_at, users.login, repos.* from repos join stars on repos.id = stars.repo join users on repos.owner = users.id order by starred_at desc; ```",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 517241040,MDU6SXNzdWU1MTcyNDEwNDA=,63,ensure_index() method,9599,simonw,closed,0,,,,,1,2019-11-04T15:51:22Z,2019-11-04T16:20:36Z,2019-11-04T16:20:35Z,OWNER,,"```python db[""table""].ensure_index([""col1"", ""col2""]) ``` This will do the following: - if the specified table or column does not exist, do nothing - if they exist and already have an index, do nothing - otherwise, create the index I want this for tools like [twitter-to-sqlite search](https://github.com/dogsheep/twitter-to-sqlite/blob/801c0c2daf17d8abce9dcb5d8d610410e7e25dbe/README.md#running-searches) where the `search_runs` table may or not have been created yet but, if it IS created, I want to put an index on the `hash` column.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/63/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 517451234,MDU6SXNzdWU1MTc0NTEyMzQ=,615,?_col= and ?_nocol= support for toggling columns on table view,9599,simonw,closed,0,,,,,16,2019-11-04T22:55:41Z,2021-05-27T04:26:10Z,2021-05-27T04:17:44Z,OWNER,,Split off from #292 (I guess this is a re-opening of #312).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/615/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 519032008,MDExOlB1bGxSZXF1ZXN0MzM3ODQ3NTcz,64,test_insert_upsert_all_empty_list,9599,simonw,closed,0,,,,,0,2019-11-07T04:24:45Z,2019-11-07T04:32:38Z,2019-11-07T04:32:38Z,OWNER,simonw/sqlite-utils/pulls/64,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/64/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 519038979,MDU6SXNzdWU1MTkwMzg5Nzk=,10,Failed to import workout points,9599,simonw,closed,0,,,,,4,2019-11-07T04:50:22Z,2019-11-08T01:18:37Z,2019-11-08T01:18:37Z,MEMBER,,"I just ran the script and it failed to import any `workout_points`, though it did import `workouts`.",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 519039316,MDExOlB1bGxSZXF1ZXN0MzM3ODUzMzk0,65,Release 1.12.1,9599,simonw,closed,0,,,,,0,2019-11-07T04:51:29Z,2019-11-07T04:58:48Z,2019-11-07T04:58:47Z,OWNER,simonw/sqlite-utils/pulls/65,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/65/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 519613116,MDU6SXNzdWU1MTk2MTMxMTY=,617,Refactor TableView.data() method,9599,simonw,closed,0,,,,,9,2019-11-08T01:55:41Z,2021-12-18T01:41:47Z,2021-12-11T19:17:11Z,OWNER,,"This is by far the most complex piece of Datasette - the `TableView.data()` method is over 500 lines long and is increasingly getting in the way of cleanly implementing new features (e.g. #615 and #613). Need to break it up into smaller, cleaner pieces.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/617/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 520507306,MDU6SXNzdWU1MjA1MDczMDY=,618,Mechanism for seeing indexes on a specific table,9599,simonw,closed,0,,,,,2,2019-11-09T20:10:41Z,2019-11-10T01:40:05Z,2019-11-10T01:30:25Z,OWNER,,"The only way to see the indexes that apply to a specific table at the moment is to run the following SQL manually: ```sql select * from sqlite_master where type = 'index' and tbl_name=? ``` For example: It would be good if this list of indexes was displayed in a neater way on the table page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/618/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 520508502,MDU6SXNzdWU1MjA1MDg1MDI=,31,"""friends"" command (similar to ""followers"")",9599,simonw,closed,0,,,,,2,2019-11-09T20:20:20Z,2022-09-20T05:05:03Z,2020-02-07T07:03:28Z,MEMBER,,"Current list of commands: ``` followers Save followers for specified user (defaults to... followers-ids Populate followers table with IDs of account followers friends-ids Populate followers table with IDs of account friends ``` Obvious omission here is `friends`, which would be powered by `https://api.twitter.com/1.1/friends/list.json`: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friends-list",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 520521843,MDU6SXNzdWU1MjA1MjE4NDM=,11,Command to fetch releases,9599,simonw,closed,0,,,,,0,2019-11-09T22:23:30Z,2019-11-09T22:57:00Z,2019-11-09T22:57:00Z,MEMBER,,"https://developer.github.com/v3/repos/releases/#list-releases-for-a-repository `GET /repos/:owner/:repo/releases`",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/11/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 520655983,MDU6SXNzdWU1MjA2NTU5ODM=,619,"""Invalid SQL"" page should let you edit the SQL",9599,simonw,closed,0,,,,,14,2019-11-10T20:54:12Z,2022-01-13T22:21:42Z,2021-06-02T04:15:54Z,OWNER,,"https://latest.datasette.io/fixtures?sql=select%0D%0A++*%0D%0Afrom%0D%0A++%5Bfoo%5D Would be useful if this page showed you the invalid SQL you entered so you can edit it and try again.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/619/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 520715188,MDU6SXNzdWU1MjA3MTUxODg=,622,Datasette should work with Python 3.8 (and drop compatibility with Python 3.5),9599,simonw,closed,0,,,,,4,2019-11-11T03:12:36Z,2019-11-12T05:52:49Z,2019-11-12T05:09:13Z,OWNER,,"See #595, #594, #404. The big thing holding me back from ditching Python 3.5 was glitch.com - but they now offer Python 3.7: https://support.glitch.com/t/can-you-upgrade-python-to-latest-version/7980/25?u=simonw",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/622/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 520718056,MDExOlB1bGxSZXF1ZXN0MzM5MjM2NjQ3,623,Test against Python 3.8 in Travis,9599,simonw,closed,0,,,,,2,2019-11-11T03:24:54Z,2019-11-11T03:45:35Z,2019-11-11T03:45:35Z,OWNER,simonw/datasette/pulls/623,Needed for #622,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/623/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 520728483,MDExOlB1bGxSZXF1ZXN0MzM5MjQ0ODg4,624,Bump pint to 0.9,9599,simonw,closed,0,,,,,0,2019-11-11T04:07:07Z,2019-11-11T04:19:02Z,2019-11-11T04:19:02Z,OWNER,simonw/datasette/pulls/624,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/624/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 520756546,MDU6SXNzdWU1MjA3NTY1NDY=,12,Add this view for seeing new releases,9599,simonw,closed,0,,,,,5,2019-11-11T06:00:12Z,2020-05-02T18:58:18Z,2020-05-02T18:58:17Z,MEMBER,,"```sql CREATE VIEW recent_releases AS select json_object(""label"", repos.full_name, ""href"", repos.html_url) as repo, json_object( ""href"", releases.html_url, ""label"", releases.name ) as release, substr(releases.published_at, 0, 11) as date, releases.body as body_markdown, releases.published_at from releases join repos on repos.id = releases.repo order by releases.published_at desc ```",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 521282013,MDU6SXNzdWU1MjEyODIwMTM=,626,Unit tests should fail under Python 3.8,9599,simonw,closed,0,,,,,1,2019-11-12T01:54:11Z,2019-11-12T04:31:26Z,2019-11-12T04:31:13Z,OWNER,,"The unit tests currently pass under Python 3.8. But... when you actually attempt to run Datasette you get an error: ``` ~/Dropbox/Development/datasette $ venv-py3.8.0/bin/datasette --memory -p 8855 Serve! files=() (immutables=()) on port 8855 Traceback (most recent call last): File ""venv-py3.8.0/bin/datasette"", line 11, in load_entry_point('datasette', 'console_scripts', 'datasette')() File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/click/core.py"", line 764, in __call__ return self.main(*args, **kwargs) File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/click/core.py"", line 717, in main rv = self.invoke(ctx) File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/click/core.py"", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/click/core.py"", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/click/core.py"", line 555, in invoke return callback(*args, **kwargs) File ""/Users/simonw/Dropbox/Development/datasette/datasette/cli.py"", line 365, in serve uvicorn.run(ds.app(), host=host, port=port, log_level=""info"") File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/uvicorn/main.py"", line 279, in run server.run() File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/uvicorn/main.py"", line 305, in run self.config.setup_event_loop() File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/uvicorn/config.py"", line 218, in setup_event_loop loop_setup() File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/uvicorn/loops/auto.py"", line 3, in auto_loop_setup import uvloop File ""/Users/simonw/Dropbox/Development/datasette/venv-py3.8.0/lib/python3.8/site-packages/uvloop/__init__.py"", line 7, in from .loop import Loop as __BaseLoop # NOQA File ""uvloop/includes/stdlib.pxi"", line 114, in init uvloop.loop AttributeError: module 'sys' has no attribute 'set_coroutine_wrapper' ~/Dropbox/Development/datasette $ ``` If Datasette doesn't work under Python 3.8 the tests should fail.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/626/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 521323012,MDExOlB1bGxSZXF1ZXN0MzM5NzIyNzkw,627,"Support Python 3.8, stop supporting Python 3.5",9599,simonw,closed,0,,,,,2,2019-11-12T04:36:33Z,2020-04-05T10:23:58Z,2019-11-12T05:09:12Z,OWNER,simonw/datasette/pulls/627,Refs #622,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/627/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 521329771,MDU6SXNzdWU1MjEzMjk3NzE=,628,Render jinja2 templates in async mode,9599,simonw,closed,0,,,,,2,2019-11-12T05:01:55Z,2019-11-14T23:28:09Z,2019-11-14T23:14:24Z,OWNER,,"I started playing with this in #404 and got good results but it didn't work in Python 3.5. As of #627 I don't support 3.5 any more so this can go ahead. Rendering templates in async mode will mean that template plugins can include async code... which opens the door to custom template functions that execute SQL queries!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/628/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 521335335,MDU6SXNzdWU1MjEzMzUzMzU=,629,"""datasette publish"" commands should deploy with Python 3.8",9599,simonw,closed,0,,,,,1,2019-11-12T05:22:31Z,2019-11-12T06:03:10Z,2019-11-12T06:03:10Z,OWNER,,Now that we support 3.8 (#627) `datasette publish` should always deploy using Python 3.8.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/629/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 521346800,MDExOlB1bGxSZXF1ZXN0MzM5NzQyNDMy,630,Use python:3.8 base Docker image,9599,simonw,closed,0,,,,,0,2019-11-12T06:02:37Z,2019-11-12T06:03:10Z,2019-11-12T06:03:10Z,OWNER,simonw/datasette/pulls/630,Closes #629,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/630/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 521868864,MDU6SXNzdWU1MjE4Njg4NjQ=,66,"The "".upsert()"" method is misnamed",9599,simonw,closed,0,,,,,15,2019-11-12T23:48:28Z,2019-12-31T01:30:21Z,2019-12-31T01:30:20Z,OWNER,,"This thread here is illuminating: https://stackoverflow.com/questions/3634984/insert-if-not-exists-else-update The term `UPSERT` in SQLite has a specific meaning as-of 3.24.0 (2018-06-04): https://www.sqlite.org/lang_UPSERT.html It means ""behave as an UPDATE or a no-op if the INSERT would violate a uniqueness constraint"". The syntax in 3.24.0+ looks like this (confusingly it does not use the term ""upsert""): ```sql INSERT INTO phonebook(name,phonenumber) VALUES('Alice','704-555-1212') ON CONFLICT(name) DO UPDATE SET phonenumber=excluded.phonenumber ``` Here's the problem: the `sqlite-utils` `.upsert()` and `.upsert_all()` methods don't do this. They use the following SQL: ```sql INSERT OR REPLACE INTO [{table}] ({columns}) VALUES {rows}; ``` If the record already exists, it will be entirely replaced by a new record - as opposed to updating any specified fields but leaving existing fields as they are (the behaviour of ""upsert"" in SQLite itself).",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/66/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 521995039,MDU6SXNzdWU1MjE5OTUwMzk=,632,Upgrade datasette publish Heroku runtime,9599,simonw,closed,0,,,,,2,2019-11-13T06:46:19Z,2019-11-13T16:44:07Z,2019-11-13T16:43:23Z,OWNER,,"``` Python has released a security update! Please consider upgrading to python-3.6.9 ``` https://devcenter.heroku.com/articles/python-support#supported-runtimes shows 3.8.0 is now supported.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/632/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 522334771,MDU6SXNzdWU1MjIzMzQ3NzE=,633,"Publish to Heroku is broken: ""WARNING: You must pass the application as an import string to enable 'reload' or 'workers""",9599,simonw,closed,0,,,,,3,2019-11-13T16:32:11Z,2020-04-28T20:37:50Z,2019-11-13T16:43:23Z,OWNER,,"``` 2019-11-13T16:27:59.821483+00:00 heroku[web.1]: Starting process with command `datasette serve --host 0.0.0.0 -i fixtures.db --cors --port 36817 --inspect-file inspect-data.json` 2019-11-13T16:28:01.856471+00:00 heroku[web.1]: State changed from starting to crashed 2019-11-13T16:28:01.750253+00:00 app[web.1]: Serve! files=() (immutables=('fixtures.db',)) on port 36817 2019-11-13T16:28:01.771524+00:00 app[web.1]: WARNING: You must pass the application as an import string to enable 'reload' or 'workers'. 2019-11-13T16:28:01.837839+00:00 heroku[web.1]: Process exited with status 1 ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/633/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 522352520,MDU6SXNzdWU1MjIzNTI1MjA=,634,Don't run tests twice when releasing a tag,9599,simonw,closed,0,,,,,2,2019-11-13T17:02:42Z,2020-09-15T20:37:58Z,2020-09-15T20:37:58Z,OWNER,,"Shipping a release currently runs the tests twice: https://travis-ci.org/simonw/datasette/builds/611463728 It does a regular test run on Python 3.6/7/8 - then the ""Release tagged version"" step runs the tests again before publishing to PyPI! This second run is not necessary.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/634/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 522566332,MDExOlB1bGxSZXF1ZXN0MzQwNzQzMjIw,635,Use Jinja async mode,9599,simonw,closed,0,,,,,0,2019-11-14T01:20:57Z,2019-11-14T23:14:23Z,2019-11-14T23:14:23Z,OWNER,simonw/datasette/pulls/635,Refs #628. Still needs documentation.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/635/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 525254973,MDU6SXNzdWU1MjUyNTQ5NzM=,636,rowid is not included in dropdown filter menus,9599,simonw,closed,0,,,,,3,2019-11-19T20:43:04Z,2019-11-19T23:01:17Z,2019-11-19T23:01:17Z,OWNER,,"For `rowid` tables the `rowid` column isn't shown in the list of filter options: This also means if you link to e.g. `?rowid__gt=1060124` the resulting filter interface will be slightly broken: clicking the ""apply"" button again will lose your filter for example.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/636/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 525993034,MDU6SXNzdWU1MjU5OTMwMzQ=,637,"Custom queries with 0 results should say ""0 results""",9599,simonw,closed,0,,,,,3,2019-11-20T18:28:14Z,2019-11-23T06:17:23Z,2019-11-23T06:07:08Z,OWNER,,"Consider https://latest.datasette.io/fixtures/neighborhood_search?text=foop It's currently not obvious that the query executed and returned 0 results.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/637/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 526913133,MDU6SXNzdWU1MjY5MTMxMzM=,638,Don't suggest column for faceting if all values are 1,9599,simonw,closed,0,,,,,3,2019-11-22T00:14:22Z,2019-11-22T01:14:59Z,2019-11-22T00:57:49Z,OWNER,,"https://www.niche-museums.com/museums/museums?_facet=wikipedia_url Challenge is how to do this efficiently, since suggested facet queries need to be lightning fast.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/638/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 528442126,MDU6SXNzdWU1Mjg0NDIxMjY=,641,Better documentation for --static option,9599,simonw,closed,0,,,,,1,2019-11-26T02:07:57Z,2019-11-26T03:30:02Z,2019-11-26T02:31:53Z,OWNER,,"This is misleading: https://github.com/simonw/datasette/blob/aca41618f8761f99c47c8ae8e81b07a6d4af4d7a/docs/datasette-serve-help.txt#L23 The correct format is e.g. `static:static/` Also it's not mentioned in the regular documentation at all.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/641/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 529376481,MDExOlB1bGxSZXF1ZXN0MzQ2MjY0OTI2,67,Run tests against 3.5 too,9599,simonw,closed,0,,,,,2,2019-11-27T14:20:35Z,2019-12-31T01:29:44Z,2019-12-31T01:29:43Z,OWNER,simonw/sqlite-utils/pulls/67,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/67/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 530653633,MDU6SXNzdWU1MzA2NTM2MzM=,645,Mechanism for register_output_renderer to suggest extension or not,9599,simonw,closed,0,,,,,4,2019-12-01T01:26:27Z,2020-05-28T02:22:18Z,2020-05-28T02:22:12Z,OWNER,,"[datasette-atom](https://github.com/simonw/datasette-atom) only works if the user constructs a SQL query with specific output columns (`atom_id` ,`atom_updated` etc). It would be good if the `.atom` link wasn't shown on the query/table page unless those columns were present. Right now you get a link which results in a 400 error: See also #581.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/645/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 531583658,MDU6SXNzdWU1MzE1ODM2NTg=,68,Add support for porter stemming in FTS,9599,simonw,closed,0,,,,,1,2019-12-02T22:35:52Z,2020-09-20T04:25:53Z,2020-09-20T04:25:47Z,OWNER,,FTS5 can have porter stemming enabled.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/68/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 534492501,MDU6SXNzdWU1MzQ0OTI1MDE=,648,Mechanism for adding arbitrary pages like /about,9599,simonw,closed,0,,,,,13,2019-12-08T04:55:19Z,2020-05-07T15:21:19Z,2020-04-26T18:46:45Z,OWNER,,"For www.niche-museums.com I solved this by creating an empty `about.db` database file - see https://simonwillison.net/2019/Nov/25/niche-museums/ I want a neater mechanism for this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/648/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 534530973,MDU6SXNzdWU1MzQ1MzA5NzM=,649,Reduce table counts on index page with many databases,9599,simonw,closed,0,,,,,2,2019-12-08T11:56:37Z,2020-02-29T01:08:29Z,2020-02-29T01:08:29Z,OWNER,,"Since #467 the index page has attempted to optimistically count times. My personal Dogsheep has enough connected databases and tables that the page can still take way too long to load - sometimes more than twenty seconds.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/649/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 541467590,MDU6SXNzdWU1NDE0Njc1OTA=,654,Template debug mode that outputs template context,9599,simonw,closed,0,,,,,3,2019-12-22T15:51:25Z,2019-12-22T16:13:11Z,2019-12-22T16:04:51Z,OWNER,,It would make writing templates (including custom templates) easier if there was an option to dump out the full template context - maybe `?_context=1`,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/654/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 542814756,MDU6SXNzdWU1NDI4MTQ3NTY=,71,Tests are failing due to missing FTS5,9599,simonw,closed,0,,,,,3,2019-12-27T09:41:16Z,2019-12-27T09:49:37Z,2019-12-27T09:49:37Z,OWNER,,"https://travis-ci.com/simonw/sqlite-utils/jobs/268436167 This is a recent change: 2 months ago they worked fine. I'm not sure what changed here. Maybe something to do with https://launchpad.net/~jonathonf/+archive/ubuntu/backports ?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/71/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 543738004,MDExOlB1bGxSZXF1ZXN0MzU3OTkyNTg4,72,Fixed implementation of upsert,9599,simonw,closed,0,,,,,0,2019-12-30T05:08:05Z,2019-12-30T05:29:24Z,2019-12-30T05:29:24Z,OWNER,simonw/sqlite-utils/pulls/72,Refs #66,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/72/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 557077945,MDExOlB1bGxSZXF1ZXN0MzY4NzM0NTAw,663,"-p argument for datasette package, plus tests - refs #661",9599,simonw,closed,0,,,,,1,2020-01-29T19:47:50Z,2020-01-29T22:46:43Z,2020-01-29T22:46:43Z,OWNER,simonw/datasette/pulls/663,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/663/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 557825032,MDU6SXNzdWU1NTc4MjUwMzI=,77,Ability to insert data that is transformed by a SQL function,9599,simonw,closed,0,,,,,2,2020-01-30T23:45:55Z,2022-02-05T00:04:25Z,2020-01-31T00:24:32Z,OWNER,,"I want to be able to run the equivalent of this SQL insert: ```python # Convert to ""Well Known Text"" format wkt = shape(geojson['geometry']).wkt # Insert and commit the record conn.execute(""INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))"", ( ""Wales"", wkt )) conn.commit() ``` From the Datasette SpatiaLite docs: https://datasette.readthedocs.io/en/stable/spatialite.html To do this, I need a way of telling `sqlite-utils` that a specific column should be wrapped in `GeomFromText(?, 4326)`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/77/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 557830332,MDExOlB1bGxSZXF1ZXN0MzY5MzQ4MDg0,78,"New conversions= feature, refs #77",9599,simonw,closed,0,,,,,0,2020-01-31T00:02:33Z,2020-09-22T07:48:29Z,2020-01-31T00:24:31Z,OWNER,simonw/sqlite-utils/pulls/78,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/78/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 557842245,MDU6SXNzdWU1NTc4NDIyNDU=,79,Helper methods for working with SpatiaLite,9599,simonw,closed,0,,,,,8,2020-01-31T00:39:19Z,2022-02-05T00:04:25Z,2022-02-04T05:55:11Z,OWNER,,"As demonstrated by this piece of documentation, using SpatiaLite with sqlite-utils requires a fair bit of boilerplate: https://github.com/simonw/sqlite-utils/blob/f7289174e66ae4d91d57de94bbd9d09fabf7aff4/docs/python-api.rst#L880-L909",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/79/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 557892819,MDExOlB1bGxSZXF1ZXN0MzY5Mzk0MDQz,80,on_create mechanism for after table creation,9599,simonw,closed,0,,,,,5,2020-01-31T03:38:48Z,2020-01-31T05:08:04Z,2020-01-31T05:08:04Z,OWNER,simonw/sqlite-utils/pulls/80,"I need this for `geojson-to-sqlite`, in particular https://github.com/simonw/geojson-to-sqlite/issues/6",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/80/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 558600274,MDU6SXNzdWU1NTg2MDAyNzQ=,81,"Remove .detect_column_types() from table, make it a documented API",9599,simonw,closed,0,,,,,4,2020-02-01T21:25:54Z,2020-02-01T21:55:35Z,2020-02-01T21:55:35Z,OWNER,,"I used it in `geojson-to-sqlite` here: https://github.com/simonw/geojson-to-sqlite/blob/f10e44264712dd59ae7dfa2e6fd5a904b682fb33/geojson_to_sqlite/utils.py#L45-L50 It would make more sense for this method to live on the Database rather than the Table - or even to exist as a separate utility method entirely. Then it should be documented.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/81/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 559374410,MDU6SXNzdWU1NTkzNzQ0MTA=,83,"Make db[""table""].exists a documented API",9599,simonw,closed,0,,,,,1,2020-02-03T22:31:44Z,2020-02-08T23:58:35Z,2020-02-08T23:56:23Z,OWNER,,Right now it's a static thing which might get out-of-sync with the database. It should probably be a live check. Maybe call it `.exists()` instead?,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/83/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 559522877,MDExOlB1bGxSZXF1ZXN0MzcwNjc1MDA3,664,Datasette.render_template() method,9599,simonw,closed,0,,,,,5,2020-02-04T06:53:59Z,2020-02-04T20:26:18Z,2020-02-04T20:26:18Z,OWNER,simonw/datasette/pulls/664,Refs #577,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/664/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 561454071,MDU6SXNzdWU1NjE0NTQwNzE=,32,"Documentation for "" favorites"" command",9599,simonw,closed,0,,,,,0,2020-02-07T06:50:11Z,2020-02-07T06:59:10Z,2020-02-07T06:59:10Z,MEMBER,,"It looks like I forgot to document this one in the README. https://github.com/dogsheep/twitter-to-sqlite/blob/6ebd482619bd94180e54bb7b56549c413077d329/twitter_to_sqlite/cli.py#L183-L194",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/32/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 561460274,MDU6SXNzdWU1NjE0NjAyNzQ=,84,.upsert() with hash_id throws error,9599,simonw,closed,0,,,,,0,2020-02-07T07:08:19Z,2020-02-07T07:17:11Z,2020-02-07T07:17:11Z,OWNER,,"```python db[table_name].upsert_all(rows, hash_id=""pk"") ``` This throws an error: `PrimaryKeyRequired('upsert() requires a pk')` The problem is, if you try this: ```python db[table_name].upsert_all(rows, hash_id=""pk"", pk=""pk"") ``` You get this error: `AssertionError('Use either pk= or hash_id=')` `hash_id=` should imply that `pk=` that column.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/84/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 561469252,MDExOlB1bGxSZXF1ZXN0MzcyMjczNjA4,33,Upgrade to sqlite-utils 2.2.1,9599,simonw,closed,0,,,,,1,2020-02-07T07:32:12Z,2020-03-20T19:21:42Z,2020-03-20T19:21:41Z,MEMBER,dogsheep/twitter-to-sqlite/pulls/33,,206156866,twitter-to-sqlite,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/33/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 562911863,MDU6SXNzdWU1NjI5MTE4NjM=,85,Create index doesn't work for columns containing spaces,9599,simonw,closed,0,,,,,1,2020-02-11T00:34:46Z,2020-02-11T05:13:20Z,2020-02-11T05:13:20Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/85/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 563347679,MDU6SXNzdWU1NjMzNDc2Nzk=,668,Make it easier to load SpatiaLite,9599,simonw,closed,0,,,,,2,2020-02-11T17:03:43Z,2022-01-20T21:29:41Z,2021-01-04T20:18:39Z,OWNER,,"``` $ datasette spatial.db Serve! files=('spatial.db',) (immutables=()) on port 8001 ERROR: conn=, sql = 'PRAGMA table_info(SpatialIndex);', params = None: no such module: VirtualSpatialIndex Usage: datasette serve [OPTIONS] [FILES]... Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module. Read more: https://datasette.readthedocs.io/en/latest/spatialite.html ``` This error message could sniff around in the common locations for the SpatiaLite module and output the CLI command you should use to enable it: ``` datasette spatial.db --load-extension=/usr/local/lib/mod_spatialite.dylib ``` Even better: if Datasette had a `--spatialite` option which automatically loads the extension from common locations, if it can find it.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/668/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 565041624,MDU6SXNzdWU1NjUwNDE2MjQ=,671,"datasette.add_database(name, db) and datasette.remove_database(name) methods",9599,simonw,closed,0,,,,,1,2020-02-14T01:05:48Z,2020-02-14T01:30:35Z,2020-02-14T01:30:30Z,OWNER,,"- `datasette.add_database(name, db)` - adds a new named database to the list of connected databases. `db` will be a `Database()` object, which may prove useful in the future for things like #670 and could also allow some plugins to provide in-memory SQLite databases. - `datasette.remove_database(name)` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/417#issuecomment-586047995_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/671/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 565518772,MDU6SXNzdWU1NjU1MTg3NzI=,673,Mechanism for checking if a SQLite database file is safe to open,9599,simonw,closed,0,,,,,11,2020-02-14T19:36:04Z,2020-02-14T20:13:59Z,2020-02-14T20:13:59Z,OWNER,,"Opening a SpatiaLite database file without SpatiaLite will result in errors later on. Same for database files which use custom extensions, like the Apple Photos database. I've figured out how to tell if a database is safe to open or not: ```sql select sql from sqlite_master where sql like 'CREATE VIRTUAL TABLE%'; ``` This returns the SQL definitions for virtual tables. The bit after `using` tells you what they need. Run this against a SpatiaLite database and you get the following: ```sql CREATE VIRTUAL TABLE SpatialIndex USING VirtualSpatialIndex() CREATE VIRTUAL TABLE ElementaryGeometries USING VirtualElementary() ``` Run it against an Apple Photos `photos.db` file (found with `find ~/Library | grep photos.db`) and you get this (partial list): ```sql CREATE VIRTUAL TABLE RidList_VirtualReader using RidList_VirtualReaderModule CREATE VIRTUAL TABLE Array_VirtualReader using Array_VirtualReaderModule CREATE VIRTUAL TABLE LiGlobals_VirtualBufferReader using VirtualBufferReaderModule CREATE VIRTUAL TABLE RKPlace_RTree using rtree (modelId,minLongitude,maxLongitude,minLatitude,maxLatitude) ``` For a database with FTS4 you get: ```sql CREATE VIRTUAL TABLE ""docs_fts"" USING FTS4 ( [title], [content], content=""docs"" ) ``` FTS5: ```sql CREATE VIRTUAL TABLE [FARA_All_Registrants_fts] USING FTS5 ( [Name], [Address_1], [Address_2], content=[FARA_All_Registrants] ) ``` So I can use this to figure out all of the `using` pieces and then compare them to a list of known support ones. _Originally posted by @simonw in https://github.com/simonw/datasette/pull/672#issuecomment-586441484_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/673/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 565552217,MDU6SXNzdWU1NjU1NTIyMTc=,674,Rethink how sanity checks work,9599,simonw,closed,0,,,,,5,2020-02-14T20:57:02Z,2020-03-26T17:19:23Z,2020-02-15T17:57:46Z,OWNER,,"If you specify a file to open using `files` or `-i` then Datasette should show a useful error message and fail to start. Files found by scanning a directory #672 should just be skipped. _Split off from comment by @simonw in https://github.com/simonw/datasette/issues/673#issuecomment-586455321_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/674/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 565837965,MDU6SXNzdWU1NjU4Mzc5NjU=,87,Should detect collections.OrderedDict as a regular dictionary,9599,simonw,closed,0,,,,,2,2020-02-16T02:06:34Z,2020-02-16T02:20:59Z,2020-02-16T02:20:59Z,OWNER,,"``` File ""...python3.7/site-packages/sqlite_utils/db.py"", line 292, in create_table column_type=COLUMN_TYPE_MAPPING[column_type], KeyError: ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/87/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 569237568,MDU6SXNzdWU1NjkyMzc1Njg=,677,The first time you click sort by ID it should show you results in reverse order,9599,simonw,closed,0,,,,,1,2020-02-21T23:38:50Z,2020-03-21T23:57:46Z,2020-03-21T23:57:46Z,OWNER,,"e.g. on https://latest.datasette.io/fixtures/roadside_attractions Clicking the ""pk"" column header doesn't actually do anything - it sorts by pk asc but since the page was already sorted like that nothing useful changes. The first click on a primary key column that the page is already implicitly sorted by should instead enable sort descending on that column.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/677/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 569253072,MDU6SXNzdWU1NjkyNTMwNzI=,678,prepare_connection() plugin hook should accept optional datasette argument,9599,simonw,closed,0,,,,,3,2020-02-22T00:50:26Z,2020-02-22T03:53:19Z,2020-02-22T02:28:51Z,OWNER,,"I want to build a plugin that allows users to configure certain database columns to be ""masked"" - so the `password` column on a users table is never revealed, for example. To do this, I need to use the `conn.set_authorizer()` SQLite mechanism. So the plugin needs to build off the `prepare_connection(conn)` hook. But that hook doesn't currently get passed `datasette` so it doesn't have a way of looking up its plugin configuration!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/678/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 569268612,MDU6SXNzdWU1NjkyNjg2MTI=,679,Release 0.36,9599,simonw,closed,0,,,,,2,2020-02-22T02:41:01Z,2020-02-22T03:52:13Z,2020-02-22T03:52:13Z,OWNER,,"I think we have enough changes to warrant a release - and I want to take advantage of the changes to the `prepare_connection()` plugin hook in #678 Changes since 0.35 so far: https://github.com/simonw/datasette/compare/0.35...be2265b0e811d0ac2875c2f748125c17b0f9289e - [x] Update ecosystem page - [x] Write release notes - [x] Ship the release",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/679/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 569275763,MDU6SXNzdWU1NjkyNzU3NjM=,680,Release automation: automate the bit that posts the GitHub release,9599,simonw,closed,0,,,,,5,2020-02-22T03:50:40Z,2020-09-12T18:18:50Z,2020-09-12T18:18:50Z,OWNER,,"The most manual part of [the release process](https://datasette.readthedocs.io/en/stable/contributing.html#release-process) right now is having to post a GitHub release that matches the updated changelog. This is particularly annoying because the changelog is in `.rst` while the GitHub release needs markdown - so I currently manually translate between the two. Having the release script automatically post a GitHub release at the end would be much more convenient.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/680/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 569613563,MDU6SXNzdWU1Njk2MTM1NjM=,682,Mechanism for writing to database via a queue,9599,simonw,closed,0,,,,,10,2020-02-24T03:10:07Z,2020-02-25T04:45:10Z,2020-02-25T04:45:10Z,OWNER,,"I've been mulling this over for a long time, and I have a new approach that I think is worth exploring. The catch with writing to SQLite is that it should only accept one write at a time. I'm now thinking that an easy way to manage that would be with a write queue for each database which is then read by a single dedicated write thread which manages its own writable connection.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/682/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 570327466,MDExOlB1bGxSZXF1ZXN0Mzc5Mzc4Nzgw,686,?_searchmode=raw option,9599,simonw,closed,0,,,,,0,2020-02-25T05:45:50Z,2020-02-25T05:56:09Z,2020-02-25T05:56:04Z,OWNER,simonw/datasette/pulls/686,Closes #676,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/686/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 571805300,MDU6SXNzdWU1NzE4MDUzMDA=,88,"table.disable_fts() method and ""sqlite-utils disable-fts ..."" command",9599,simonw,closed,0,,,,,5,2020-02-27T04:00:50Z,2020-02-27T04:40:44Z,2020-02-27T04:40:44Z,OWNER,,This would make it easier to iterate on the FTS configuration for a database without having to wipe and recreate the database each time.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/88/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 573088799,MDExOlB1bGxSZXF1ZXN0MzgxNjY2Nzc3,688,Don't count rows on homepage for DBs > 100MB,9599,simonw,closed,0,,,,,0,2020-02-29T01:01:06Z,2020-02-29T01:08:30Z,2020-02-29T01:08:29Z,OWNER,simonw/datasette/pulls/688,Closes #649.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/688/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 573740712,MDU6SXNzdWU1NzM3NDA3MTI=,90,Cannot .enable_fts() for columns with spaces in their names,9599,simonw,closed,0,,,,,0,2020-03-02T06:06:03Z,2020-03-02T06:10:49Z,2020-03-02T06:10:49Z,OWNER,,"``` import sqlite_utils db = sqlite_utils.Database(memory=True) db[""test""].insert({""space in name"": ""hello""}) db[""test""].enable_fts([""space in name""]) --------------------------------------------------------------------------- OperationalError Traceback (most recent call last) in ----> 1 db['test'].enable_fts([""space in name""]) /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in enable_fts(self, columns, fts_version, create_triggers) 755 ) 756 self.db.conn.executescript(sql) --> 757 self.populate_fts(columns) 758 759 if create_triggers: /usr/local/lib/python3.7/site-packages/sqlite_utils/db.py in populate_fts(self, columns) 787 table=self.name, columns="", "".join(columns) 788 ) --> 789 self.db.conn.executescript(sql) 790 return self 791 OperationalError: near ""in"": syntax error ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/90/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 574043218,MDU6SXNzdWU1NzQwNDMyMTg=,693,Variables from extra_template_vars() not exposed in _context=1,9599,simonw,closed,0,,,,,3,2020-03-02T15:14:51Z,2020-04-05T19:12:48Z,2020-04-05T19:12:48Z,OWNER,,The `_context=1` debugging mode does not show variables that should have been added to the context by the `extra_template_vars()` plugin hook.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/693/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 576582604,MDU6SXNzdWU1NzY1ODI2MDQ=,694,datasette publish cloudrun --memory option,9599,simonw,closed,0,,,,,8,2020-03-05T22:59:57Z,2020-06-23T17:10:51Z,2020-03-05T23:49:41Z,OWNER,,"Got this error deploying large (603MB) database with Cloud Run ``` X Deploying... Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revi sion might contain more information. X Creating Revision... Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revision might contain more information. . Routing traffic... ✓ Setting IAM Policy... Deployment failed ERROR: (gcloud.run.deploy) Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revision might contain more information. ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/694/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 576711589,MDU6SXNzdWU1NzY3MTE1ODk=,695,Update SQLite bundled with Docker container,9599,simonw,closed,0,,,,,7,2020-03-06T05:42:12Z,2020-03-08T23:33:23Z,2020-03-06T06:15:27Z,OWNER,,"It's 3.26.0 at the moment: https://github.com/simonw/datasette/blob/af9cd4ca64652fae262e6f7b5d201f6e0adc989b/Dockerfile#L9-L11 Most recent release is 3.31.1: https://www.sqlite.org/releaselog/3_31_1.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/695/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 577578306,MDU6SXNzdWU1Nzc1NzgzMDY=,697,index.html is not reliably loaded from a plugin,9599,simonw,closed,0,,,,,7,2020-03-08T22:37:55Z,2020-03-08T23:33:28Z,2020-03-08T23:11:27Z,OWNER,,"Lots of detail in https://github.com/simonw/datasette-search-all/issues/2 - short version is that I have a plugin with its own `index.html` template and Datasette intermittently fails to load it and uses the default `index.html` that ships with Datasette instead. Related: * #689: ""Templates considered"" comment broken in >=0.35 * #693: Variables from extra_template_vars() not exposed in _context=1 (may as well fix this while I'm in there)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/697/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 578883725,MDU6SXNzdWU1Nzg4ODM3MjU=,17,Command for importing commits,9599,simonw,closed,0,,,,,2,2020-03-10T21:55:12Z,2020-03-11T02:47:37Z,2020-03-11T02:47:37Z,MEMBER,,Using this API: https://api.github.com/repos/dogsheep/github-to-sqlite/commits,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 581339961,MDU6SXNzdWU1ODEzMzk5NjE=,92,.columns_dict doesn't work for all possible column types,9599,simonw,closed,0,,,,,7,2020-03-14T19:30:35Z,2020-03-15T18:37:43Z,2020-03-14T20:04:14Z,OWNER,,"Got this error: ``` File "".../python3.7/site-packages/sqlite_utils/db.py"", line 462, in for column in self.columns KeyError: 'REAL' ``` `.columns_dict` uses `REVERSE_COLUMN_TYPE_MAPPING`: https://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L457-L463 `REVERSE_COLUMN_TYPE_MAPPING` defines `FLOAT` not `REAL`A https://github.com/simonw/sqlite-utils/blob/43f1c6ab4e3a6b76531fb6f5447adb83d26f3971/sqlite_utils/db.py#L68-L74",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/92/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 582713554,MDU6SXNzdWU1ODI3MTM1NTQ=,700,Request object utility for handling POST form data,9599,simonw,closed,0,,,,,1,2020-03-17T02:44:59Z,2020-03-17T02:47:50Z,2020-03-17T02:47:50Z,OWNER,,"> This is also going to need me to handle POST form submissions which means I need to be able to parse the form body. I guess that will go in [datasette/utils/asgi.py](https://github.com/simonw/datasette/blob/master/datasette/utils/asgi.py). _Originally posted by @simonw in https://github.com/simonw/datasette/issues/698#issuecomment-599704264_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/700/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 585266763,MDU6SXNzdWU1ODUyNjY3NjM=,34,IndexError running user-timeline command,9599,simonw,closed,0,,,,,2,2020-03-20T18:54:08Z,2020-03-20T19:20:52Z,2020-03-20T19:20:37Z,MEMBER,,"``` $ twitter-to-sqlite user-timeline data.db --screen_name Allen_Joines Traceback (most recent call last): File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/bin/twitter-to-sqlite"", line 11, in load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')() File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 764, in __call__ return self.main(*args, **kwargs) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 717, in main rv = self.invoke(ctx) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 555, in invoke return callback(*args, **kwargs) File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py"", line 256, in user_timeline utils.save_tweets(db, chunk) File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py"", line 289, in save_tweets db[""users""].upsert(user, pk=""id"", alter=True) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1128, in upsert conversions=conversions, File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1157, in upsert_all upsert=True, File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1096, in insert_all row = list(self.rows_where(""rowid = ?"", [self.last_rowid]))[0] IndexError: list index out of range ```",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 585282212,MDU6SXNzdWU1ODUyODIyMTI=,35,twitter-to-sqlite user-timeline [screen_names] --sql / --attach,9599,simonw,closed,0,,,,,5,2020-03-20T19:26:07Z,2020-03-20T20:17:00Z,2020-03-20T20:16:35Z,MEMBER,,Split from #8.,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 585306847,MDU6SXNzdWU1ODUzMDY4NDc=,36,twitter-to-sqlite followers/friends --sql / --attach,9599,simonw,closed,0,,,,,0,2020-03-20T20:20:33Z,2020-03-20T23:12:38Z,2020-03-20T23:12:38Z,MEMBER,,"Split from #8. The `friends` and `followers` commands don't yet support `--sql` and `--attach`. (`friends-ids` and `followers-ids` do though).",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/36/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 585353598,MDU6SXNzdWU1ODUzNTM1OTg=,37,"Handle ""User not found"" error",9599,simonw,closed,0,,,,,3,2020-03-20T22:14:32Z,2020-04-17T23:43:46Z,2020-04-17T23:43:46Z,MEMBER,,"While running `user-timeline` I got this bug (because a screen name I asked for didn't exist): ``` File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py"", line 185, in transform_user user[""created_at""] = parser.parse(user[""created_at""]) KeyError: 'created_at' >>> import pdb >>> pdb.pm() > /Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py(185)transform_user() -> user[""created_at""] = parser.parse(user[""created_at""]) (Pdb) user {'errors': [{'code': 50, 'message': 'User not found.'}]} ```",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 585359363,MDU6SXNzdWU1ODUzNTkzNjM=,38,Screen name display for user-timeline is uneven,9599,simonw,closed,0,,,,,1,2020-03-20T22:30:23Z,2020-03-20T22:37:17Z,2020-03-20T22:37:17Z,MEMBER,,"``` CDPHE [####################################] 67 CHFSKy [####################################] 3216 DHSWI [####################################] 41 DPHHSMT [####################################] 742 Delaware_DHSS [####################################] 3231 DhhsNevada [####################################] 639 ``` I could format them to match the length of the longest screen name instead.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/38/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 585526292,MDU6SXNzdWU1ODU1MjYyOTI=,1,Set up full text search,9599,simonw,closed,0,,,,,1,2020-03-21T15:57:35Z,2020-03-21T19:47:46Z,2020-03-21T19:45:52Z,MEMBER,,"Should run against `title` and `text` in `items`, and `about` and `id` in `users`.",248903544,hacker-news-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 585597133,MDExOlB1bGxSZXF1ZXN0MzkxOTI0NTA5,703,WIP implementation of writable canned queries,9599,simonw,closed,0,,,,,3,2020-03-21T22:23:51Z,2020-06-03T00:08:14Z,2020-06-02T23:57:35Z,OWNER,simonw/datasette/pulls/703,Refs #698.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/703/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1, 586477757,MDU6SXNzdWU1ODY0Nzc3NTc=,94,"If column data is a mixture of integers and nulls, detected type should be INTEGER",9599,simonw,closed,0,,,,,0,2020-03-23T19:51:46Z,2020-03-23T19:57:10Z,2020-03-23T19:57:10Z,OWNER,,It looks like detected type for that case is TEXT at the moment.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/94/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 586486367,MDU6SXNzdWU1ODY0ODYzNjc=,95,Columns with only null values are no longer created in the database,9599,simonw,closed,0,,,,,0,2020-03-23T20:07:42Z,2020-03-23T20:31:15Z,2020-03-23T20:31:15Z,OWNER,,"Bug introduced in #94, and released in `2.4.3`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/95/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 586567379,MDU6SXNzdWU1ODY1NjczNzk=,22,Handle empty git repositories,9599,simonw,closed,0,,,,,0,2020-03-23T22:49:48Z,2020-03-23T23:13:11Z,2020-03-23T23:13:11Z,MEMBER,,"Got this error: ``` github_to_sqlite.utils.GitHubError: {'message': 'Git Repository is empty.', 'documentation_url': 'https://developer.github.com/v3/repos/commits/#list-commits-on-a-repository'} ``` From https://api.github.com/repos/dogsheep/beta/commits",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/22/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 587222354,MDU6SXNzdWU1ODcyMjIzNTQ=,707,"Consider configuring Jinja in Datasette() constructor, not .app()",9599,simonw,closed,0,,,,,0,2020-03-24T19:19:58Z,2020-03-27T01:12:57Z,2020-03-27T01:12:57Z,OWNER,,"Right now the following fails with an error: ```python ds = Datasette([], template_dir=""."") rendered = await ds.render_template(""index.html"") ``` The error is: ``` async def render_template( self, templates, context=None, request=None, view_name=None ): context = context or {} if isinstance(templates, Template): template = templates select_templates = [] else: if isinstance(templates, str): templates = [templates] > template = self.jinja_env.select_template(templates) E AttributeError: 'Datasette' object has no attribute 'jinja_env' ``` This is because `jinja_env` is configured in the `.app()` method, here: https://github.com/simonw/datasette/blob/a498d0fe6590f9bdbc4faf9e0dd5faeb3b06002c/datasette/app.py#L609-L633 This is a little surprising, especially now that `.render_template()` is part of the documented internals API: https://datasette.readthedocs.io/en/stable/internals.html#render-template-template-context-none-request-none Maybe this should happen in the Datasette class constructor instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/707/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 587322443,MDU6SXNzdWU1ODczMjI0NDM=,710,Remove Zeit Now v1 support,9599,simonw,closed,0,,,,,2,2020-03-24T22:39:49Z,2020-04-04T23:05:12Z,2020-04-04T23:05:12Z,OWNER,,It will remain supported as a plugin but since no-one can sign up for Docker hosting any more (for over a year now) there's no point including it in Datasette core.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/710/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 588108428,MDU6SXNzdWU1ODgxMDg0Mjg=,712,base_url doesn't entirely work for running Datasette inside Binder,9599,simonw,closed,0,,,,,12,2020-03-26T02:25:55Z,2020-03-26T15:11:49Z,2020-03-26T14:35:43Z,OWNER,,"> Thanks! I'm trying to launch Datasette from *within* a notebook using the jupyter-server-proxy and the new `base_url` parameter. While the assets load ok, and the breadcrumb navigation works, the facet links don't seem to use the `base_url`. Or have I missed something? _Originally posted by @wragge in https://github.com/simonw/datasette/issues/394#issuecomment-604166918_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/712/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 589402939,MDU6SXNzdWU1ODk0MDI5Mzk=,4,"Store authentication information as ""pocket_access_token"" etc",9599,simonw,closed,0,,,,,0,2020-03-27T20:43:22Z,2020-03-27T20:43:59Z,2020-03-27T20:43:59Z,MEMBER,,The `pocket_` prefix will mean that the same `auth.json` file can be used for other Dogsheep tools without Pocket over-riding a value set by some other tool.,213286752,pocket-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 589491711,MDU6SXNzdWU1ODk0OTE3MTE=,7,Upgrade to sqlite-utils 2.x,9599,simonw,closed,0,,,,,0,2020-03-28T02:24:51Z,2020-03-28T02:25:03Z,2020-03-28T02:25:03Z,MEMBER,,,205429375,swarm-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 590666760,MDU6SXNzdWU1OTA2NjY3NjA=,39,--since feature can be confused by retweets,9599,simonw,closed,0,,,,,11,2020-03-30T23:25:33Z,2020-04-01T03:45:16Z,2020-04-01T03:45:16Z,MEMBER,,"If you run `twitter-to-sqlite user-timeline ... --since` it's supposed to fetch Tweets those specific users tweeted since last time the command was run. It does this by seeking out the max ID of their previous tweets: https://github.com/dogsheep/twitter-to-sqlite/blob/810cb2af5a175837204389fd7f4b5721f8b325ab/twitter_to_sqlite/cli.py#L305-L311 BUT... this has a nasty flaw: if another account had retweeted one of their recent tweets the retweeted-tweet will have been loaded into the database - so we may treat that as the most recent since ID and miss a bunch of their tweets!",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 590669793,MDU6SXNzdWU1OTA2Njk3OTM=,40,Feature: record history of follower counts,9599,simonw,closed,0,,,,,5,2020-03-30T23:32:28Z,2020-04-01T04:13:05Z,2020-04-01T04:13:05Z,MEMBER,,"We currently over-write the follower count every time we import a tweet (when we import that user profile again): https://github.com/dogsheep/twitter-to-sqlite/blob/810cb2af5a175837204389fd7f4b5721f8b325ab/twitter_to_sqlite/utils.py#L293-L294 It would be neat if we noticed if that user's follower count (and maybe other counts?) had changed since we last saved them and recorded that change in a separate history table. This would be an inexpensive way of building up rough charts of follower count over time.",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 591613579,MDU6SXNzdWU1OTE2MTM1Nzk=,41,"Bug: recorded a since_id for None, None",9599,simonw,closed,0,,,,,0,2020-04-01T04:29:43Z,2020-04-01T04:31:11Z,2020-04-01T04:31:11Z,MEMBER,,"This shouldn't happen in the `since_ids` table (relates to #39): ",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/41/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 592829135,MDU6SXNzdWU1OTI4MjkxMzU=,713,Support YAML in metadata - metadata.yaml,9599,simonw,closed,0,,,,,6,2020-04-02T18:10:05Z,2020-04-02T19:36:17Z,2020-04-02T19:30:55Z,OWNER,,"I was originally going to do this with a plugin - see #357 - but the more I work with `metadata.json` the more I want it to just accept YAML as an optional alternative to JSON. The best example why is still this one: https://github.com/simonw/russian-ira-facebook-ads-datasette/blob/master/russian-ads-metadata.yaml YAML is just SO much better than JSON for multi-line strings - in particular HTML and SQL, both of which are common in `metadata.json` files.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/713/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 592844348,MDExOlB1bGxSZXF1ZXN0Mzk3NzQ5NjUz,714,--metadata accepts YAML as well as JSON,9599,simonw,closed,0,,,,,1,2020-04-02T18:36:02Z,2020-04-02T19:30:54Z,2020-04-02T19:30:54Z,OWNER,simonw/datasette/pulls/714,Refs #713. Still needs tests and documentation.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/714/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 594168758,MDU6SXNzdWU1OTQxNjg3NTg=,716,extra_template_vars() sending wrong view_name for index,9599,simonw,closed,0,,,,,8,2020-04-04T23:57:09Z,2020-04-05T20:04:08Z,2020-04-05T18:28:48Z,OWNER,,"See https://github.com/simonw/museums/issues/20#issuecomment-609103663 - at some point between 286ed286b68793532c2a38436a08343b45cfbc91 and current master (e0e7a0facfc935a835cd73c720bc46661462f0b1 today) a bug was introduced where the `extra_template_vars(request, view_name)` plugin hook started being passed `None` instead of `index` for the `view_name` parameter on the site index page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/716/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 594189527,MDU6SXNzdWU1OTQxODk1Mjc=,717,See if I can get Datasette working on Zeit Now v2,9599,simonw,closed,0,,,,,10,2020-04-05T00:56:48Z,2020-04-06T22:47:22Z,2020-04-06T22:47:21Z,OWNER,,"I thought this was impossible because AWS Lambda doesn't ship the `sqlite3` standard library module... but apparenttly that's not the case on Now v2 any more! https://now-2-python-versions-ks69olzpi.now.sh/api ``` _________________________________________________________________________________________________________________________________________________________________ / Hello from Python from a ZEIT Now Serverless Function! Version is 3.6.10 (default, Mar 10 2020, 22:54:43) \ \ [GCC 4.8.3 20140911 (Red Hat 4.8.3-9)], sqlite3 module = , sqlite3 version = [('3.7.17',)] / ----------------------------------------------------------------------------------------------------------------------------------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || ``` That's from shipping this code as `api/index.py`: ```python from http.server import BaseHTTPRequestHandler from cowpy import cow import sys try: import sqlite3 except ImportError: sqlite3 = None class handler(BaseHTTPRequestHandler): def do_GET(self): self.send_response(200) self.send_header(""Content-type"", ""text/plain"") self.end_headers() message = cow.Cowacter().milk( ""Hello from Python from a ZEIT Now Serverless Function! Version is {}, sqlite3 module = {}, sqlite3 version = {}"".format( sys.version, sqlite3, sqlite3.connect("":memory:"").execute(""select sqlite_version()"").fetchall() ) ) self.wfile.write(message.encode()) return ``` Now v2 supports ASGI so this might be possible without too much work: https://zeit.co/docs/runtimes#advanced-usage/advanced-python-usage/asynchronous-server-gateway-interface",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/717/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 597671518,MDU6SXNzdWU1OTc2NzE1MTg=,98,"Only set .last_rowid and .last_pk for single update/inserts, not for .insert_all()/.upsert_all() with multiple records",9599,simonw,closed,0,,,,,7,2020-04-10T03:19:40Z,2021-09-28T04:38:44Z,2020-04-13T03:29:15Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/98/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 598013965,MDU6SXNzdWU1OTgwMTM5NjU=,724,--plugin-secret over-rides existing metadata.json plugin config,9599,simonw,closed,0,,,,,3,2020-04-10T17:56:30Z,2020-04-16T04:58:12Z,2020-04-10T18:34:21Z,OWNER,,"This means if you use `--plugin-secret` at all (with e.g. `publish cloudrun`) any existing plugin configuration in your `metadata.json` will be ignored. https://github.com/simonw/datasette/blob/af9cd4ca64652fae262e6f7b5d201f6e0adc989b/datasette/publish/cloudrun.py#L98-L109 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/724/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 598640234,MDU6SXNzdWU1OTg2NDAyMzQ=,99,.upsert_all() should maybe error if dictionaries passed to it do not have the same keys,9599,simonw,closed,0,,,,,2,2020-04-13T03:02:25Z,2020-04-13T03:05:20Z,2020-04-13T03:05:04Z,OWNER,,"While investigating #98 I stumbled across this: ``` def test_upsert_compound_primary_key(fresh_db): table = fresh_db[""table""] table.upsert_all( [ {""species"": ""dog"", ""id"": 1, ""name"": ""Cleo"", ""age"": 4}, {""species"": ""cat"", ""id"": 1, ""name"": ""Catbag""}, ], pk=(""species"", ""id""), ) table.upsert_all( [ {""species"": ""dog"", ""id"": 1, ""age"": 5}, {""species"": ""dog"", ""id"": 2, ""name"": ""New Dog"", ""age"": 1}, ], pk=(""species"", ""id""), ) > assert [ {""species"": ""dog"", ""id"": 1, ""name"": ""Cleo"", ""age"": 5}, {""species"": ""cat"", ""id"": 1, ""name"": ""Catbag"", ""age"": None}, {""species"": ""dog"", ""id"": 2, ""name"": ""New Dog"", ""age"": 1}, ] == list(table.rows) E AssertionError: assert [{'age': 5, '...cies': 'dog'}] == [{'age': 5, '...cies': 'dog'}] E At index 0 diff: {'species': 'dog', 'id': 1, 'name': 'Cleo', 'age': 5} != {'species': 'dog', 'id': 1, 'name': None, 'age': 5} E Full diff: E - [{'age': 5, 'id': 1, 'name': 'Cleo', 'species': 'dog'}, E ? ^^^ -- E + [{'age': 5, 'id': 1, 'name': None, 'species': 'dog'}, E ? ^^^ E {'age': None, 'id': 1, 'name': 'Catbag', 'species': 'cat'}, E {'age': 1, 'id': 2, 'name': 'New Dog', 'species': 'dog'}] ``` If you run `.upsert_all()` with multiple dictionaries it doesn't quite have the effect you might expect.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/99/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 600583271,MDU6SXNzdWU2MDA1ODMyNzE=,727,Custom CSS class on body for styling canned queries,9599,simonw,closed,0,,,,,5,2020-04-15T20:57:32Z,2020-04-15T21:14:58Z,2020-04-15T21:07:50Z,OWNER,,"https://latest.datasette.io/fixtures/neighborhood_search is a canned query page. One of the templates scanned is `query-fixtures-neighborhood_search.html` BUT... the body CSS class just looks like this: ```html ``` I would be useful if that included a class that can be used to style that specific canned query page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/727/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 601265023,MDU6SXNzdWU2MDEyNjUwMjM=,25,Improvements to demo instance,9599,simonw,closed,0,,,,,1,2020-04-16T17:26:55Z,2020-04-16T18:07:12Z,2020-04-16T18:07:12Z,MEMBER,,- [x] Demo should pull issue-comments as well,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 601271612,MDU6SXNzdWU2MDEyNzE2MTI=,26,Topics are missing from repositories,9599,simonw,closed,0,,,,,2,2020-04-16T17:36:32Z,2020-04-16T17:41:11Z,2020-04-16T17:41:11Z,MEMBER,,"I'm sure this used to work, but right now repositories are fetched without their topics. https://developer.github.com/v3/repos/ says you need to send a custom `Accept` header of `application/vnd.github.mercy-preview+json` to get topics.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 601330277,MDU6SXNzdWU2MDEzMzAyNzc=,27,Repos have a big blob of JSON in the organization column,9599,simonw,closed,0,,,,,5,2020-04-16T18:43:14Z,2020-04-18T00:19:16Z,2020-04-18T00:18:52Z,MEMBER,,"e.g. https://github-to-sqlite.dogsheep.net/github/repos ![github__repos__11_rows_where_sorted_by_updated_at_descending](https://user-images.githubusercontent.com/9599/79494124-5640b980-7fd7-11ea-99a2-17ffbd82f9ce.png) This appears to be obsolete because the `owner` column already links to that record, albeit in the `users` table with `type` set to `Organization`: https://github-to-sqlite.dogsheep.net/github/users/53015001",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 601333634,MDU6SXNzdWU2MDEzMzM2MzQ=,28,Pull repository contributors,9599,simonw,closed,0,,,,,3,2020-04-16T18:46:40Z,2020-04-18T15:05:10Z,2020-04-18T15:05:10Z,MEMBER,,"https://developer.github.com/v3/repos/#list-contributors `GET /repos/:owner/:repo/contributors` Not sure if this should be a separate command or should be part of the existing `repos` command. I'm leaning towards a new `contributors` command.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 601358649,MDU6SXNzdWU2MDEzNTg2NDk=,100,"Mechanism for forcing column-type, over-riding auto-detection",9599,simonw,closed,0,,,,,3,2020-04-16T19:12:52Z,2020-04-17T23:53:32Z,2020-04-17T23:53:32Z,OWNER,,"As seen in https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614843406 - there's a problem where you insert a record with a `None` value for a column and that column is created as `TEXT` - but actually you intended it to be an `INT` (as later examples will demonstrate). Some kind of mechanism for over-riding the detected types of columns would be useful here.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/100/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 601392318,MDU6SXNzdWU2MDEzOTIzMTg=,101,README should include an example of CLI data insertion,9599,simonw,closed,0,,,,,0,2020-04-16T19:45:37Z,2020-04-17T23:59:49Z,2020-04-17T23:59:49Z,OWNER,,Maybe using `curl` from the GitHub API.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/101/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 602173589,MDU6SXNzdWU2MDIxNzM1ODk=,42,Error running user-timeline with --sql and --ids together,9599,simonw,closed,0,,,,,0,2020-04-17T19:02:06Z,2020-04-17T23:34:40Z,2020-04-17T23:34:40Z,MEMBER,,"``` $ twitter-to-sqlite user-timeline tweets.db --sql='select id from users' --ids Traceback (most recent call last): File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/bin/twitter-to-sqlite"", line 11, in load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')() File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 764, in __call__ return self.main(*args, **kwargs) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 717, in main rv = self.invoke(ctx) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 555, in invoke return callback(*args, **kwargs) File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py"", line 284, in user_timeline ""@{:"" + str(max(len(identifier) for identifier in identifiers)) + ""}"" File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py"", line 284, in ""@{:"" + str(max(len(identifier) for identifier in identifiers)) + ""}"" TypeError: object of type 'int' has no len() ``` But this DID work - casting to strings: ``` $ twitter-to-sqlite user-timeline tweets.db --sql='select """" || id from users' --ids ... this worked ... ```",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/42/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 602176870,MDU6SXNzdWU2MDIxNzY4NzA=,43,"""twitter-to-sqlite lists"" command for retrieving a user's owned lists",9599,simonw,closed,0,,,,,1,2020-04-17T19:08:59Z,2020-04-17T23:48:28Z,2020-04-17T23:30:39Z,MEMBER,,"https://developer.twitter.com/en/docs/accounts-and-users/create-manage-lists/api-reference/get-lists-ownerships `https://api.twitter.com/1.1/lists/ownerships.json `",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/43/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 602181581,MDU6SXNzdWU2MDIxODE1ODE=,44,"tweet[""source""] can be an empty string",9599,simonw,closed,0,,,,,0,2020-04-17T19:18:26Z,2020-04-17T22:01:44Z,2020-04-17T22:01:44Z,MEMBER,,"Got this excepion: ``` File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py"", line 641, in extract_and_save_source details = m.groupdict() AttributeError: 'NoneType' object has no attribute 'groupdict' ``` I traced it back to this tweet: https://twitter.com/osder/status/578712651393576960 ``` (Pdb) source_re re.compile('.*?)"".*?>(?P.*?)') (Pdb) locals()['source'] '' (Pdb) u > /Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py(393)save_tweets() -> tweet[""source""] = extract_and_save_source(db, tweet[""source""]) (Pdb) tweet {'created_at': '2015-03-20T00:20:22+00:00', 'id': 578712651393576960, 'full_text': '@osder', 'truncated': False, 'display_text_range': [0, 6], 'source': '', 'in_reply_to_status_id': 578712521382715392, 'in_reply_to_user_id': 1545741, 'in_reply_to_screen_name': 'osder', 'geo': None, 'coordinates': None, 'place': None, 'contributors': None, 'is_quote_status': False, 'retweet_count': 0, 'favorite_count': 0, 'favorited': False, 'retweeted': False, 'lang': 'und', 'user': 1545741} ```",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/44/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 602551638,MDU6SXNzdWU2MDI1NTE2Mzg=,5,photos-to-sqlite s3-auth command,9599,simonw,closed,0,,,,,1,2020-04-18T21:05:25Z,2020-04-18T21:08:44Z,2020-04-18T21:08:44Z,MEMBER,,Modeled on `github-to-sqlite auth` - prompts the user for their S3 credentials and saves them to `auth.json`.,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 602569315,MDU6SXNzdWU2MDI1NjkzMTU=,102,Can't store an array or dictionary containing a bytes value,9599,simonw,closed,0,,,,,0,2020-04-18T22:49:21Z,2020-05-01T20:45:45Z,2020-05-01T20:45:45Z,OWNER,,"``` In [1]: import sqlite_utils In [2]: db = sqlite_utils.Database(memory=True) In [3]: db[""t""].insert({""id"": 1, ""data"": {""foo"": b""bytes""}}) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in ----> 1 db[""t""].insert({""id"": 1, ""data"": {""foo"": b""bytes""}}) ~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert(self, record, pk, foreign_keys, column_order, not_null, defaults, hash_id, alter, ignore, replace, extracts, conversions, columns) 950 extracts=extracts, 951 conversions=conversions, --> 952 columns=columns, 953 ) 954 ~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, extracts, conversions, columns, upsert) 1052 for key in all_columns: 1053 value = jsonify_if_needed( -> 1054 record.get(key, None if key != hash_id else _hash(record)) 1055 ) 1056 if key in extracts: ~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in jsonify_if_needed(value) 1318 def jsonify_if_needed(value): 1319 if isinstance(value, (dict, list, tuple)): -> 1320 return json.dumps(value) 1321 elif isinstance(value, (datetime.time, datetime.date, datetime.datetime)): 1322 return value.isoformat() /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw) 229 cls is None and indent is None and separators is None and 230 default is None and not sort_keys and not kw): --> 231 return _default_encoder.encode(obj) 232 if cls is None: 233 cls = JSONEncoder /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in encode(self, o) 197 # exceptions aren't as detailed. The list call should be roughly 198 # equivalent to the PySequence_Fast that ''.join() would do. --> 199 chunks = self.iterencode(o, _one_shot=True) 200 if not isinstance(chunks, (list, tuple)): 201 chunks = list(chunks) /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in iterencode(self, o, _one_shot) 255 self.key_separator, self.item_separator, self.sort_keys, 256 self.skipkeys, _one_shot) --> 257 return _iterencode(o, 0) 258 259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr, /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in default(self, o) 177 178 """""" --> 179 raise TypeError(f'Object of type {o.__class__.__name__} ' 180 f'is not JSON serializable') 181 TypeError: Object of type bytes is not JSON serializable ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/102/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 602575575,MDU6SXNzdWU2MDI1NzU1NzU=,6,Add progress bar to upload command,9599,simonw,closed,0,,,,,2,2020-04-18T23:32:41Z,2020-04-19T00:15:24Z,2020-04-19T00:15:24Z,MEMBER,,Upload was added in #4 ,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 603295970,MDU6SXNzdWU2MDMyOTU5NzA=,729,Visually distinguish integer and text columns,9599,simonw,closed,0,,,,,8,2020-04-20T14:47:26Z,2020-05-18T17:20:02Z,2020-05-15T18:16:56Z,OWNER,,It would be useful if I could tell from looking at the table page if a column was a integer or a text (or a float I guess?). This is particularly important for knowing if it safe to sort by that column.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/729/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 603617013,MDU6SXNzdWU2MDM2MTcwMTM=,29,Milestones should have foreign key to creator and repo,9599,simonw,closed,0,,,,,1,2020-04-21T00:20:44Z,2020-04-21T00:43:58Z,2020-04-21T00:43:58Z,MEMBER,,"https://github-to-sqlite.dogsheep.net/github/milestones Creator is an integer but not a foreign key to users Repo is missing entirely!",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/29/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 603618244,MDU6SXNzdWU2MDM2MTgyNDQ=,30,Issues milestone column is the wrong type,9599,simonw,closed,0,,,,,2,2020-04-21T00:24:34Z,2020-04-21T00:45:23Z,2020-04-21T00:36:22Z,MEMBER,,"https://github-to-sqlite.dogsheep.net/github/issues?milestone=2857392 ![2A4C1185-2434-4F29-9EA0-3246E2F03F77](https://user-images.githubusercontent.com/9599/79811760-b7e08b00-832b-11ea-9ad7-684a6ae097a6.jpeg) It is TEXT when it should be an INTEGER - which is why the foreign key label is not correctly displayed.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/30/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 603624862,MDU6SXNzdWU2MDM2MjQ4NjI=,31,Issue and milestone should have foreign key to repo,9599,simonw,closed,0,,,,,3,2020-04-21T00:46:24Z,2020-04-22T01:20:19Z,2020-04-22T01:20:19Z,MEMBER,,"Currently the `repo` column on those tables is a string `simonw/datasette` rather than an ID referencing a row in `repos`. _Originally posted by @simonw in https://github.com/dogsheep/github-to-sqlite/issues/29#issuecomment-616883275_",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 604222295,MDU6SXNzdWU2MDQyMjIyOTU=,32,Issue comments don't appear to populate issues foreign key,9599,simonw,closed,0,,,,,3,2020-04-21T19:17:32Z,2020-04-22T01:17:44Z,2020-04-22T01:17:44Z,MEMBER,,"https://github-to-sqlite.dogsheep.net/github?sql=select+html_url%2C+id%2C+issue+from+issue_comments+order+by+updated_at+desc+limit+101 ",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/32/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 605110015,MDU6SXNzdWU2MDUxMTAwMTU=,731,Option to automatically configure based on directory layout,9599,simonw,closed,0,,,,,9,2020-04-22T22:17:47Z,2020-04-27T16:32:44Z,2020-04-27T16:30:26Z,OWNER,,"My Datasette projects increasingly take on the following structure: - `metadata.json` with the metadata - One or more `something.db` database files - A `templates/` folder with some custom templates - A `plugins/` folder with some custom plugins Then I have to run Datasette like this: datasette *.db -m metadata.json --template-dir=templates --plugins-dir=plugins It would be really interesting if Datasette had a special mode where you could point it at a directory with the above layout and it would automatically configure itself based on the contents. Maybe even allow `datasette serve` to detect if it was passed a single argument that's a directory, not a file, and kick in to ""directory layout configuration mode"" in that case: datasette . ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/731/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 605147638,MDU6SXNzdWU2MDUxNDc2Mzg=,8,Should I have used MD5 instead of SHA256?,9599,simonw,closed,0,,,,,2,2020-04-23T00:02:08Z,2020-04-23T00:03:35Z,2020-04-23T00:03:35Z,MEMBER,,"https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html > Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-S3 or plaintext, have ETags that are an MD5 digest of their object data. ",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 605938063,MDU6SXNzdWU2MDU5MzgwNjM=,9,"upload command should be resumable, should only upload photos not already uploaded",9599,simonw,closed,0,,,,,2,2020-04-23T23:31:08Z,2020-04-23T23:39:14Z,2020-04-23T23:39:14Z,MEMBER,,Follow on from #4. ,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 606028272,MDU6SXNzdWU2MDYwMjgyNzI=,10,Speed up hashing step using threads,9599,simonw,closed,0,,,,,0,2020-04-24T04:20:08Z,2020-04-24T04:32:35Z,2020-04-24T04:32:35Z,MEMBER,,"This TODO from the code: https://github.com/dogsheep/photos-to-sqlite/blob/2e7f2c67cc18b02c75bb64992a05b0196e507252/photos_to_sqlite/cli.py#L82-L90",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 606032950,MDU6SXNzdWU2MDYwMzI5NTA=,11,Try running S3 uploads in a thread pool,9599,simonw,closed,0,,,,,0,2020-04-24T04:34:31Z,2020-04-24T16:45:41Z,2020-04-24T16:45:41Z,MEMBER,,"Since #10 provided such a speedup, can the same thing be done for the actual uploads? http://ls.pwd.io/2013/06/parallel-s3-uploads-using-boto-and-threads-in-python/ suggests it can really help performance.",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/11/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 607067303,MDExOlB1bGxSZXF1ZXN0NDA5MTIzODk3,737,"Custom pages mechanism, refs #648",9599,simonw,closed,0,,,,,4,2020-04-26T17:31:41Z,2020-04-26T18:46:43Z,2020-04-26T18:46:43Z,OWNER,simonw/datasette/pulls/737,"Refs #648. TODO: - [x] Pass a `view_name` to `render_template()` - [x] Mechanism for custom status code / headers / redirect - [x] Documentation",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/737/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 607086780,MDU6SXNzdWU2MDcwODY3ODA=,738,Pass a request object to custom page templates,9599,simonw,closed,0,,,,,1,2020-04-26T18:57:48Z,2020-04-26T19:01:54Z,2020-04-26T19:01:54Z,OWNER,,"Follow-up to #648. I'm not passing a request object to `.render_template()` at the moment, which breaks any other custom plugins using e.g. `extra_template_vars()` that were expecting to be able to access the request.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/738/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 607107849,MDExOlB1bGxSZXF1ZXN0NDA5MTUzODcw,739,Configuration directory mode,9599,simonw,closed,0,,,,,3,2020-04-26T20:37:46Z,2020-04-27T16:30:25Z,2020-04-27T16:30:25Z,OWNER,simonw/datasette/pulls/739,"Refs #731 TODO: - [x] Decide how to combine explicit command-line options with items detected from the directory structure - [x] Add unit tests - [x] Implement `inspect-data.json` mechanism for populating `immutables` - [x] Add documentation",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/739/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 607211058,MDU6SXNzdWU2MDcyMTEwNTg=,740,Don't throw 500 error on attempted directory browse,9599,simonw,closed,0,,,,,1,2020-04-27T03:50:11Z,2020-04-27T18:29:15Z,2020-04-27T18:29:15Z,OWNER,," This should be a 403 error instead, because the `--static` mechanism doesn't allow directory browsing.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/740/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 607243940,MDU6SXNzdWU2MDcyNDM5NDA=,742,"Speed up tests with scope=""session""?",9599,simonw,closed,0,,,,,1,2020-04-27T05:23:54Z,2020-04-27T18:24:53Z,2020-04-27T18:24:53Z,OWNER,,"Tests are pretty slow - could I speed them up with pytest `scope=""session""` on some of the fixtures? Eg https://travis-ci.org/github/simonw/datasette/jobs/679940036 ran 452 tests in 3m53s - the `test_html` ones seem particularly slow.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/742/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 607770595,MDU6SXNzdWU2MDc3NzA1OTU=,743,escape_fts() does not correctly escape * wildcards,9599,simonw,closed,0,,,,,4,2020-04-27T18:48:53Z,2020-04-27T19:11:30Z,2020-04-27T19:11:01Z,OWNER,,"Spotted in #732. This should not return any results... but it does: https://latest.datasette.io/fixtures/searchable?_search=bar%2A&_trace=1 The query from trace is: ``` ""sql"": ""select count(*) from searchable where rowid in (select rowid from searchable_fts where searchable_fts match escape_fts(:search))"", ""params"": { ""search"": ""bar*"" } ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/743/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 608613033,MDU6SXNzdWU2MDg2MTMwMzM=,745,Extract the hash-URL mechanism out into a plugin,9599,simonw,closed,0,,,,,2,2020-04-28T21:00:38Z,2020-10-23T19:47:18Z,2020-10-23T19:47:10Z,OWNER,,"0.28 in May 2019 made this feature not-the-default: https://datasette.readthedocs.io/en/stable/changelog.html#v0-28 - see #418 I've not felt the need to use it myself since. I think I should move it into a plugin.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/745/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 608752766,MDExOlB1bGxSZXF1ZXN0NDEwNDY5Mjcy,746,"shutil.Error, not OSError",9599,simonw,closed,0,,,,,1,2020-04-29T03:30:51Z,2020-04-29T07:07:24Z,2020-04-29T07:07:23Z,OWNER,simonw/datasette/pulls/746,Refs #744,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/746/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 610192152,MDU6SXNzdWU2MTAxOTIxNTI=,747,Directory configuration mode should support metadata.yaml,9599,simonw,closed,0,,,,,4,2020-04-30T16:05:30Z,2020-04-30T19:04:19Z,2020-04-30T19:04:19Z,OWNER,,Refs #739 - `metadata.yml` or `metadata.yaml` should be detected in the same way as `metadata.json` is.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/747/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610284471,MDU6SXNzdWU2MTAyODQ0NzE=,46,Error running 'search' for the first time,9599,simonw,closed,0,,,,,0,2020-04-30T18:11:20Z,2020-04-30T18:11:58Z,2020-04-30T18:11:58Z,MEMBER,,"``` % twitter-to-sqlite search infodemic.db '#infodemic' Traceback (most recent call last): File ""/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/bin/twitter-to-sqlite"", line 11, in load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')() File ""/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py"", line 829, in __call__ return self.main(*args, **kwargs) File ""/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py"", line 782, in main rv = self.invoke(ctx) File ""/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py"", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py"", line 610, in invoke return callback(*args, **kwargs) File ""/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py"", line 867, in search for tweet in tweets: File ""/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py"", line 165, in fetch_timeline [since_type_id, since_key], sqlite3.OperationalError: no such table: since_ids ```",206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/46/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610342575,MDU6SXNzdWU2MTAzNDI1NzU=,748,?_searchmode=raw should be documented on full-text search page,9599,simonw,closed,0,,,,,0,2020-04-30T19:50:06Z,2020-04-30T21:06:12Z,2020-04-30T21:06:12Z,OWNER,,"It's currently documented here: https://datasette.readthedocs.io/en/stable/json_api.html#special-table-arguments But it should also be described here: https://datasette.readthedocs.io/en/stable/full_text_search.html#the-table-view-api",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/748/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610408908,MDU6SXNzdWU2MTA0MDg5MDg=,34,Command for retrieving dependents for a repo,9599,simonw,closed,0,,,,,6,2020-04-30T21:47:51Z,2020-05-03T15:53:01Z,2020-05-03T15:53:01Z,MEMBER,,"I really, really want to start grabbing this data: https://github.com/simonw/datasette/network/dependents",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610511450,MDU6SXNzdWU2MTA1MTE0NTA=,35,Create index on issue_comments(user) and other foreign keys,9599,simonw,closed,0,,,,,3,2020-05-01T02:06:56Z,2020-05-02T18:26:24Z,2020-05-02T18:26:24Z,MEMBER,,"``` create index issue_comments_user on issue_comments(user) ``` I'm sure there are other user columns that could benefit from an index.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610829227,MDU6SXNzdWU2MTA4MjkyMjc=,749,Cloud Run fails to serve database files larger than 32MB,9599,simonw,closed,0,,,,,4,2020-05-01T16:06:46Z,2020-12-03T00:31:15Z,2020-12-03T00:31:14Z,OWNER,,"https://cloud.google.com/run/quotas lists the maximum response size as 32MB. I spotted a bug where attempting to download a database file larger than that from a Cloud Run deployment (in this case it was https://github-to-sqlite.dogsheep.net/github.db after I [accidentally increased the size of that database](https://github.com/dogsheep/github-to-sqlite/commit/630bdba68a23c0ac453e015518ef0bf41107a952)) returned a 500 error because of this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/749/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610842926,MDU6SXNzdWU2MTA4NDI5MjY=,36,Add view for better display of dependent repos,9599,simonw,closed,0,,,,,2,2020-05-01T16:33:44Z,2020-05-02T16:50:31Z,2020-05-02T16:30:11Z,MEMBER,,"```sql select repos.full_name as repo, 'https://github.com/' || repos2.full_name as dependent, repos2.created_at as dependent_repo_created, repos2.updated_at as dependent_repo_updated, repos2.stargazers_count as dependent_repo_stars, repos2.watchers_count as dependent_repo_watchers from dependents join repos as repos2 on dependents.dependent = repos2.id join repos on dependents.repo = repos.id order by repos2.created_at desc ``` https://dogsheep.simonwillison.net/github?sql=select%0D%0A++repos.full_name+as+repo%2C%0D%0A++%27https%3A%2F%2Fgithub.com%2F%27+%7C%7C+repos2.full_name+as+dependent%2C%0D%0A++repos2.created_at+as+dependent_repo_created%2C%0D%0A++repos2.updated_at+as+dependent_repo_updated%2C%0D%0A++repos2.stargazers_count+as+dependent_repo_stars%2C%0D%0A++repos2.watchers_count+as+dependent_repo_watchers%0D%0Afrom%0D%0A++dependents%0D%0A++join+repos+as+repos2+on+dependents.dependent+%3D+repos2.id%0D%0A++join+repos+on+dependents.repo+%3D+repos.id%0D%0Aorder+by%0D%0A++repos2.created_at+desc",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/36/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610843136,MDU6SXNzdWU2MTA4NDMxMzY=,37,Mechanism for creating views if they don't yet exist,9599,simonw,closed,0,,,,,3,2020-05-01T16:34:10Z,2020-05-02T16:19:47Z,2020-05-02T16:19:31Z,MEMBER,,Needed for #36 #10 #12 ,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610853393,MDU6SXNzdWU2MTA4NTMzOTM=,104,"--schema option to ""sqlite-utils tables""",9599,simonw,closed,0,,,,,0,2020-05-01T16:55:49Z,2020-05-01T17:12:37Z,2020-05-01T17:12:37Z,OWNER,,Adds output showing the table schema.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/104/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 610853576,MDU6SXNzdWU2MTA4NTM1NzY=,105,"""sqlite-utils views"" command",9599,simonw,closed,0,,,,,1,2020-05-01T16:56:11Z,2020-05-01T20:40:07Z,2020-05-01T20:38:36Z,OWNER,,Similar to `sqlite-utils tables`. See also #104.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/105/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 611216862,MDU6SXNzdWU2MTEyMTY4NjI=,106,"create_view(..., ignore=True, replace=True) parameters",9599,simonw,closed,0,,,,,1,2020-05-02T15:45:21Z,2020-05-02T16:04:51Z,2020-05-02T16:02:10Z,OWNER,,"Two new parameters which specify what should happen if the view already exists. I want this for https://github.com/dogsheep/github-to-sqlite/issues/37 Here's the current `create_view()` implementation: https://github.com/simonw/sqlite-utils/blob/b4d953d3ccef28bb81cea40ca165a647b59971fa/sqlite_utils/db.py#L325-L332 `ignore=True` will not do anything if the view exists already. `replace=True` will drop and redefine the view - but only if its SQL definition differs, otherwise it will be left alone.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/106/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 611222968,MDU6SXNzdWU2MTEyMjI5Njg=,107,sqlite-utils create-view CLI command,9599,simonw,closed,0,,,,,2,2020-05-02T16:15:13Z,2020-05-03T15:36:58Z,2020-05-03T15:36:37Z,OWNER,,Can go with #27 - `sqlite-utils create-table`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/107/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 611252244,MDU6SXNzdWU2MTEyNTIyNDQ=,750,Add notlike table filter,9599,simonw,closed,0,,,,,3,2020-05-02T18:54:36Z,2020-05-02T19:10:44Z,2020-05-02T19:10:44Z,OWNER,,"I found myself wanting that for applying the opposite of this: https://github-to-sqlite.dogsheep.net/github/dependent_repos?dependent__like=%25simonw%2F%25&_sort_desc=dependent_stars ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/750/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 611326701,MDU6SXNzdWU2MTEzMjY3MDE=,108,Documentation unit tests for CLI commands,9599,simonw,closed,0,,,,,2,2020-05-03T03:58:42Z,2020-05-03T04:13:57Z,2020-05-03T04:13:57Z,OWNER,,Have a test that ensures all CLI commands are documented.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/108/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 611997130,MDU6SXNzdWU2MTE5OTcxMzA=,754,Clean up aiofiles warnings on 3.8,9599,simonw,closed,0,,,,,2,2020-05-04T16:14:59Z,2020-05-04T16:22:30Z,2020-05-04T16:22:30Z,OWNER,,"https://travis-ci.org/github/simonw/datasette/jobs/682624476 Lots of warnings like this: ``` /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33 /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33 /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33: DeprecationWarning: ""@coroutine"" decorator is deprecated since Python 3.8, use ""async def"" instead def method(self, *args, **kwargs): /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/__init__.py:27 /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/__init__.py:27: DeprecationWarning: ""@coroutine"" decorator is deprecated since Python 3.8, use ""async def"" instead def _open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/754/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 612082842,MDU6SXNzdWU2MTIwODI4NDI=,755,"Fix ""no such column: id"" output in tests",9599,simonw,closed,0,,,,,1,2020-05-04T18:37:49Z,2020-05-04T18:42:14Z,2020-05-04T18:42:14Z,OWNER,,"``` pytest ... tests/test_custom_pages.py ........ [ 33%] tests/test_database.py ......no such column: id ... [ 35%] ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/755/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 612089949,MDU6SXNzdWU2MTIwODk5NDk=,756,Add pipx to installation documentation,9599,simonw,closed,0,,,,,2,2020-05-04T18:49:01Z,2020-05-04T19:19:06Z,2020-05-04T19:10:33Z,OWNER,,"Add to this page: https://datasette.readthedocs.io/en/stable/installation.html Here's how to install plugins: https://twitter.com/simonw/status/1257348687979778050 ``` $ datasette plugins [] $ pipx inject datasette datasette-json-html injected package datasette-json-html into venv datasette done! ✨ 🌟 ✨ $ datasette plugins [ { ""name"": ""datasette-json-html"", ""static"": false, ""templates"": false, ""version"": ""0.6"" } ] ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/756/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 612151767,MDU6SXNzdWU2MTIxNTE3Njc=,15,Expose scores from ZCOMPUTEDASSETATTRIBUTES,9599,simonw,closed,0,,,,,7,2020-05-04T20:36:07Z,2020-12-20T04:44:22Z,2020-05-05T00:11:45Z,MEMBER,,"The Apple Photos database has a `ZCOMPUTEDASSETATTRIBUTES` that looks absurdly interesting... it has calculated scores for every photo: ",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 612658444,MDU6SXNzdWU2MTI2NTg0NDQ=,109,"table.create_index(..., ignore=True)",9599,simonw,closed,0,,,,,1,2020-05-05T14:44:21Z,2020-05-05T14:46:53Z,2020-05-05T14:46:53Z,OWNER,,Option to silently do nothing if the index already exists.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/109/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 612860531,MDU6SXNzdWU2MTI4NjA1MzE=,17,Only install osxphotos if running on macOS,9599,simonw,closed,0,,,,,3,2020-05-05T20:03:26Z,2020-05-05T20:20:05Z,2020-05-05T20:11:23Z,MEMBER,,The build is broken right now because you can't `pip install osxphotos` on Ubuntu.,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 613002220,MDU6SXNzdWU2MTMwMDIyMjA=,19,apple-photos command should work even if upload has not run,9599,simonw,closed,0,,,,,1,2020-05-06T02:02:25Z,2020-05-19T20:59:59Z,2020-05-19T20:59:59Z,MEMBER,,"I want people to be able to query their Apple Photos metadata without having to first run `upload` to upload all of their files to their own S3 bucket. To do this I can have `apple-photos` calculate SHA256 hashes of each photo if the `uploads` table does not yet exist (or does not contain that photo).",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 613006393,MDU6SXNzdWU2MTMwMDYzOTM=,20,Ability to serve thumbnailed Apple Photo from its place on disk,9599,simonw,closed,0,,,,,10,2020-05-06T02:17:50Z,2020-05-25T20:14:22Z,2020-05-25T20:09:41Z,MEMBER,,"A custom Datasette plugin that can be run locally on a Mac laptop which knows how to serve photos such that they can be seen in the browser. _Originally posted by @simonw in https://github.com/dogsheep/photos-to-sqlite/issues/19#issuecomment-624406285_",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 613467382,MDU6SXNzdWU2MTM0NjczODI=,761,Allow-list pragma_table_info(tablename) and similar,9599,simonw,closed,0,,,,,8,2020-05-06T16:54:14Z,2020-05-07T03:09:05Z,2020-05-06T17:18:38Z,OWNER,,"It would be great if `pragma_table_info(tablename)` was allowed to be used in queries. See also https://github.com/simonw/til/blob/master/sqlite/list-all-columns-in-a-database.md > `select * from pragma_table_info(tablename);` is currently disallowed for user-provided queries via a regex restriction - but could help here too. > > https://github.com/simonw/datasette/blob/d349d57cdf3d577afb62bdf784af342a4d5be660/datasette/utils/__init__.py#L174 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/760#issuecomment-624729459_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/761/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 613777056,MDU6SXNzdWU2MTM3NzcwNTY=,39,issues foreign key to repo isn't working,9599,simonw,closed,0,,,,,1,2020-05-07T05:11:48Z,2020-08-18T14:24:46Z,2020-08-18T14:23:56Z,MEMBER,,"https://github-to-sqlite.dogsheep.net/github/issues?_facet=repo If the foreign key was working those would be repository names. From the schema at the bottom of the page: ``` [repo] TEXT, ``` That's the wrong type and not a foreign key.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/39/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 614806683,MDExOlB1bGxSZXF1ZXN0NDE1Mjg2MTA1,763,Documentation + improvements for db.execute() and Results class,9599,simonw,closed,0,,,,,0,2020-05-08T15:16:02Z,2020-06-11T16:05:48Z,2020-05-08T16:05:46Z,OWNER,simonw/datasette/pulls/763,"Refs #685 Still TODO: - [x] Implement `results.first()` - [x] Implement `results.single_value()` - [x] Unit tests for the above ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/763/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 615474990,MDU6SXNzdWU2MTU0NzQ5OTA=,21,bpylist.archiver.CircularReference: archive has a cycle with uid(13),9599,simonw,closed,0,,,,,11,2020-05-10T20:58:06Z,2020-12-19T07:44:49Z,2020-05-10T21:57:13Z,MEMBER,,"``` % python -i $(which photos-to-sqlite) apple-photos photos.db Traceback (most recent call last): File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py"", line 611, in place return self._place # pylint: disable=access-member-before-definition AttributeError: 'PhotoInfo' object has no attribute '_place' During handling of the above exception, another exception occurred: Traceback (most recent call last): File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/bin/photos-to-sqlite"", line 11, in load_entry_point('photos-to-sqlite', 'console_scripts', 'photos-to-sqlite')() File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py"", line 829, in __call__ return self.main(*args, **kwargs) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py"", line 782, in main rv = self.invoke(ctx) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py"", line 610, in invoke return callback(*args, **kwargs) File ""/Users/simon/Dropbox/Development/photos-to-sqlite/photos_to_sqlite/cli.py"", line 249, in apple_photos photo_row = osxphoto_to_row(sha256, photo) File ""/Users/simon/Dropbox/Development/photos-to-sqlite/photos_to_sqlite/utils.py"", line 91, in osxphoto_to_row place = photo.place File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py"", line 614, in place self._place = PlaceInfo5(self._info[""reverse_geolocation""]) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/placeinfo.py"", line 505, in __init__ self._plrevgeoloc = archiver.unarchive(revgeoloc_bplist) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 16, in unarchive return Unarchive(plist).top_object() File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 256, in top_object return self.decode_object(self.top_uid) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 247, in decode_object obj = klass.decode_archive(ArchivedObject(raw_obj, self)) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/placeinfo.py"", line 126, in decode_archive mapItem = archive.decode(""mapItem"") File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 140, in decode return self._unarchiver.decode_key(self._object, key) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 216, in decode_key return self.decode_object(val) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 247, in decode_object obj = klass.decode_archive(ArchivedObject(raw_obj, self)) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/placeinfo.py"", line 180, in decode_archive sortedPlaceInfos = archive.decode(""sortedPlaceInfos"") File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 140, in decode return self._unarchiver.decode_key(self._object, key) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 216, in decode_key return self.decode_object(val) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 247, in decode_object obj = klass.decode_archive(ArchivedObject(raw_obj, self)) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 112, in decode_archive return [archive._decode_index(index) for index in uids] File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 112, in return [archive._decode_index(index) for index in uids] File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 137, in _decode_index return self._unarchiver.decode_object(index) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 247, in decode_object obj = klass.decode_archive(ArchivedObject(raw_obj, self)) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/placeinfo.py"", line 217, in decode_archive placeType = archive.decode(""placeType"") File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 140, in decode return self._unarchiver.decode_key(self._object, key) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 216, in decode_key return self.decode_object(val) File ""/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/bpylist/archiver.py"", line 227, in decode_object raise CircularReference(index) bpylist.archiver.CircularReference: archive has a cycle with uid(13) ``` In the debugger I traced this back to: ``` 178 @staticmethod 179 def decode_archive(archive): 180 -> sortedPlaceInfos = archive.decode(""sortedPlaceInfos"") 181 finalPlaceInfos = archive.decode(""finalPlaceInfos"") 182 return PLRevGeoMapItem(sortedPlaceInfos, finalPlaceInfos) ```",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 615477131,MDU6SXNzdWU2MTU0NzcxMzE=,111,sqlite-utils drop-table and drop-view commands,9599,simonw,closed,0,,,,,2,2020-05-10T21:10:42Z,2020-05-11T01:58:36Z,2020-05-11T00:44:26Z,OWNER,,Would be useful to be able to drop views and tables from the CLI.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/111/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 621280529,MDU6SXNzdWU2MjEyODA1Mjk=,23,create-subset command for creating a publishable subset of a photos database,9599,simonw,closed,0,,,,,1,2020-05-19T20:58:20Z,2020-05-19T22:32:48Z,2020-05-19T22:32:37Z,MEMBER,,"I want to share a subset of my photos, without sharing everything. Idea: $ photos-to-sqlite create-subset photos.db public.db ""select sha256 from ... where ..."" So the command takes a SQL query that returns sha256 hashes, then creates a new file called `public.db` containing just the data corresponding to those photos.",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/23/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 621286870,MDU6SXNzdWU2MjEyODY4NzA=,113,Syntactic sugar for ATTACH DATABASE,9599,simonw,closed,0,,,,,2,2020-05-19T21:10:00Z,2021-02-19T05:09:12Z,2021-02-19T04:56:36Z,OWNER,,"https://www.sqlite.org/lang_attach.html Maybe something like this: ```python db.attach(""other_db"", ""other_db.db"") ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/113/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 621332242,MDU6SXNzdWU2MjEzMzIyNDI=,25,Create a public demo,9599,simonw,closed,0,,,,,5,2020-05-19T22:47:20Z,2020-05-21T22:26:16Z,2020-05-20T05:54:18Z,MEMBER,,"So I can show people what this does, using some of my photos.",256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 621444763,MDU6SXNzdWU2MjE0NDQ3NjM=,26,Rename project to dogsheep-photos,9599,simonw,closed,0,,,,,8,2020-05-20T04:12:34Z,2020-05-20T04:31:02Z,2020-05-20T04:30:40Z,MEMBER,,`photos-to-sqlite` doesn't really capture the full scope of this project anymore.,256834907,dogsheep-photos,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 626131309,MDU6SXNzdWU2MjYxMzEzMDk=,775,Move test plugins into datasette/tests/plugins/ directory,9599,simonw,closed,0,,,,,1,2020-05-28T00:46:58Z,2020-05-28T00:57:31Z,2020-05-28T00:57:31Z,OWNER,,"Right now the plugins used during test runs are baked into strings. It would be nicer if they were actual files on disk. Will make #581 easier to write tests for.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/775/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 626663119,MDU6SXNzdWU2MjY2NjMxMTk=,781,request.url and request.scheme should obey force_https_urls config setting,9599,simonw,closed,0,,,,,3,2020-05-28T16:54:47Z,2020-05-28T17:39:54Z,2020-05-28T17:10:13Z,OWNER,,"I'm trying to get the https://www.niche-museums.com/browse/feed.atom feed to validate and I git this from https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom > This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations. > > [line 6](https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom#l6), column 73: Self reference doesn't match document location [[help](https://validator.w3.org/feed/docs/warning/SelfDoesntMatchLocation.html ""more information about this error"")] > > I tried to fix this using `force_https_urls` ([commit](https://github.com/simonw/museums/commit/5dc8e2c717c59f9e949b65e47a59878e01f929e4)) but it didn't work - because that setting isn't respected by the Request class: https://github.com/simonw/datasette/blob/40885ef24e32d91502b6b8bbad1c7376f50f2830/datasette/utils/asgi.py#L15-L32",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/781/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 627836898,MDExOlB1bGxSZXF1ZXN0NDI1NTMxMjA1,783,Authentication: plugin hooks plus default --root auth mechanism,9599,simonw,closed,0,,,,,0,2020-05-30T22:25:47Z,2020-06-01T01:16:44Z,2020-06-01T01:16:43Z,OWNER,simonw/datasette/pulls/783,See #699,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/783/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 629459637,MDU6SXNzdWU2Mjk0NTk2Mzc=,792,"Replace response.body.decode(""utf8"") with response.text in tests",9599,simonw,closed,0,,,,,0,2020-06-02T19:32:24Z,2020-06-02T21:29:58Z,2020-06-02T21:29:58Z,OWNER,,"Make use of the `response.text` property to clean up the tests a tiny bit: https://github.com/simonw/datasette/blob/57cf5139c552cb7feab9947daa949ca434cc0a66/tests/fixtures.py#L26-L38",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/792/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 631789422,MDU6SXNzdWU2MzE3ODk0MjI=,799,TestResponse needs to handle multiple set-cookie headers,9599,simonw,closed,0,,,,,2,2020-06-05T17:39:52Z,2020-06-05T18:34:10Z,2020-06-05T18:34:10Z,OWNER,,"Seeing this test failure on #798: ``` _______________________ test_auth_token _______________________ app_client = def test_auth_token(app_client): ""The /-/auth-token endpoint sets the correct cookie"" assert app_client.ds._root_token is not None path = ""/-/auth-token?token={}"".format(app_client.ds._root_token) response = app_client.get(path, allow_redirects=False,) assert 302 == response.status assert ""/"" == response.headers[""Location""] > assert {""id"": ""root""} == app_client.ds.unsign(response.cookies[""ds_actor""], ""actor"") E KeyError: 'ds_actor' datasette/tests/test_auth.py:12: KeyError ``` It looks like that's happening because the ASGI middleware is adding another set-cookie header - but those two set-cookie headers are combined into one when the TestResponse is constructed: https://github.com/simonw/datasette/blob/0c064c5fe220b7b3d8dcf85b02b4e60452c47232/tests/fixtures.py#L113-L127",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/799/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 632056825,MDU6SXNzdWU2MzIwNTY4MjU=,802,"""datasette plugins"" command is broken",9599,simonw,closed,0,,,,,1,2020-06-05T23:33:01Z,2020-06-05T23:46:43Z,2020-06-05T23:46:43Z,OWNER,,I broke it in https://github.com/simonw/datasette/commit/a7137dfe069e5fceca56f78631baebd4a6a19967 - and it turns out there was no test coverage so I didn't realize it was broken.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/802/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 632645865,MDExOlB1bGxSZXF1ZXN0NDI5MzY2NjQx,803,Canned query permissions,9599,simonw,closed,0,,,,,0,2020-06-06T18:20:00Z,2020-06-06T19:40:21Z,2020-06-06T19:40:20Z,OWNER,simonw/datasette/pulls/803,Refs #800. Closes #786,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/803/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 634844634,MDU6SXNzdWU2MzQ4NDQ2MzQ=,817,Drop resource_type from permission_allowed system,9599,simonw,closed,0,,,,,1,2020-06-08T18:41:37Z,2020-06-08T19:00:12Z,2020-06-08T19:00:12Z,OWNER,,"Current signature: permission_allowed(datasette, actor, action, resource_type, resource_identifier) It turns out the `resource_type` is always the same thing for any given action, so it's not actually useful. I'm going to drop it. New signature will be: permission_allowed(datasette, actor, action, resource) Refs #811.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/817/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 635049296,MDU6SXNzdWU2MzUwNDkyOTY=,820,Idea: Plugin hook for registering canned queries,9599,simonw,closed,0,,,,,2,2020-06-09T01:58:21Z,2020-06-18T17:58:02Z,2020-06-18T17:58:02Z,OWNER,,"Thought of this while thinking about possible permissions plugins (#818). Imagine an API key plugin which allows access for API keys. It could let users register new API keys by providing a writable canned query for writing to the `api_keys` table. To do this the plugin needs to register the query. At the moment queries have to be registered in `metadata.json` - a plugin hook for registering additional queries could help solve this. One challenge: how does the plugin know which named database the query should be registered for? It could default to the first attached database and allow users to optionally tell the plugin ""actually use this named database instead"" in plugin configuration.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/820/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 637889964,MDU6SXNzdWU2Mzc4ODk5NjQ=,115,Ability to execute insert/update statements with the CLI,9599,simonw,closed,0,,,,,1,2020-06-12T17:01:17Z,2020-06-12T17:51:11Z,2020-06-12T17:41:10Z,OWNER,,"``` $ sqlite-utils github.db ""update stars set starred_at = ''"" Traceback (most recent call last): File ""/Users/simon/.local/bin/sqlite-utils"", line 8, in sys.exit(cli()) File ""/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py"", line 829, in __call__ return self.main(*args, **kwargs) File ""/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py"", line 782, in main rv = self.invoke(ctx) File ""/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/click/core.py"", line 610, in invoke return callback(*args, **kwargs) File ""/Users/simon/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 673, in query headers = [c[0] for c in cursor.description] TypeError: 'NoneType' object is not iterable ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/115/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 637899539,MDU6SXNzdWU2Mzc4OTk1Mzk=,40,Demo deploy is broken,9599,simonw,closed,0,,,,,2,2020-06-12T17:20:17Z,2020-06-12T18:06:48Z,2020-06-12T18:06:48Z,MEMBER,,"https://github.com/dogsheep/github-to-sqlite/runs/766180404?check_suite_focus=true ``` The following NEW packages will be installed: sqlite3 0 upgraded, 1 newly installed, 0 to remove and 11 not upgraded. Need to get 752 kB of archives. After this operation, 2482 kB of additional disk space will be used. Ign:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 sqlite3 amd64 3.22.0-1ubuntu0.3 Err:1 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 sqlite3 amd64 3.22.0-1ubuntu0.3 404 Not Found [IP: 52.177.174.250 80] E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/s/sqlite3/sqlite3_3.22.0-1ubuntu0.3_amd64.deb 404 Not Found [IP: 52.177.174.250 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ##[error]Process completed with exit code 100. ```",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 638104520,MDU6SXNzdWU2MzgxMDQ1MjA=,841,Research feasibility of 100% test coverage,9599,simonw,closed,0,,,,,9,2020-06-13T06:07:01Z,2020-06-13T21:38:46Z,2020-06-13T21:38:46Z,OWNER,,"Inspired by https://twitter.com/mikeal/status/1271473021593636866 > Almost every library I’ve written in the last 2 years has had 100% coverage and that’s probably not going to change in the future. It’s not that hard to start at 100% and hold onto it and the workflow it enables is so much nicer.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/841/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 638229448,MDU6SXNzdWU2MzgyMjk0NDg=,843,Configure codecov.io,9599,simonw,closed,0,,,,,2,2020-06-13T20:45:00Z,2020-06-13T21:36:52Z,2020-06-13T21:36:52Z,OWNER,,_Originally posted by @simonw in https://github.com/simonw/datasette/issues/841#issuecomment-643660757_,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/843/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 638230433,MDExOlB1bGxSZXF1ZXN0NDM0MDU1NzUy,844,Action to run tests and upload coverage report,9599,simonw,closed,0,,,,,1,2020-06-13T20:52:47Z,2020-06-13T21:36:52Z,2020-06-13T21:36:50Z,OWNER,simonw/datasette/pulls/844,Refs #843,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/844/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 638241779,MDU6SXNzdWU2MzgyNDE3Nzk=,846,"""Too many open files"" error running tests",9599,simonw,closed,0,,,,,6,2020-06-13T22:11:40Z,2020-06-14T00:26:31Z,2020-06-14T00:26:31Z,OWNER,,"I got this on my laptop: ```pytest ... /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.7/site-packages/jinja2/loaders.py:171: in get_source f = open_if_exists(filename) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ filename = '/Users/simon/Dropbox/Development/datasette/datasette/templates/400.html', mode = 'rb' def open_if_exists(filename, mode='rb'): """"""Returns a file descriptor for the filename if that file exists, otherwise `None`. """""" try: > return open(filename, mode) E OSError: [Errno 24] Too many open files: '/Users/simon/Dropbox/Development/datasette/datasette/templates/400.html' /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.7/site-packages/jinja2/utils.py:154: OSError ``` Based on the conversation in https://github.com/pytest-dev/pytest/issues/2970 I'm worried that my tests are opening too many files without closing them. In particular... I call `sqlite3.connect(filepath)` a LOT - and I don't ever call `conn.close()` on those opened connections: https://github.com/simonw/datasette/blob/cf7a2bdb404734910ec07abc7571351a2d934828/datasette/database.py#L58-L60 Could this be resulting in my tests eventually opening too many unclosed file handles? How could I confirm this?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/846/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 638259643,MDU6SXNzdWU2MzgyNTk2NDM=,847,Take advantage of .coverage being a SQLite database,9599,simonw,closed,0,,,,,4,2020-06-14T00:41:25Z,2020-06-28T20:50:21Z,2020-06-28T20:50:21Z,OWNER,,"The `.coverage` file generated by running `pytest-cov` is now a SQLite database! I could do something interesting with this. Maybe after each test run for a new commit I could store that database file somewhere? Lots of interesting challenges here. I got a change into `coveragepy` last year which helps make the custom SQL functions available for doing fun things in Datasette: https://github.com/nedbat/coveragepy/issues/868 Bigger challenge: if I have a DB file for every commit, that's hundreds (potentially thousands) of DB files. Datasette isn't designed to handle thousands of files like that. So, do I figure out how to have Datasette open a file on-command for just a single request? Or, an easier option, do I copy data from those files into a single database with a modified schema to include the commit hash in each table row? (Following on from #841 and #844)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/847/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 642651572,MDU6SXNzdWU2NDI2NTE1NzI=,860,Plugin hook for instance/database/table metadata,9599,simonw,closed,0,,,,,10,2020-06-21T22:20:25Z,2022-01-13T22:21:42Z,2021-06-26T22:56:28Z,OWNER,,"I'm not happy with how `metadata.(json|yaml)` keeps growing new features. Rather than having a single plugin hook for all of `metadata.json` I'm going to split out the feature that shows actual real metadata for tables and databases - `source`, `license` etc - into its own plugin-powered mechanism. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/357#issuecomment-647189045_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/860/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 642652808,MDU6SXNzdWU2NDI2NTI4MDg=,861,Script to generate larger SQLite test files,9599,simonw,closed,0,,,,,3,2020-06-21T22:30:58Z,2020-06-23T03:44:18Z,2020-06-23T03:44:18Z,OWNER,,"> I'll write a little script which generates a 300MB SQLite file with a bunch of tables with lots of randomly generated rows in to help test this. > > Having a tool like that which can generate larger databases with different gnarly performance characteristics will be useful for other performance work too. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/859#issuecomment-647189948_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/861/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 644122661,MDU6SXNzdWU2NDQxMjI2NjE=,116,Documentation for table.pks introspection property,9599,simonw,closed,0,,,,,2,2020-06-23T20:27:24Z,2020-06-23T21:21:33Z,2020-06-23T21:03:14Z,OWNER,,https://github.com/simonw/sqlite-utils/blob/4d9a3204361d956440307a57bd18c829a15861db/sqlite_utils/db.py#L535-L540,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/116/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 648421105,MDU6SXNzdWU2NDg0MjExMDU=,877,Consider dropping explicit CSRF protection entirely?,9599,simonw,closed,0,,,,,9,2020-06-30T19:00:55Z,2020-09-15T20:42:05Z,2020-09-15T20:42:04Z,OWNER,,"https://scotthelme.co.uk/csrf-is-dead/ from Feb 2017 has background here. The `SameSite=lax` cookie property effectively eliminates CSRF in modern browsers. https://caniuse.com/#search=SameSite shows 92.13% global support for it. Datasette already uses `SameSite=lax` when it sets cookies by default: https://github.com/simonw/datasette/blob/af350ba4571b8e3f9708c40f2ddb48fea7ac1084/datasette/utils/asgi.py#L327-L341 A few options then. I could ditch CSRF protection entirely. I could make it optional - turn it off by default, but let users who care about that remaining 7.87% of global users opt back into it. One catch: login CSRF: I don't see how `SameSite=lax` protects against that attack.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/877/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 651159727,MDU6SXNzdWU2NTExNTk3Mjc=,41,Demo is failing to deploy,9599,simonw,closed,0,,,,,7,2020-07-05T22:40:33Z,2020-07-06T01:07:03Z,2020-07-06T01:07:02Z,MEMBER,,"https://github.com/dogsheep/github-to-sqlite/runs/837714622?check_suite_focus=true ``` Creating Revision.........................................................................................................................................failed Deployment failed ERROR: (gcloud.run.deploy) Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revision might contain more information. Traceback (most recent call last): File ""/opt/hostedtoolcache/Python/3.8.3/x64/bin/datasette"", line 8, in sys.exit(cli()) File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py"", line 829, in __call__ return self.main(*args, **kwargs) File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py"", line 782, in main rv = self.invoke(ctx) File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py"", line 610, in invoke return callback(*args, **kwargs) File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/publish/cloudrun.py"", line 138, in cloudrun check_call( File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/subprocess.py"", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/datasette-222320/datasette github-to-sqlite' returned non-zero exit status 1. ##[error]Process completed with exit code 1. ```",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 652700770,MDU6SXNzdWU2NTI3MDA3NzA=,119,Ability to remove a foreign key,9599,simonw,closed,0,,,,,3,2020-07-07T22:31:37Z,2020-09-24T20:36:59Z,2020-09-24T20:36:59Z,OWNER,,Useful if you add one but make a mistake and need to undo it without recreating the database from scratch.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/119/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 654405302,MDU6SXNzdWU2NTQ0MDUzMDI=,42,Option for importing just specific repos,9599,simonw,closed,0,,,,,0,2020-07-09T23:20:15Z,2020-07-09T23:25:35Z,2020-07-09T23:25:35Z,MEMBER,,"For if you know which specific repos you care about, as opposed to loading everything owned by the authenticated user. github-to-sqlite repos specific.db -r simonw/datasette -r simonw/github-contents ",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/42/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 655465863,MDU6SXNzdWU2NTU0NjU4NjM=,892,"""latest"" in new documentation navbar is invisible",9599,simonw,closed,0,,,,,2,2020-07-12T19:57:21Z,2020-07-12T20:02:35Z,2020-07-12T20:02:17Z,OWNER,,"On https://datasette.readthedocs.io/en/latest/ Compare with https://datasette.readthedocs.io/en/0.45/ Some custom CSS should fix it. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/892/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 657747959,MDU6SXNzdWU2NTc3NDc5NTk=,895,SQL query output should show numeric values in a different colour,9599,simonw,closed,0,,,,,1,2020-07-16T00:28:03Z,2020-09-15T20:40:08Z,2020-09-15T20:40:08Z,OWNER,,"Compare https://latest.datasette.io/fixtures/sortable with https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+content%2C+sortable%2C+sortable_with_nulls%2C+sortable_with_nulls_2%2C+text+from+sortable+order+by+pk1%2C+pk2+limit+101 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/895/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 658476055,MDU6SXNzdWU2NTg0NzYwNTU=,896,Use white-space: pre-wrap on ALL table cell contents,9599,simonw,closed,0,,,,,4,2020-07-16T19:05:21Z,2020-07-17T01:26:08Z,2020-07-17T01:26:08Z,OWNER,,"Is there any reason NOT to apply `white-space: pre-wrap` to the contents of all table cells in Datasette? The default display mechanism of HTML (stripping leading/trailing slashes and collapsing all other whitespace) doesn't really make sense for displaying the kind of data that Datasette works with.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/896/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 659580487,MDU6SXNzdWU2NTk1ODA0ODc=,897,Request method for retrieving the unparsed request body,9599,simonw,closed,0,,,,,1,2020-07-17T19:51:40Z,2020-07-17T20:16:02Z,2020-07-17T20:12:50Z,OWNER,,"I'm writing a plugin (https://github.com/simonw/datasette-update-api/issues/2) that implements an API for inserting JSON data. As such, I'd like to `POST` a JSON blob rather than using `key=value` form encoded data. Right now there's a `request.post_vars()` method but no `request.post_body()` one: https://github.com/simonw/datasette/blob/c5f06bc356fb5917ef7fbb6fe4693f30d711cdb3/datasette/utils/asgi.py#L93-L103",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/897/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 660355904,MDU6SXNzdWU2NjAzNTU5MDQ=,43,github-to-sqlite tags command for fetching tags,9599,simonw,closed,0,,,,,4,2020-07-18T20:14:12Z,2020-07-18T23:05:56Z,2020-07-18T21:52:15Z,MEMBER,,Fetches paginated data from https://api.github.com/repos/simonw/datasette/tags,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 660413281,MDU6SXNzdWU2NjA0MTMyODE=,44,Rename tags.repo_id column to tags.repo,9599,simonw,closed,0,,,,,0,2020-07-18T22:13:46Z,2020-07-18T22:15:12Z,2020-07-18T22:15:12Z,MEMBER,,"For improved consistency with other tables. https://observablehq.com/@simonw/datasette-table-diagram ![datasette-table-diagram(1)](https://user-images.githubusercontent.com/9599/87862843-3cca4900-c909-11ea-9c76-58b3f4aca43f.png) ",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/44/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 660429601,MDU6SXNzdWU2NjA0Mjk2MDE=,45,Fix the demo - it breaks because of the tags table change,9599,simonw,closed,0,,,,,5,2020-07-18T22:49:32Z,2020-07-18T23:03:14Z,2020-07-18T23:03:13Z,MEMBER,,"https://github.com/dogsheep/github-to-sqlite/runs/885773677 ``` File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 476, in save_tags db[""tags""].insert_all( File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/sqlite_utils/db.py"", line 1145, in insert_all result = self.db.conn.execute(query, params) sqlite3.OperationalError: table tags has no column named repo ``` That's because I changed the name in #44. I thought this would be safe since no-one else could possibly be using this yet (it hadn't shipped in a release) but turns out I broke my demo!",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 663145122,MDU6SXNzdWU2NjMxNDUxMjI=,903,Add temporary plugin testing pattern to the testing docs,9599,simonw,closed,0,,,,,1,2020-07-21T16:22:34Z,2022-07-18T21:34:33Z,2022-07-18T21:31:22Z,OWNER,,"https://til.simonwillison.net/pytest/registering-plugins-in-tests Would be useful to include this pattern on https://datasette.readthedocs.io/en/stable/testing_plugins.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/903/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 663317875,MDU6SXNzdWU2NjMzMTc4NzU=,905,/database.db download should include content-length header,9599,simonw,closed,0,,,,,2,2020-07-21T21:23:48Z,2020-07-22T04:59:46Z,2020-07-22T04:52:45Z,OWNER,,I can do this by modifying this function: https://github.com/simonw/datasette/blob/02dc6298bdbfb1d63e0d2a39ff597b5fcc60e06b/datasette/utils/asgi.py#L248-L270,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/905/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 663976976,MDU6SXNzdWU2NjM5NzY5NzY=,48,Add a table of contents to the README,9599,simonw,closed,0,,,,,3,2020-07-22T18:54:33Z,2020-07-23T17:46:07Z,2020-07-22T19:03:02Z,MEMBER,,Using https://github.com/jonschlinkert/markdown-toc,206156866,twitter-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 665700495,MDU6SXNzdWU2NjU3MDA0OTU=,122,CLI utility for inserting binary files into SQLite,9599,simonw,closed,0,,,,,10,2020-07-26T03:27:39Z,2020-07-27T07:10:41Z,2020-07-27T07:09:03Z,OWNER,,"SQLite BLOB columns can store entire binary files. The challenge is inserting them, since they don't neatly fit into JSON objects. It would be great if the `sqlite-utils` CLI had a trick for helping with this. Inspired by https://github.com/simonw/datasette-media/issues/14",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/122/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 665701216,MDU6SXNzdWU2NjU3MDEyMTY=,123,--raw option for outputting binary content,9599,simonw,closed,0,,,,,0,2020-07-26T03:35:39Z,2020-07-26T16:44:11Z,2020-07-26T16:44:11Z,OWNER,,"Related to the `insert-files` work in #122 - it should be easy to get binary data back out of the database again. One way to do that could be: sqlite-utils files.db ""select content from files where key = 'foo.jpg'"" --raw The `--raw` option would cause just the contents of the first column to be output directly to stdout.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/123/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 665802405,MDU6SXNzdWU2NjU4MDI0MDU=,124,sqlite-utils query should support named parameters,9599,simonw,closed,0,,,,,1,2020-07-26T15:25:10Z,2020-07-30T22:57:51Z,2020-07-27T03:53:58Z,OWNER,,"To help out with escaping - so you can run this: sqlite-utils query ""insert into foo (blah) values (:blah)"" --param blah `something here`",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/124/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 665817570,MDU6SXNzdWU2NjU4MTc1NzA=,125,"Output binary columns in ""sqlite-utils query"" JSON",9599,simonw,closed,0,,,,,4,2020-07-26T16:47:02Z,2020-07-27T00:49:41Z,2020-07-27T00:48:45Z,OWNER,,You get an error if you try to run a query that returns data from a BLOB.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/125/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 665819048,MDU6SXNzdWU2NjU4MTkwNDg=,126,Ability to insert binary data on the CLI using JSON,9599,simonw,closed,0,,,,,2,2020-07-26T16:54:14Z,2020-07-27T04:00:33Z,2020-07-27T03:59:45Z,OWNER,,"> I could solve round tripping (at least a bit) by allowing insert to be run with a flag that says ""these columns are base64 encoded, store the decoded data in a BLOB"". > > That would solve inserting binary data using JSON too. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/125#issuecomment-664012247_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/126/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 666040390,MDU6SXNzdWU2NjYwNDAzOTA=,127,Ability to insert files piped to insert-files stdin,9599,simonw,closed,0,,,,,3,2020-07-27T07:09:33Z,2020-07-30T03:08:52Z,2020-07-30T03:08:18Z,OWNER,,"> Inserting files by piping them in should work - but since a filename cannot be derived this will need a `--name blah.gif` option. > > cat blah.gif | sqlite-utils insert-files files.db files - --name=blah.gif > _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/122#issuecomment-664128071_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/127/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 666639051,MDU6SXNzdWU2NjY2MzkwNTE=,128,Support UUID and memoryview types,9599,simonw,closed,0,,,,,1,2020-07-27T23:08:34Z,2020-07-30T01:10:43Z,2020-07-30T01:10:43Z,OWNER,,`psycopg2` can return data from PostgreSQL as `uuid.UUID` or `memoryview` objects. These should to be supported by `sqlite-utils` - mainly for https://github.com/simonw/db-to-sqlite,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/128/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 667467128,MDU6SXNzdWU2Njc0NjcxMjg=,909,AsgiFileDownload: filename not correctly passed,9599,simonw,closed,0,,,,,2,2020-07-29T00:41:43Z,2020-07-30T00:56:17Z,2020-07-29T21:34:48Z,OWNER,,"https://github.com/simonw/datasette/blob/3c33b421320c0be81a625ca7307b2e4416a9ed5b/datasette/utils/asgi.py#L396-L405 `self.filename` should be passed to `asgi_send_file()`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/909/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 668064778,MDU6SXNzdWU2NjgwNjQ3Nzg=,912,"Add ""publishing to Vercel"" to the publish docs",9599,simonw,closed,0,,,,,0,2020-07-29T18:50:58Z,2020-07-31T17:06:35Z,2020-07-31T17:06:35Z,OWNER,,"https://datasette.readthedocs.io/en/0.45/publish.html#datasette-publish currently only lists Cloud Run, Heroku and Fly. It should list Vercel too. (I should probably rename `datasette-publish-now` to `datasette-publish-vercel`)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/912/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 668308777,MDU6SXNzdWU2NjgzMDg3Nzc=,129,"""insert-files --sqlar"" for creating SQLite archives",9599,simonw,closed,0,,,,,2,2020-07-30T02:28:29Z,2020-07-30T22:41:01Z,2020-07-30T22:40:55Z,OWNER,,"A `--sqlar` option could cause `insert-files` to behave in the same way as SQLite's own sqlar mechanism. https://www.sqlite.org/sqlar.html and https://sqlite.org/sqlar/doc/trunk/README.md",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/129/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 671056788,MDU6SXNzdWU2NzEwNTY3ODg=,914,"""Object of type bytes is not JSON serializable"" for _nl=on",9599,simonw,closed,0,,,,,1,2020-08-01T17:43:10Z,2020-08-16T21:10:27Z,2020-08-16T18:26:59Z,OWNER,,"https://latest.datasette.io/fixtures/binary_data.json?_sort_desc=data&_shape=array returns this: ```json [ { ""rowid"": 1, ""data"": ""this is binary data"" } ] ``` But adding `&_nl=on` returns this: https://latest.datasette.io/fixtures/binary_data.json?_sort_desc=data&_shape=array&_nl=on ```json { ""ok"": false, ""error"": ""Object of type bytes is not JSON serializable"", ""status"": 500, ""title"": null } ``` I found this error by running `wget -r 127.0.0.1:8001` against my local `fixtures.db`. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/914/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 671130371,MDU6SXNzdWU2NzExMzAzNzE=,130,Support tokenize option for FTS,9599,simonw,closed,0,,,,,3,2020-08-01T19:27:22Z,2020-08-01T20:51:28Z,2020-08-01T20:51:14Z,OWNER,,"FTS5 supports things like porter stemming using a `tokenize=` option: https://www.sqlite.org/fts5.html#tokenizers Something like this in code: ``` CREATE VIRTUAL TABLE [{table}_fts] USING {fts_version} ( {columns}, tokenize='porter', content=[{table}] ); ``` I tried this out just now and it worked exactly as expected. So... `db[table].enable_fts(...) should accept a 'tokenize=` argument, and `sqlite-utils enable-fts ...` should support a `--tokenize` option.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/130/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 671763164,MDU6SXNzdWU2NzE3NjMxNjQ=,915,Refactor TableView class so things like datasette-graphql can reuse the logic,9599,simonw,closed,0,,,,,3,2020-08-03T03:13:33Z,2020-08-18T22:28:37Z,2020-08-18T22:28:37Z,OWNER,,_Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-667780040_,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/915/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 675724951,MDU6SXNzdWU2NzU3MjQ5NTE=,918,Security issue: read-only canned queries leak CSRF token in URL,9599,simonw,closed,0,,,,,4,2020-08-09T16:03:01Z,2020-08-09T16:56:48Z,2020-08-09T16:11:59Z,OWNER,,"The HTML form for a read-only canned query includes the hidden CSRF token field added in #798 for writable canned queries (#698). This means that submitting those read-only forms exposes the CSRF token in the URL - for example on https://latest.datasette.io/fixtures/neighborhood_search submitting the form took me to: https://latest.datasette.io/fixtures/neighborhood_search?text=down&csrftoken=IlFubnoxVVpLU1NGT3NMVUoi.HbOPd2YH_epQmp8f_aAt0s-MxtU This token could potentially leak to an attacker if the resulting page has a link to an external site on it and the user clicks the link, since the token would be exposed in the referral logs.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/918/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 675727366,MDU6SXNzdWU2NzU3MjczNjY=,919,"Travis should not build the master branch, only the main branch",9599,simonw,closed,0,,,,,3,2020-08-09T16:18:25Z,2020-08-09T16:26:18Z,2020-08-09T16:19:37Z,OWNER,,"Caused by #849 - since we are mirroring the two branches (to ensure old links to `master` keep working) Travis is building both. The following in `.travis.yml` should fix that: ``` branches: except: - master ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/919/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 675839512,MDU6SXNzdWU2NzU4Mzk1MTI=,132,Features for enabling and disabling WAL mode,9599,simonw,closed,0,,,,,5,2020-08-10T03:25:44Z,2020-08-10T18:59:35Z,2020-08-10T18:59:35Z,OWNER,,I finally figured out how to enable WAL - turns out it's a property of the database file itself: https://github.com/simonw/til/blob/master/sqlite/enabling-wal-mode.md,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 677037043,MDU6SXNzdWU2NzcwMzcwNDM=,923,Add homebrew installation to documentation,9599,simonw,closed,0,,,,,5,2020-08-11T16:54:31Z,2020-08-11T22:53:07Z,2020-08-11T22:52:46Z,OWNER,,"> ``` > $ brew tap simonw/datasette > $ brew install simonw/datasette/datasette > $ datasette --version > datasette, version 0.46 > ``` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/335#issuecomment-672088880_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/923/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 677227912,MDU6SXNzdWU2NzcyMjc5MTI=,925,"""datasette install"" and ""datasette uninstall"" commands",9599,simonw,closed,0,,,,,3,2020-08-11T22:04:32Z,2020-08-11T22:34:37Z,2020-08-11T22:32:12Z,OWNER,,"When installing Datasette plugins it's crucial that they end up in the same virtual environment as Datasette itself. It's not necessarily obvious how to do this, especially if you install Datasette via pipx or homebrew. Solution: `datasette install datasette-vega` and `datasette uninstall datasette-vega` commands that know how to install to the correct place - a very thin wrapper around `pip install`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/925/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 677250834,MDU6SXNzdWU2NzcyNTA4MzQ=,926,"datasette fixtures.db --get ""/fixtures.json""",9599,simonw,closed,0,,,,,2,2020-08-11T22:55:36Z,2020-08-12T00:26:17Z,2020-08-12T00:24:42Z,OWNER,,"I can expose ALL of Datasette's functionality on the command-line (without even running a web server) by adding `--get` and `--post` options to `datasette serve`. datasette fixtures.db --get ""/fixtures.json"" This would instantiate the Datasette ASGI app, run a fake request for `/fixtures.json` through it, dump the results out to standard output and quit. A `--post` option could do the same for a POST request. Treating that as a stretch goal for the moment.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/926/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 677265716,MDExOlB1bGxSZXF1ZXN0NDY2NDEwNzU1,927,"'datasette --get' option, refs #926",9599,simonw,closed,0,,,,,5,2020-08-11T23:31:52Z,2020-08-12T00:24:42Z,2020-08-12T00:24:41Z,OWNER,simonw/datasette/pulls/927,"Refs #926, #898",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/927/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 677272618,MDU6SXNzdWU2NzcyNzI2MTg=,928,Test failures caused by failed attempts to mock pip,9599,simonw,closed,0,,,,,4,2020-08-11T23:53:18Z,2022-02-23T16:19:47Z,2020-08-12T00:07:49Z,OWNER,,"Errors like this one: https://github.com/simonw/datasette/pull/927/checks?check_run_id=973559696 ``` 2020-08-11T23:36:39.8801334Z =================================== FAILURES =================================== 2020-08-11T23:36:39.8802411Z _________________________________ test_install _________________________________ 2020-08-11T23:36:39.8803242Z 2020-08-11T23:36:39.8804935Z thing = 2020-08-11T23:36:39.8806663Z comp = 'main', import_path = 'pip._internal.cli.main' 2020-08-11T23:36:39.8807696Z 2020-08-11T23:36:39.8808728Z def _dot_lookup(thing, comp, import_path): 2020-08-11T23:36:39.8810573Z try: 2020-08-11T23:36:39.8812262Z > return getattr(thing, comp) 2020-08-11T23:36:39.8817136Z E AttributeError: module 'pip._internal.cli' has no attribute 'main' 2020-08-11T23:36:39.8843043Z 2020-08-11T23:36:39.8855951Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/unittest/mock.py:1215: AttributeError 2020-08-11T23:36:39.8873372Z 2020-08-11T23:36:39.8877803Z During handling of the above exception, another exception occurred: 2020-08-11T23:36:39.8906532Z 2020-08-11T23:36:39.8925767Z def get_src_prefix(): 2020-08-11T23:36:39.8928277Z # type: () -> str 2020-08-11T23:36:39.8930068Z if running_under_virtualenv(): 2020-08-11T23:36:39.8949721Z src_prefix = os.path.join(sys.prefix, 'src') 2020-08-11T23:36:39.8951813Z else: 2020-08-11T23:36:39.8969014Z # FIXME: keep src in cwd for now (it is not a temporary folder) 2020-08-11T23:36:39.9012110Z try: 2020-08-11T23:36:39.9013489Z > src_prefix = os.path.join(os.getcwd(), 'src') 2020-08-11T23:36:39.9014538Z E FileNotFoundError: [Errno 2] No such file or directory 2020-08-11T23:36:39.9016122Z 2020-08-11T23:36:39.9017617Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pip/_internal/locations.py:50: FileNotFoundError 2020-08-11T23:36:39.9018802Z 2020-08-11T23:36:39.9020070Z During handling of the above exception, another exception occurred: 2020-08-11T23:36:39.9020930Z 2020-08-11T23:36:39.9022275Z args = (), keywargs = {} 2020-08-11T23:36:39.9023183Z 2020-08-11T23:36:39.9024077Z @wraps(func) 2020-08-11T23:36:39.9024984Z def patched(*args, **keywargs): 2020-08-11T23:36:39.9028770Z > with self.decoration_helper(patched, 2020-08-11T23:36:39.9031861Z args, 2020-08-11T23:36:39.9038358Z keywargs) as (newargs, newkeywargs): 2020-08-11T23:36:39.9039654Z 2020-08-11T23:36:39.9040566Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/unittest/mock.py:1322: 2020-08-11T23:36:39.9041492Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/928/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 677326155,MDU6SXNzdWU2NzczMjYxNTU=,930,Datasette sdist is missing templates (hence broken when installing from Homebrew),9599,simonw,closed,0,,,,,6,2020-08-12T02:20:16Z,2020-08-12T03:30:59Z,2020-08-12T03:30:59Z,OWNER,,Pretty nasty bug this: I'm getting 500 errors for all pages that try to render a template after installing the newly released Datasette 0.47 - both from `pip install` and via Homebrew.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/930/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 677839979,MDU6SXNzdWU2Nzc4Mzk5Nzk=,133,Release a sdist to PyPI,9599,simonw,closed,0,,,,,1,2020-08-12T16:55:09Z,2020-08-12T17:05:06Z,2020-08-12T17:05:06Z,OWNER,,https://pypi.org/project/sqlite-utils/#files currently just has a wheel. I need this to package for homebrew: https://github.com/simonw/homebrew-datasette/issues/10,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/133/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 677926613,MDU6SXNzdWU2Nzc5MjY2MTM=,931,Docker container is no longer being pushed (it's stuck on 0.45),9599,simonw,closed,0,,,,,7,2020-08-12T19:33:03Z,2020-08-12T21:36:20Z,2020-08-12T21:36:20Z,OWNER,,"e.g. https://travis-ci.org/github/simonw/datasette/jobs/717123725 Here's how it broke: ``` --2020-08-12 03:08:17-- https://www.gaia-gis.it/gaia-sins/freexl-1.0.5.tar.gz Resolving www.gaia-gis.it (www.gaia-gis.it)... 212.83.162.51 Connecting to www.gaia-gis.it (www.gaia-gis.it)|212.83.162.51|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2020-08-12 03:08:18 ERROR 404: Not Found. The command '/bin/sh -c wget ""https://www.gaia-gis.it/gaia-sins/freexl-1.0.5.tar.gz"" && tar zxf freexl-1.0.5.tar.gz && cd freexl-1.0.5 && ./configure && make && make install' returned a non-zero code: 8 The command ""docker build -f Dockerfile -t $REPO:$TRAVIS_TAG ."" exited with 8. 0.07s$ docker tag $REPO:$TRAVIS_TAG $REPO:latest Error response from daemon: No such image: [secure]/datasette:0.47.1 The command ""docker tag $REPO:$TRAVIS_TAG $REPO:latest"" exited with 1. 0.08s$ docker push $REPO The push refers to repository [docker.io/[secure]/datasette] An image does not exist locally with the tag: [secure]/datasette The command ""docker push $REPO"" exited with 1. cache.2 store build cache ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/931/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 679637501,MDU6SXNzdWU2Nzk2Mzc1MDE=,934,--get doesn't fully invoke the startup routine,9599,simonw,closed,0,,,,,0,2020-08-15T20:30:25Z,2020-08-15T20:53:49Z,2020-08-15T20:53:49Z,OWNER,,"https://github.com/simonw/datasette/blob/7702ea602188899ee9b0446a874a6a9b546b564d/datasette/cli.py#L417-L433 Spotted this working on https://github.com/simonw/latest-datasette-with-all-plugins/issues/3 - I'd like to be able to use `datasette --get /` as a sanity checking test, but that doesn't work if the init hooks aren't fully executed.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/934/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 679646710,MDU6SXNzdWU2Nzk2NDY3MTA=,935,"db.execute_write_fn(create_tables, block=True) hangs a thread if connection fails",9599,simonw,closed,0,,,,,3,2020-08-15T21:49:17Z,2020-08-15T22:35:33Z,2020-08-15T22:35:33Z,OWNER,,Discovered in https://github.com/simonw/latest-datasette-with-all-plugins/issues/3#issuecomment-674449757,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/935/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 679650632,MDExOlB1bGxSZXF1ZXN0NDY4MzcwNjU4,936,Don't hang in db.execute_write_fn() if connection fails,9599,simonw,closed,0,,,,,2,2020-08-15T22:20:12Z,2020-08-15T22:35:33Z,2020-08-15T22:35:32Z,OWNER,simonw/datasette/pulls/936,Refs #935,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/936/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 679660778,MDExOlB1bGxSZXF1ZXN0NDY4Mzc3MjEy,937,Docs now live at docs.datasette.io,9599,simonw,closed,0,,,,,0,2020-08-15T23:53:52Z,2020-08-15T23:57:06Z,2020-08-15T23:57:05Z,OWNER,simonw/datasette/pulls/937,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/937/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 679700269,MDU6SXNzdWU2Nzk3MDAyNjk=,938,Pass columns to extra CSS/JS/etc plugin hooks,9599,simonw,closed,0,,,,,3,2020-08-16T06:37:47Z,2020-09-30T20:36:12Z,2020-08-16T18:09:59Z,OWNER,,"I'd like `datasette-cluster-map` to only add links to JavaScript on pages that have tables with latitude and longitude columns. Passing the names of the columns to the plugin hook can support this and will be backwards compatible thanks to pluggy.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/938/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 679779797,MDU6SXNzdWU2Nzk3Nzk3OTc=,939,extra_ plugin hooks should take the same arguments,9599,simonw,closed,0,,,,,6,2020-08-16T16:04:54Z,2020-08-16T18:25:05Z,2020-08-16T16:50:29Z,OWNER,,"- [x] `extra_css_urls(template, database, table, datasette)` - [x] `extra_js_urls(template, database, table, datasette)` - [x] `extra_body_script(template, database, table, view_name, datasette)` - [x] `extra_template_vars(template, database, table, view_name, request, datasette)` _Originally posted by @simonw in https://github.com/simonw/datasette/issues/938#issuecomment-674544691_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/939/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 679809281,MDExOlB1bGxSZXF1ZXN0NDY4NDg0MDMx,941,"Run CI on GitHub Actions, not Travis",9599,simonw,closed,0,,,,,1,2020-08-16T19:13:39Z,2020-08-18T05:09:36Z,2020-08-18T05:09:35Z,OWNER,simonw/datasette/pulls/941,Refs #940,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/941/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 681086659,MDU6SXNzdWU2ODEwODY2NTk=,47,emojis command,9599,simonw,closed,0,,,,,1,2020-08-18T14:26:26Z,2020-08-18T14:52:13Z,2020-08-18T14:52:13Z,MEMBER,,For fun - it can import https://api.github.com/emojis - maybe with an option to fetch the binary representations in addition to the URLs.,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/47/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 681334912,MDU6SXNzdWU2ODEzMzQ5MTI=,942,Support column descriptions in metadata.json,9599,simonw,closed,0,,,,,18,2020-08-18T20:52:00Z,2022-01-13T22:21:42Z,2021-08-12T23:53:24Z,OWNER,,"Could look something like this: ```json { ""title"": ""Five Thirty Eight"", ""license"": ""CC Attribution 4.0 License"", ""license_url"": ""https://creativecommons.org/licenses/by/4.0/"", ""source"": ""fivethirtyeight/data on GitHub"", ""source_url"": ""https://github.com/fivethirtyeight/data"", ""databases"": { ""fivethirtyeight"": { ""tables"": { ""mueller-polls/mueller-approval-polls"": { ""description_html"": ""

....

"", ""columns"": { ""name_of_column"": ""column_description goes here"" } ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/942/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 682184050,MDU6SXNzdWU2ODIxODQwNTA=,946,Exception in tracing code,9599,simonw,closed,0,,,,,1,2020-08-19T21:12:27Z,2020-09-15T20:16:50Z,2020-09-15T20:16:50Z,OWNER,,"When using `?_trace=1`: ``` Traceback (most recent call last): File ""/Users/simon/.local/share/virtualenvs/rockybeaches-09H592sC/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py"", line 390, in run_asgi result = await app(self.scope, self.receive, self.send) File ""/Users/simon/.local/share/virtualenvs/rockybeaches-09H592sC/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py"", line 45, in __call__ return await self.app(scope, receive, send) File ""/Users/simon/.local/share/virtualenvs/rockybeaches-09H592sC/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 150, in __call__ await self.app(scope, receive, send) File ""/Users/simon/.local/share/virtualenvs/rockybeaches-09H592sC/lib/python3.8/site-packages/datasette/tracer.py"", line 137, in __call__ await self.app(scope, receive, wrapped_send) File ""/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/contextlib.py"", line 120, in __exit__ next(self.gen) File ""/Users/simon/.local/share/virtualenvs/rockybeaches-09H592sC/lib/python3.8/site-packages/datasette/tracer.py"", line 63, in capture_traces del tracers[task_id] KeyError: 4575365856 ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/946/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 683804172,MDU6SXNzdWU2ODM4MDQxNzI=,134,--load-extension option for sqlite-utils query,9599,simonw,closed,0,,,,,4,2020-08-21T20:12:42Z,2020-08-21T21:06:26Z,2020-08-21T20:54:19Z,OWNER,,"I got this error: ``` % sqlite-utils calands.db 'create table superunits_with_maps_view_concrete as select * from superunits_with_maps_view' Traceback (most recent call last): ... cursor = db.conn.execute(sql, dict(param)) sqlite3.OperationalError: no such function: AsGeoJSON ``` A `--load-extension=/usr/local/lib/mod_spatialite.dylib` option (imitating the same option for Datasette) would help.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/134/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 683805434,MDU6SXNzdWU2ODM4MDU0MzQ=,135,Code for finding SpatiaLite in the usual locations,9599,simonw,closed,0,,,,,3,2020-08-21T20:15:34Z,2022-02-05T00:04:26Z,2020-08-21T20:30:13Z,OWNER,,"I built this for `shapefile-to-sqlite` but it would be useful in `sqlite-utils` too: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L16-L19 ```python SPATIALITE_PATHS = ( ""/usr/lib/x86_64-linux-gnu/mod_spatialite.so"", ""/usr/local/lib/mod_spatialite.dylib"", ) ``` https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L105-L109 ```python def find_spatialite(): for path in SPATIALITE_PATHS: if os.path.exists(path): return path return None ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/135/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 683812642,MDU6SXNzdWU2ODM4MTI2NDI=,136,--load-extension=spatialite shortcut option,9599,simonw,closed,0,,,,,3,2020-08-21T20:31:25Z,2022-02-05T00:04:26Z,2020-10-16T19:14:32Z,OWNER,,In conjunction with #135 - this would do the same thing as `--load-extension=path-to-spatialite` (see #134),140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/136/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 683830416,MDU6SXNzdWU2ODM4MzA0MTY=,137,--load-extension for other sqlite-utils commands,9599,simonw,closed,0,,,,,1,2020-08-21T21:12:56Z,2020-10-16T19:14:32Z,2020-10-16T19:14:32Z,OWNER,,"e.g. for this: ``` calands-datasette % sqlite-utils tables calands.db --counts [{""table"": ""spatial_ref_sys"", ""count"": 4924}, {""table"": ""spatialite_history"", ""count"": 14}, {""table"": ""sqlite_sequence"", ""count"": 1}, {""table"": ""geometry_columns"", ""count"": 2}, {""table"": ""spatial_ref_sys_aux"", ""count"": 4873}, {""table"": ""views_geometry_columns"", ""count"": 0}, {""table"": ""virts_geometry_columns"", ""count"": 0}, {""table"": ""geometry_columns_statistics"", ""count"": 2}, {""table"": ""views_geometry_columns_statistics"", ""count"": 0}, {""table"": ""virts_geometry_columns_statistics"", ""count"": 0}, {""table"": ""geometry_columns_field_infos"", ""count"": 0}, {""table"": ""views_geometry_columns_field_infos"", ""count"": 0}, {""table"": ""virts_geometry_columns_field_infos"", ""count"": 0}, {""table"": ""geometry_columns_time"", ""count"": 2}, {""table"": ""geometry_columns_auth"", ""count"": 2}, {""table"": ""views_geometry_columns_auth"", ""count"": 0}, {""table"": ""virts_geometry_columns_auth"", ""count"": 0}, Traceback (most recent call last): File ""/usr/local/bin/sqlite-utils"", line 8, in sys.exit(cli()) File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py"", line 829, in __call__ return self.main(*args, **kwargs) File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py"", line 782, in main rv = self.invoke(ctx) File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/click/core.py"", line 610, in invoke return callback(*args, **kwargs) File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 143, in tables for line in output_rows(_iter(), headers, nl, arrays, json_cols): File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 922, in output_rows for row, next_row in itertools.zip_longest(current_iter, next_iter): File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 123, in _iter row.append(db[name].count) File ""/usr/local/Cellar/sqlite-utils/2.15.1/libexec/lib/python3.8/site-packages/sqlite_utils/db.py"", line 458, in count return self.db.conn.execute( sqlite3.OperationalError: no such module: VirtualSpatialIndex ``` The `tables` command could take `--load-extension` too - as could `rows` and other similar commands. Follow-on from #134 ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/137/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 684118950,MDU6SXNzdWU2ODQxMTg5NTA=,138,extracts= doesn't configure foreign keys,9599,simonw,closed,0,,,,,2,2020-08-23T05:21:15Z,2020-09-24T22:47:01Z,2020-09-24T22:46:52Z,OWNER,,"In using `extracts=` for `shapefiles-to-sqlite` in https://github.com/simonw/shapefile-to-sqlite/issues/9 I've run into a couple of pretty serious flaws: - The columns in the original table are still `TEXT` even when the foreign key they are supposed to reference is an `INTEGER` - which means Datasette foreign key features don't actually work - Those foreign key relationships aren't setup automatically - creating them is left as an exercise for the developer",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/138/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 684961449,MDU6SXNzdWU2ODQ5NjE0NDk=,949,Try out CodeMirror SQL hints,9599,simonw,closed,0,,,,,5,2020-08-24T20:58:21Z,2023-11-03T05:28:58Z,2020-11-01T03:29:48Z,OWNER,,"> It would also be interesting to try out the SQL hint mode, which can autocomplete against tables and columns. This demo shows how to configure that: https://codemirror.net/mode/sql/ > > Some missing documentation: https://stackoverflow.com/questions/20023381/codemirror-how-add-tables-to-sql-hint _Originally posted by @simonw in https://github.com/simonw/datasette/issues/948#issuecomment-679355426_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/949/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 685806511,MDU6SXNzdWU2ODU4MDY1MTE=,950,Private/secret databases: database files that are only visible to plugins,9599,simonw,closed,0,,,,,6,2020-08-25T20:46:17Z,2023-08-24T22:26:09Z,2023-08-24T22:26:08Z,OWNER,,"In thinking about the best way to implement https://github.com/simonw/datasette-auth-passwords/issues/6 (SQL-backed user accounts for `datasette-auth-passwords`) I realized that there are a few different use-cases where a plugin might want to store data that isn't visible to regular Datasette users: - Storing password hashes - Storing API tokens - Storing secrets that are used for data import integrations (secrets for talking to the Twitter API for example) Idea: allow one or more private database files to be attached to Datasette, something like this: datasette github.db linkedin.db -s secrets.db -m metadata.yml The `secrets.db` file would not be visible using any of the Datasette's usual interface or API routes - but plugins would be able to run queries against it. So `datasette-auth-passwords` might then be configured like this: ```yaml plugins: datasette-auth-passwords: database: secrets sql: ""select password_hash from passwords where username = :username"" ``` The plugin could even refuse to operate against a database that hadn't been loaded as a secret database.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/950/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 688389933,MDU6SXNzdWU2ODgzODk5MzM=,143,Move to GitHub Actions CI,9599,simonw,closed,0,,,,,1,2020-08-28T22:34:11Z,2020-08-28T22:41:35Z,2020-08-28T22:41:35Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/143/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 688395275,MDU6SXNzdWU2ODgzOTUyNzU=,144,Run some tests against numpy,9599,simonw,closed,0,,,,,2,2020-08-28T22:53:00Z,2020-08-28T22:57:05Z,2020-08-28T22:57:04Z,OWNER,,"Accidentally removed in #143: https://github.com/simonw/sqlite-utils/blob/d7d3f962861ef32c5ead8f514c8756f5b6f7c4a0/.travis.yml#L18-L19",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/144/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 688427751,MDU6SXNzdWU2ODg0Mjc3NTE=,956,Push to Docker Hub failed - but it shouldn't run for alpha releases anyway,9599,simonw,closed,0,,,,,7,2020-08-29T01:09:12Z,2020-09-15T20:46:41Z,2020-09-15T20:36:34Z,OWNER,,"https://github.com/simonw/datasette/runs/1043709494?check_suite_focus=true - [x] This step should not run if a release is an alpha or beta - [x] When it DOES run it should work - [x] See it work for both an alpha and a non-alpha release, then close this ticket",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/956/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 689800307,MDU6SXNzdWU2ODk4MDAzMDc=,1,Add an index on the timestamp column,9599,simonw,closed,0,,,,,0,2020-09-01T04:33:37Z,2020-09-01T04:49:23Z,2020-09-01T04:49:23Z,MEMBER,,Since default view will likely be ordered by timestamp descending.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/1/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 689809225,MDU6SXNzdWU2ODk4MDkyMjU=,2,Apply porter stemming,9599,simonw,closed,0,,,,,2,2020-09-01T04:57:55Z,2020-09-01T20:42:00Z,2020-09-01T20:40:24Z,MEMBER,,This can be on by default. You can turn it off for a table in the config file using `stemming: none` - or maybe `tokenize: none` to match the terminology used by SQLite and `sqlite-utils`: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#enabling-full-text-search,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 689810340,MDU6SXNzdWU2ODk4MTAzNDA=,3,"Datasette plugin to provide custom page for running faceted, ranked searches",9599,simonw,closed,0,,,,,3,2020-09-01T05:00:22Z,2020-09-03T21:01:41Z,2020-09-03T21:01:41Z,MEMBER,,"This will be a page at `/-/beta` which renders using a custom template. It will offer a default timeline view plus search and facet by type/date.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 689839399,MDU6SXNzdWU2ODk4MzkzOTk=,4,Optimize the FTS table,9599,simonw,closed,0,,,,,1,2020-09-01T05:58:17Z,2020-09-01T06:10:08Z,2020-09-01T06:10:08Z,MEMBER,,,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 689847361,MDU6SXNzdWU2ODk4NDczNjE=,5,Add a context column that's not searchable,9599,simonw,closed,0,,,,,1,2020-09-01T06:13:42Z,2020-09-03T18:43:50Z,2020-09-03T18:43:50Z,MEMBER,,"I sometimes like to configure titles that are things like ""Comment on issue X"" or ""Photo in Golden Gate Park"" - these shouldn't be included in the search index but should be stored so they can be displayed to provide context. Add a column for this - probably called `context` - and make it so it can be populated.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/5/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 691265198,MDU6SXNzdWU2OTEyNjUxOTg=,7,"Mechanism for differentiating between ""by me"" and ""liked by me""",9599,simonw,closed,0,,,,,6,2020-09-02T17:44:37Z,2020-09-02T21:06:28Z,2020-09-02T21:06:28Z,MEMBER,,"Some of the content I'm indexing is by me - photos I've taken, tweets I wrote, commits, comments I posted. Some of it is stuff that I've ""liked"" or ""bookmarked"" in some way - favourited tweets, Pocket articles, starred GitHub repos. It woud be useful to be able to differentiate between the two.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 691369691,MDU6SXNzdWU2OTEzNjk2OTE=,8,Create a view for running faceted searches,9599,simonw,closed,0,,,,,1,2020-09-02T19:44:07Z,2020-09-02T19:50:47Z,2020-09-02T19:50:47Z,MEMBER,,"```sql select search_index_fts.rank, search_index.rowid, search_index.[table], search_index.key, search_index.title, search_index.timestamp, search_index.search_1 from search_index join search_index_fts on search_index.rowid = search_index_fts.rowid order by search_index_fts.rank, search_index.timestamp desc ```",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 691521965,MDU6SXNzdWU2OTE1MjE5NjU=,9,Mechanism for defining custom display of results,9599,simonw,closed,0,,,,,8,2020-09-03T00:14:07Z,2020-09-03T21:12:14Z,2020-09-03T21:09:55Z,MEMBER,,Part of #3 - in particular I want to make sure my photos are displayed with a thumbnail.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 691557547,MDU6SXNzdWU2OTE1NTc1NDc=,10,Category 3: received,9599,simonw,closed,0,,,,,1,2020-09-03T01:40:36Z,2020-09-03T17:38:51Z,2020-09-03T17:38:51Z,MEMBER,,"A category for things that were sent to me: DMs, emails etc. Follows #7.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 692125110,MDU6SXNzdWU2OTIxMjUxMTA=,11,Public / Private mechanism,9599,simonw,closed,0,,,,,1,2020-09-03T16:47:03Z,2020-09-03T17:33:52Z,2020-09-03T17:33:52Z,MEMBER,,"Some of the data in Dogsheep is stuff that was written publicly - tweets, blog posts, GitHub commits to public repos. Some of it is private data - emails, photos, direct messages, Swarm checkins, commits to private repos. Being able to filter for just one or the other (or both) would be useful. Especially when giving demos!",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/11/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 692386625,MDU6SXNzdWU2OTIzODY2MjU=,13,Support advanced FTS queries,9599,simonw,closed,0,,,,,1,2020-09-03T21:29:56Z,2020-09-03T21:40:51Z,2020-09-03T21:40:51Z,MEMBER,,`simon willison NOT screenshot` for example.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 693318095,MDU6SXNzdWU2OTMzMTgwOTU=,14,On FTS exception rerun the query with quoting,9599,simonw,closed,0,,,,,0,2020-09-04T15:44:18Z,2020-09-05T16:23:01Z,2020-09-05T16:23:01Z,MEMBER,,"Searching for eg `#dogfest` currently throws an FTS exception - but I want to support advanced FTS query tricks as seen in #13. https://dogsheep.simonwillison.net/-/beta?q=%23dogfest > fts5: syntax error near ""#"" Idea: catch that error and re-run the query with FTS escaping applied! ",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 694500679,MDU6SXNzdWU2OTQ1MDA2Nzk=,17,"Rename ""table"" to ""type""",9599,simonw,closed,0,,,,,2,2020-09-06T19:34:41Z,2020-09-09T03:03:22Z,2020-09-09T03:03:22Z,MEMBER,,"I think ""table"" is the wrong name for the concept I'm using it for here. Two reasons: firstly, `table` is a reserved word in SQLite. More importantly, it turns out there's not a direct mapping from tables to types of search result. In particular, for GitHub I ended up having two different ""tables"" of repositories - one for repos created by me, another for repos that I have starred.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/17/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 695276328,MDU6SXNzdWU2OTUyNzYzMjg=,148,More attractive indentation of created FTS table schema,9599,simonw,closed,0,,,,,1,2020-09-07T16:49:30Z,2020-09-07T18:12:50Z,2020-09-07T18:12:50Z,OWNER,,"On https://github-to-sqlite.dogsheep.net/github/licenses_fts the create table SQL is displayed as: ```sql CREATE VIRTUAL TABLE [licenses_fts] USING FTS5 ( [name], content=[licenses] ); ``` It would be more aesthetically pleasing if it looked like this: ```sql CREATE VIRTUAL TABLE [licenses_fts] USING FTS5 ( [name], content=[licenses] ); ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/148/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 695319258,MDU6SXNzdWU2OTUzMTkyNTg=,149,"FTS table with 7 rows has _fts_docsize table with 9,141 rows",9599,simonw,closed,0,,,,,10,2020-09-07T18:06:16Z,2020-09-07T21:16:34Z,2020-09-07T21:16:34Z,OWNER,,"I'm seeing a weird issue with some of the SQLite databases that I am using with the FTS5 module. I have a database with a `licenses` table that contains 7 rows: The FTS table also has 7 rows: Somehow the accompanying `licenses_fts_docsize` shadow table now has 9,141 rows in it! And `licenses_fts_data` has 41 rows - should I expect that to have 7 rows? I have a hunch that it might be a problem with the triggers. These are the triggers that are updating that FTS table: | type | name | tbl_name | rootpage | sql | | --- | --- | --- | --- | --- | | trigger | licenses_ai | licenses | 0 | `CREATE TRIGGER [licenses_ai] AFTER INSERT ON [licenses] BEGIN INSERT INTO [licenses_fts] (rowid, [name]) VALUES (new.rowid, new.[name]); END` | | trigger | licenses_ad | licenses | 0 | `CREATE TRIGGER [licenses_ad] AFTER DELETE ON [licenses] BEGIN INSERT INTO [licenses_fts] ([licenses_fts], rowid, [name]) VALUES('delete', old.rowid, old.[name]); END` | | trigger | licenses_au | licenses | 0 | `CREATE TRIGGER [licenses_au] AFTER UPDATE ON [licenses] BEGIN INSERT INTO [licenses_fts] ([licenses_fts], rowid, [name]) VALUES('delete', old.rowid, old.[name]); INSERT INTO [licenses_fts] (rowid, [name]) VALUES (new.rowid, new.[name]); END` |",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/149/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 695359607,MDU6SXNzdWU2OTUzNTk2MDc=,150,Feature for tracing SQL queries,9599,simonw,closed,0,,,,,0,2020-09-07T19:43:08Z,2020-09-07T21:57:01Z,2020-09-07T21:57:01Z,OWNER,,"Debugging `sqlite-utils` when something weird happens (e.g. #149) can be a bit tricky since it runs a bunch of different SQL statements behind the scenes. An optional ""tracing"" mechanism for seeing what SQL is being executed would be useful.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/150/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 695360889,MDExOlB1bGxSZXF1ZXN0NDgxNjE2NzA0,151,Tracer mechanism for seeing underlying SQL,9599,simonw,closed,0,,,,,0,2020-09-07T19:46:43Z,2020-09-07T21:57:00Z,2020-09-07T21:57:00Z,OWNER,simonw/sqlite-utils/pulls/151,"Refs #150. Needs tests and documentation, including for the new `db.execute()` and `db.executescript()` methods.",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/151/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 695376054,MDU6SXNzdWU2OTUzNzYwNTQ=,152,Turn on recursive_triggers by default,9599,simonw,closed,0,,,,,2,2020-09-07T20:26:36Z,2020-09-07T21:17:48Z,2020-09-07T20:45:14Z,OWNER,,"https://www.sqlite.org/pragma.html#pragma_recursive_triggers says: > Prior to SQLite [version 3.6.18](https://www.sqlite.org/releaselog/3_6_18.html) (2009-09-11), recursive triggers were not supported. The behavior of SQLite was always as if this pragma was set to OFF. Support for recursive triggers was added in version 3.6.18 but was initially turned OFF by default, for compatibility. Recursive triggers may be turned on by default in future versions of SQLite. So I think the fix for the complex issue in #149 is to turn on `recursive_triggers` globally by default for `sqlite-utils`. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688499924_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/152/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 695377804,MDU6SXNzdWU2OTUzNzc4MDQ=,153,table.optimize() should delete junk rows from *_fts_docsize,9599,simonw,closed,0,,,,,3,2020-09-07T20:31:09Z,2020-09-24T20:35:46Z,2020-09-07T21:16:33Z,OWNER,,"> The second challenge here is cleaning up all of those junk rows in existing `*_fts_docsize` tables. Doing that just to the demo database from https://github-to-sqlite.dogsheep.net/github.db dropped its size from 22MB to 16MB! Here's the SQL: > ```sql > DELETE FROM [licenses_fts_docsize] WHERE id NOT IN ( > SELECT rowid FROM [licenses_fts]); > ``` > I can do that as part of the existing `table.optimize()` method, which optimizes FTS tables. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/149#issuecomment-688501064_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/153/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 696045581,MDU6SXNzdWU2OTYwNDU1ODE=,155,rebuild-fts command and table.rebuild_fts() method,9599,simonw,closed,0,,,,,2,2020-09-08T17:19:26Z,2020-09-24T20:35:46Z,2020-09-08T23:16:10Z,OWNER,,"https://sqlite.org/forum/forumpost/fa777fff86 > Easiest thing would be to run a 'rebuild' to rebuild the FTS index from scratch based on the contents of the content table. i.e. > > INSERT INTO licenses_fts(licenses_fts) VALUES('rebuild'); > > https://www.sqlite.org/fts5.html#the_rebuild_command",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/155/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 701584448,MDU6SXNzdWU3MDE1ODQ0NDg=,966,Remove _request_ip example from canned queries documentation,9599,simonw,closed,0,,,,,0,2020-09-15T03:51:33Z,2020-09-15T03:52:45Z,2020-09-15T03:52:45Z,OWNER,,"`_request_ip` isn't valid, so it shouldn't be in the example: https://github.com/simonw/datasette/blob/cb515a9d75430adaf5e545a840bbc111648e8bfd/docs/sql_queries.rst#L320-L322",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/966/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 702069429,MDU6SXNzdWU3MDIwNjk0Mjk=,967,Writable canned queries with magic parameters fail if POST body is empty,9599,simonw,closed,0,,,,,11,2020-09-15T16:14:43Z,2020-09-15T20:13:10Z,2020-09-15T20:13:10Z,OWNER,,"When I try to use the new `?_json=1` feature from #880 with magic parameters from #842 I get this error: > Incorrect number of bindings supplied. The current statement uses 1, and there are 0 supplied",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/967/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 703951918,MDU6SXNzdWU3MDM5NTE5MTg=,21,Option to sort search results by date,9599,simonw,closed,0,,,,,0,2020-09-17T22:32:39Z,2020-09-17T22:55:35Z,2020-09-17T22:55:35Z,MEMBER,,"Sometimes I want to sort by date, not by relevance.",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/21/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 703962917,MDU6SXNzdWU3MDM5NjI5MTc=,22,Bug: UI says sorted by relevance in timeline view,9599,simonw,closed,0,,,,,0,2020-09-17T23:02:07Z,2020-09-17T23:13:14Z,2020-09-17T23:13:14Z,MEMBER,,"In regular timeline view sort defaults to newest, not relevance - so this UI is incorrect: ",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/22/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 703970713,MDU6SXNzdWU3MDM5NzA3MTM=,23,Sort option should persist between multiple searches,9599,simonw,closed,0,,,,,0,2020-09-17T23:21:26Z,2020-09-18T22:39:12Z,2020-09-18T22:39:12Z,MEMBER,,Following #21 ,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/23/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 703970814,MDU6SXNzdWU3MDM5NzA4MTQ=,24,"the JSON object must be str, bytes or bytearray, not 'Undefined'",9599,simonw,closed,0,,,,,8,2020-09-17T23:21:41Z,2020-09-18T22:33:32Z,2020-09-18T22:33:32Z,MEMBER,,Got this on a search results page.,197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 704685890,MDU6SXNzdWU3MDQ2ODU4OTA=,25,template_debug mechanism,9599,simonw,closed,0,,,,,2,2020-09-18T22:11:09Z,2020-09-18T22:12:21Z,2020-09-18T22:12:03Z,MEMBER,,"> I'd prefer it if errors in these template fragments were displayed as errors inline where the fragment should have been inserted, rather than 500ing the whole page - especially since the template fragments are user-provided and could have all kinds of odd errors in them which should be as easy to debug as possible. _Originally posted by @simonw in https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694554584_",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 705827457,MDU6SXNzdWU3MDU4Mjc0NTc=,971,Support the dbstat table,9599,simonw,closed,0,,,,,7,2020-09-21T18:38:53Z,2020-09-21T19:00:02Z,2020-09-21T18:59:52Z,OWNER,,"`dbstat` is a table that is usually available on SQLite giving statistics about the database. For example: https://fivethirtyeight.datasettes.com/fivethirtyeight?sql=SELECT+*+FROM+%22dbstat%22+WHERE+name%3D%27bachelorette%2Fbachelorette%27%3B | name | path | pageno | pagetype | ncell | payload | unused | mx_payload | pgoffset | pgsize | |---------------------------|--------|----------|------------|---------|-----------|----------|--------------|------------|----------| | bachelorette/bachelorette | / | 89 | internal | 13 | 0 | 3981 | 0 | 360448 | 4096 | | bachelorette/bachelorette | /000/ | 91 | leaf | 66 | 3792 | 32 | 74 | 368640 | 4096 | | bachelorette/bachelorette | /001/ | 92 | leaf | 67 | 3800 | 14 | 74 | 372736 | 4096 | | bachelorette/bachelorette | /002/ | 93 | leaf | 65 | 3717 | 46 | 70 | 376832 | 4096 | | bachelorette/bachelorette | /003/ | 94 | leaf | 68 | 3742 | 6 | 71 | 380928 | 4096 | | bachelorette/bachelorette | /004/ | 95 | leaf | 70 | 3696 | 42 | 66 | 385024 | 4096 | | bachelorette/bachelorette | /005/ | 96 | leaf | 69 | 3721 | 22 | 71 | 389120 | 4096 | | bachelorette/bachelorette | /006/ | 97 | leaf | 70 | 3737 | 1 | 72 | 393216 | 4096 | | bachelorette/bachelorette | /007/ | 98 | leaf | 69 | 3728 | 15 | 69 | 397312 | 4096 | | bachelorette/bachelorette | /008/ | 99 | leaf | 73 | 3715 | 8 | 64 | 401408 | 4096 | | bachelorette/bachelorette | /009/ | 100 | leaf | 73 | 3705 | 18 | 62 | 405504 | 4096 | | bachelorette/bachelorette | /00a/ | 101 | leaf | 75 | 3681 | 32 | 62 | 409600 | 4096 | | bachelorette/bachelorette | /00b/ | 102 | leaf | 77 | 3694 | 9 | 62 | 413696 | 4096 | | bachelorette/bachelorette | /00c/ | 103 | leaf | 74 | 3673 | 45 | 62 | 417792 | 4096 | | bachelorette/bachelorette | /00d/ | 104 | leaf | 5 | 228 | 3835 | 48 | 421888 | 4096 | Other than direct `select * from dbsat` queries it is completely invisible. It would be cool if https://fivethirtyeight.datasettes.com/fivethirtyeight/dbstat didn't 404 (on databases for which that table was available).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/971/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 705995722,MDU6SXNzdWU3MDU5OTU3MjI=,162,A decorator for registering custom SQL functions,9599,simonw,closed,0,,,,,2,2020-09-22T00:18:32Z,2020-09-22T00:40:44Z,2020-09-22T00:32:17Z,OWNER,,"Syntactic sugar for `db.conn.create_function` - it would work something like this: ```python db = sqlite_utils.Database(""mydb.db"") @db.register_function def scramble(text): chars = list(text) random.shuffle(chars) return """".join(chars) ``` The decorator would inspect the function to find its name and arity (number of arguments). Having run the above you could then do: ```python db.execute(""select scramble('hello')"").fetchall() ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/162/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 706092617,MDExOlB1bGxSZXF1ZXN0NDkwNzAzMTcz,166,Keyword only arguments for transform(),9599,simonw,closed,0,,,,,0,2020-09-22T05:41:44Z,2020-09-22T06:39:11Z,2020-09-22T06:39:11Z,OWNER,simonw/sqlite-utils/pulls/166,Refs #165,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/166/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 706167456,MDU6SXNzdWU3MDYxNjc0NTY=,168,Automate (as much as possible) updates published to Homebrew,9599,simonw,closed,0,,,,,2,2020-09-22T08:08:37Z,2020-11-09T07:43:30Z,2020-11-09T07:43:30Z,OWNER,,I'd like to get new `sqlite-utils` (and Datasette) releases submitted to Homebrew as painlessly as possible.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/168/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 707427200,MDU6SXNzdWU3MDc0MjcyMDA=,172,Improve performance of extract operations,9599,simonw,closed,0,,,,,9,2020-09-23T14:40:50Z,2020-09-24T15:43:57Z,2020-09-24T15:43:57Z,OWNER,,"This command took about 12 minutes (against a 150MB file with 680,000 rows in it): ``` sqlite-utils extract salaries.db salaries \ 'Organization Group Code' 'Organization Group' \ --table 'organization_groups' \ --fk-column 'organization_group_id' \ --rename 'Organization Group Code' code \ --rename 'Organization Group' name ``` I'm pretty confident we can do better than that.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/172/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 707478649,MDU6SXNzdWU3MDc0Nzg2NDk=,173,Progress bar for sqlite-utils insert,9599,simonw,closed,0,,,,,6,2020-09-23T15:43:56Z,2021-11-01T08:42:24Z,2020-10-27T18:16:04Z,OWNER,,"It would be nice if `sqlite-utils insert` had a progress bar, for when it's churning through huge CSV files.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/173/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 707944044,MDExOlB1bGxSZXF1ZXN0NDkyMjU3NDA1,174,"Much, much faster extract() implementation",9599,simonw,closed,0,,,,,7,2020-09-24T07:52:31Z,2020-09-24T15:44:00Z,2020-09-24T15:43:56Z,OWNER,simonw/sqlite-utils/pulls/174,Takes my test down from ten minutes to four seconds. Refs #172.,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/174/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 708261775,MDU6SXNzdWU3MDgyNjE3NzU=,175,Add docs for .transform(column_order=),9599,simonw,closed,0,,,,,3,2020-09-24T15:19:04Z,2020-09-24T20:35:48Z,2020-09-24T16:00:56Z,OWNER,,"> Need to update docs for `.transform()` now that `column_order=` is available. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/174#discussion_r494403327_ Maybe also add this as an option to `sqlite-utils transform` - since reordering columns is actually a pretty nice capability.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/175/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 708289783,MDU6SXNzdWU3MDgyODk3ODM=,976,Idea: -o could open to a more convenient location,9599,simonw,closed,0,,,,,2,2020-09-24T15:56:35Z,2020-10-26T05:07:10Z,2020-10-26T05:06:26Z,OWNER,,"> Idea: if a database only has a single table, this could open straight to `/db/table`. If it has multiple tables but a single database it could open straight to `/db`. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/970#issuecomment-698434236_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/976/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 708293114,MDU6SXNzdWU3MDgyOTMxMTQ=,176,sqlite-utils transform column order option,9599,simonw,closed,0,,,,,2,2020-09-24T16:01:21Z,2020-09-24T20:34:51Z,2020-09-24T16:11:59Z,OWNER,,Split from #175,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/176/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 708301810,MDU6SXNzdWU3MDgzMDE4MTA=,177,Simplify .transform(drop_foreign_keys=) and sqlite-transform --drop-foreign-key,9599,simonw,closed,0,,,,,1,2020-09-24T16:13:50Z,2020-09-24T20:35:03Z,2020-09-24T16:19:13Z,OWNER,,"These both currently require you to provide three strings, for `column`, `other_table`, `other_column`. Just providing `column` should be enough information.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/177/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 709577625,MDU6SXNzdWU3MDk1Nzc2MjU=,179,sqlite-utils transform/insert --detect-types,9599,simonw,closed,0,,,,,4,2020-09-26T17:28:55Z,2021-06-19T03:36:16Z,2021-06-19T03:36:05Z,OWNER,,"Idea from https://github.com/simonw/datasette-edit-tables/issues/13 - provide Python utility methods and accompanying CLI options for detecting the likely types of TEXT columns. So if you have a text column that actually contained exclusively integer string values, it can let you know and let you run transform against it.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/179/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 709861194,MDU6SXNzdWU3MDk4NjExOTQ=,180,Try running some tests using Hypothesis,9599,simonw,closed,0,,,,,1,2020-09-28T01:11:30Z,2020-10-19T04:51:55Z,2020-10-19T04:51:55Z,OWNER,,Inspired by this Twitter conversation: https://twitter.com/simonw/status/1310386009465479168,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/180/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 709920027,MDU6SXNzdWU3MDk5MjAwMjc=,181,"pk=[""id""] should have same effect as pk=""id""",9599,simonw,closed,0,,,,,1,2020-09-28T04:28:07Z,2020-10-14T21:59:47Z,2020-10-14T21:59:47Z,OWNER,,"``` In [11]: db['one'].insert({""id"": 1, ""name"": ""oentuh""}, pk=""id"") Out[11]: In [12]: db['two'].insert({""id"": 1, ""name"": ""oentuh""}, pk=[""id""]) Out[12]:
In [13]: db['one'].schema Out[13]: 'CREATE TABLE [one] (\n [id] INTEGER PRIMARY KEY,\n [name] TEXT\n)' In [14]: db['two'].schema Out[14]: 'CREATE TABLE [two] (\n [id] INTEGER,\n [name] TEXT\n)' ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/181/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 710650633,MDU6SXNzdWU3MTA2NTA2MzM=,979,Default table view JSON should include CREATE TABLE,9599,simonw,closed,0,,,,,3,2020-09-28T23:54:58Z,2023-01-09T15:32:39Z,2023-01-09T15:32:22Z,OWNER,,"https://latest.datasette.io/fixtures/facetable.json doesn't currently include the CREATE TABLE statement for the page, even though it's available on the HTML version at https://latest.datasette.io/fixtures/facetable",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/979/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 712316959,MDU6SXNzdWU3MTIzMTY5NTk=,183,Try out GitHub code scanning,9599,simonw,closed,0,,,,,1,2020-09-30T22:16:14Z,2020-09-30T22:23:44Z,2020-09-30T22:23:44Z,OWNER,,https://github.blog/2020-09-30-code-scanning-is-now-available/,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/183/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 715146588,MDExOlB1bGxSZXF1ZXN0NDk4MTA0ODMw,994,Run tests against Python 3.9,9599,simonw,closed,0,,,,,1,2020-10-05T20:40:13Z,2020-10-09T16:22:51Z,2020-10-09T16:22:50Z,OWNER,simonw/datasette/pulls/994,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/994/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 716955793,MDExOlB1bGxSZXF1ZXN0NDk5NjAzMzU5,184,Test against Python 3.9,9599,simonw,closed,0,,,,,0,2020-10-08T01:37:05Z,2020-10-08T01:44:06Z,2020-10-08T01:44:06Z,OWNER,simonw/sqlite-utils/pulls/184,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/184/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 718484082,MDU6SXNzdWU3MTg0ODQwODI=,1010,json / CSV links are broken in Datasette 0.50,9599,simonw,closed,0,,,,,3,2020-10-10T00:07:42Z,2020-10-10T02:32:03Z,2020-10-10T02:32:03Z,OWNER,,"e.g. on https://latest.datasette.io/fixtures/sortable That export link block is broken. The HTML is: ```html

This data as json, CSV (advanced)

```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1010/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 718938046,MDU6SXNzdWU3MTg5MzgwNDY=,2,Convert dates to a better format,9599,simonw,closed,0,,,,,0,2020-10-11T22:12:33Z,2020-10-11T23:15:03Z,2020-10-11T23:15:03Z,MEMBER,,"They currently look like this: https://github.com/dogsheep/evernote-to-sqlite/blob/9d8efd17580f6ddf76745c145d1e69dd24e52b64/tests/test_evernote_to_sqlite.py#L35-L36",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/2/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 718938321,MDU6SXNzdWU3MTg5MzgzMjE=,3,Use a content hash for the note IDs,9599,simonw,closed,0,,,,,0,2020-10-11T22:13:46Z,2020-10-11T23:15:04Z,2020-10-11T23:15:04Z,MEMBER,,"Without a GUID note IDs are pretty ineffective, but using a hash of the contents will at least avoid creating identical duplicates in the future. https://sqlite-utils.readthedocs.io/en/stable/python-api.html#setting-an-id-based-on-the-hash-of-the-row-contents",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/3/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 718938508,MDU6SXNzdWU3MTg5Mzg1MDg=,4,Configure FTS + add an index on the date columns,9599,simonw,closed,0,,,,,2,2020-10-11T22:14:40Z,2020-10-11T23:41:29Z,2020-10-11T23:41:29Z,MEMBER,,"Sort by date descending is likely the most common way of sorting, so that column should be indexed. Also add FTS configuration for both notes and the OCR column on resources.",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 718949182,MDU6SXNzdWU3MTg5NDkxODI=,6,Better handling of OCR data,9599,simonw,closed,0,,,,,2,2020-10-11T23:20:52Z,2020-10-12T00:04:10Z,2020-10-12T00:04:10Z,MEMBER,,"> I haven't done the FTS on OCR yet. I'm going to move that to another ticket because it requires more thought. _Originally posted by @simonw in https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706784028_",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 718952107,MDU6SXNzdWU3MTg5NTIxMDc=,185,Use db[table] consistently in documentation,9599,simonw,closed,0,,,,,0,2020-10-11T23:39:12Z,2020-10-12T00:13:41Z,2020-10-12T00:13:41Z,OWNER,,"The Python docs have a bunch of examples like this: https://sqlite-utils.readthedocs.io/en/stable/python-api.html ```python dogs.enable_fts([""name"", ""twitter""], create_triggers=True) ``` This would be easier for people to understand if it looked like this instead: ```python db[""dogs""].enable_fts([""name"", ""twitter""], create_triggers=True) ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/185/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 723460107,MDU6SXNzdWU3MjM0NjAxMDc=,187,Maybe: Utility method / CLI tool for initializing SpatiaLite,9599,simonw,closed,0,,,,,2,2020-10-16T19:04:03Z,2022-02-05T00:04:26Z,2020-10-16T19:15:13Z,OWNER,,"> I think this should initialize SpatiaLite against the current database if it has not been initialized already. > > Relevant code: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L112-L126",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/187/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 723861683,MDU6SXNzdWU3MjM4NjE2ODM=,28,Switch to using datasette.client,9599,simonw,closed,0,,,,,1,2020-10-17T22:42:26Z,2020-10-17T23:00:47Z,2020-10-17T23:00:47Z,MEMBER,,"`datasette.client` is designed for this kind of thing, to replace this code: https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L223-L232",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/28/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 724264574,MDU6SXNzdWU3MjQyNjQ1NzQ=,52,Option to fetch README and/or HTML-rendered README for repos,9599,simonw,closed,0,,,,,0,2020-10-19T05:10:24Z,2020-10-19T05:33:42Z,2020-10-19T05:33:42Z,MEMBER,,"I'm thinking: github-to-sqlite repos ... --readme # Populates readme column with raw text github-to-sqlite repos ... --readme-html # Populates readme_html column with raw HTML https://developer.github.com/v3/repos/contents/#get-a-repository-readme",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/52/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 729096595,MDU6SXNzdWU3MjkwOTY1OTU=,1051,Better display of binary data on arbitrary query results page,9599,simonw,closed,0,,,,,6,2020-10-25T19:38:06Z,2020-10-29T22:12:16Z,2020-10-29T22:01:39Z,OWNER,,"https://latest.datasette.io/fixtures?sql=select+rowid%2C+data+from+binary_data+order+by+rowid+limit+101 Problem: if these were larger fields that HTML page could have multiple megabytes of Python binary string representations on it. It should behave more like the regular table view does: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1051/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 730693696,MDExOlB1bGxSZXF1ZXN0NTEwOTU2MTM0,190,Progress bar for sqlite-utils insert command,9599,simonw,closed,0,,,,,0,2020-10-27T18:08:53Z,2020-10-27T18:16:03Z,2020-10-27T18:16:03Z,OWNER,simonw/sqlite-utils/pulls/190,Refs #173,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/190/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 731740458,MDU6SXNzdWU3MzE3NDA0NTg=,191,Idea: @db.register_function(deterministic=True),9599,simonw,closed,0,,,,,2,2020-10-28T19:45:18Z,2020-10-28T21:31:06Z,2020-10-28T21:31:06Z,OWNER,,"Python 3.8 added a `deterministic` parameter to `db.create_function()`: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.create_function `sqlite-utils` could expose this in the decorator, only actually applying it if the Python version supports it (using feature detection) - since nothing will break if it's not applied. https://sqlite-utils.readthedocs.io/en/stable/python-api.html#registering-custom-sql-functions ```python db = Database(memory=True) @db.register_function(deterministic=True) def reverse_string(s): return """".join(reversed(list(s))) ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/191/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 735663855,MDExOlB1bGxSZXF1ZXN0NTE1MDE0ODgz,195,table.search() improvements plus sqlite-utils search command,9599,simonw,closed,0,,,,,3,2020-11-03T22:02:08Z,2020-11-06T18:30:49Z,2020-11-06T18:30:42Z,OWNER,simonw/sqlite-utils/pulls/195,Refs #192. Still needs tests.,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/195/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 736520310,MDU6SXNzdWU3MzY1MjAzMTA=,196,Introspect if table is FTS4 or FTS5,9599,simonw,closed,0,,,,,19,2020-11-05T00:45:50Z,2020-11-05T03:54:07Z,2020-11-05T03:54:07Z,OWNER,,"> I want `.search()` to work against both FTS5 and FTS4 tables - but sort by rank should only work for FTS5. > > This means I need to be able to introspect and tell if a table is FTS4 or FTS5. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/192#issuecomment-722054264_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/196/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 737476423,MDU6SXNzdWU3Mzc0NzY0MjM=,198,Support order by relevance against FTS4,9599,simonw,closed,0,,,,,6,2020-11-06T05:36:31Z,2020-11-06T18:30:44Z,2020-11-06T18:30:44Z,OWNER,,"For #192 and #197 I've decided I want to be able to order by relevance in FTS4 as well as FTS5. This means I need to port over my work on bm25() from https://github.com/simonw/sqlite-fts4 (since I don't want to add a full dependency).",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/198/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 737855731,MDU6SXNzdWU3Mzc4NTU3MzE=,199,"@db.register_function(..., replace=False) to avoid double-registering custom functions",9599,simonw,closed,0,,,,,1,2020-11-06T15:39:21Z,2020-11-06T18:30:44Z,2020-11-06T18:30:44Z,OWNER,,"I'd like a mechanism to optionally avoid registering a custom function if it has already been registered. SQLite doesn't seem to offer a way to introspect registered custom functions so I'll need to track what has already been registered in `sqlite-utils` instead. > Should I register the custom `rank_bm25` SQLite function for every connection, or should I register it against the connection just the first time the user attempts an FTS4 search? I think I'd rather register it only if it is needed. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/198#issuecomment-723145383_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/199/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 738514367,MDU6SXNzdWU3Mzg1MTQzNjc=,202,sqlite-utils insert -f colname - for configuring full-text search,9599,simonw,closed,0,,,,,2,2020-11-08T17:30:09Z,2021-01-03T05:00:36Z,2021-01-03T05:00:27Z,OWNER,,"A mechanism for specifying columns that should be configured for full-text search as part of the initial data import: sqlite-utils insert mydb.db articles articles.csv --csv -f title -f body",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/202/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 740512882,MDExOlB1bGxSZXF1ZXN0NTE4OTg4ODc5,1085,Use FTS4 in fixtures,9599,simonw,closed,0,,,,,1,2020-11-11T06:44:30Z,2020-11-12T00:02:59Z,2020-11-12T00:02:58Z,OWNER,simonw/datasette/pulls/1085,Refs #1081,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1085/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 742041667,MDU6SXNzdWU3NDIwNDE2Njc=,1092,Make cascading permission checks available to plugins,9599,simonw,closed,0,,,,,1,2020-11-13T01:02:55Z,2023-08-30T22:17:42Z,2023-08-30T22:17:41Z,OWNER,,"The `BaseView` class has a method for cascading permission checks, but it's not easily accessible to plugins. https://github.com/simonw/datasette/blob/5eb8e9bf250b26e30b017d39a392c33973997656/datasette/views/base.py#L75-L99 This leaves plugins like `datasette-graphql` having to implement their own versions of this logic, which is bad: https://github.com/simonw/datasette-graphql/issues/65 > First check `view-database` - if that says `False` then disallow access, if it says `True` then allow access. If it says `None` check `view-instance`. This should become a supported API that plugins are encouraged to use.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1092/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 743369188,MDExOlB1bGxSZXF1ZXN0NTIxMjc2Mjk2,1097,Use f-strings,9599,simonw,closed,0,,,,,1,2020-11-15T23:12:36Z,2020-11-15T23:24:24Z,2020-11-15T23:24:23Z,OWNER,simonw/datasette/pulls/1097,"Since Datasette now requires Python 3.6, how about some f-strings? I ran this in the `datasette` root checkout: ``` pip install flynt flynt . black . ```",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1097/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 743370900,MDU6SXNzdWU3NDMzNzA5MDA=,1098,Foreign key links break for compound foreign keys,9599,simonw,closed,0,,,,,5,2020-11-15T23:22:14Z,2020-11-29T19:50:31Z,2020-11-29T19:30:23Z,OWNER,,"Reported on Twitter here: https://twitter.com/ZaneSelvans/status/1328093641395548161 > Maybe I'm doing something wrong here but the automatically generated links based foreign key relationships seem to be working here for utility_id_eia, but not for plant_id_eia & generator_id which seems odd: https://pudl-datasette-xl7xwcpe2a-uc.a.run.app/pudl/generators_eia860 > > Right now it seems like they're trying to, but with only one of the two keys, so it gives ""Error 500. You did not supply a value for binding 2."" Maybe only create the links when it's a simple foreign key?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1098/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 749289611,MDU6SXNzdWU3NDkyODk2MTE=,1102,Plugin testing docs should show datasette.client,9599,simonw,closed,0,,,,,1,2020-11-24T02:34:46Z,2020-11-29T07:47:22Z,2020-11-29T07:44:58Z,OWNER,,https://docs.datasette.io/en/stable/testing_plugins.html currently shows how to use HTTPX directly.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1102/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 752789159,MDU6SXNzdWU3NTI3ODkxNTk=,1113,500 error on row page if query against foreign keys hits time limit,9599,simonw,closed,0,,,,,1,2020-11-28T23:20:08Z,2020-11-29T02:40:01Z,2020-11-28T23:23:31Z,OWNER,,"This page exhibited the following error: https://data.catalyst.coop/ferc1/f1_respondent_id/145 `(OperationalError('interrupted'), 'select (select count(*) from f1_acb_epda where respondent_id=:id), (select count(*) from f1_accumdepr_prvsn where respondent_id=:id), (select count(*) from f1_accumdfrrdtaxcr where respondent_id=:id), (select count(*) from f1_adit_190_detail where respondent_id=:id), (select count(*) from f1_adit_190_notes where respondent_id=:id), (select count(*) from f1_adit_amrt_prop where respondent_id=:id), (select count(*) from f1_adit_other where respondent_id=:id), (select count(*) from f1_adit_other_prop where respondent_id=:id), (select count(*) from f1_allowances where respondent_id=:id), (select count(*) from f1_bal_sheet_cr where respondent_id=:id), (select count(*) from f1_capital_stock where respondent_id=:id), (select count(*) from f1_cash_flow where respondent_id=:id), (select count(*) from f1_cmmn_utlty_p_e where respondent_id=:id), (select count(*) from f1_comp_balance_db where respondent_id=:id), (select count(*) from f1_construction where respondent_id=:id), (select count(*) from f1_control_respdnt where respondent_id=:id), (select count(*) from f1_co_directors where respondent_id=:id), (select count(*) from f1_cptl_stk_expns where respondent_id=:id), (select count(*) from f1_csscslc_pcsircs where respondent_id=:id), (select count(*) from f1_dacs_epda where respondent_id=:id), (select count(*) from f1_dscnt_cptl_stk where respondent_id=:id), (select count(*) from f1_edcfu_epda where respondent_id=:id), (select count(*) from f1_elctrc_erg_acct where respondent_id=:id), (select count(*) from f1_elctrc_oper_rev where respondent_id=:id), (select count(*) from f1_elc_oper_rev_nb where respondent_id=:id), (select count(*) from f1_elc_op_mnt_expn where respondent_id=:id), (select count(*) from f1_electric where respondent_id=:id), (select count(*) from f1_envrnmntl_expns where respondent_id=:id), (select count(*) from f1_envrnmntl_fclty where respondent_id=:id), (select count(*) from f1_fuel where respondent_id=:id), (select count(*) from f1_general_info where respondent_id=:id), (select count(*) from f1_gnrt_plant where respondent_id=:id), (select count(*) from f1_important_chg where respondent_id=:id), (select count(*) from f1_incm_stmnt_2 where respondent_id=:id), (select count(*) from f1_income_stmnt where respondent_id=:id), (select count(*) from f1_miscgen_expnelc where respondent_id=:id), (select count(*) from f1_misc_dfrrd_dr where respondent_id=:id), (select count(*) from f1_mthly_peak_otpt where respondent_id=:id), (select count(*) from f1_mtrl_spply where respondent_id=:id), (select count(*) from f1_nbr_elc_deptemp where respondent_id=:id), (select count(*) from f1_nonutility_prop where respondent_id=:id), (select count(*) from f1_nuclear_fuel where respondent_id=:id), (select count(*) from f1_officers_co where respondent_id=:id), (select count(*) from f1_othr_dfrrd_cr where respondent_id=:id), (select count(*) from f1_othr_pd_in_cptl where respondent_id=:id), (select count(*) from f1_othr_reg_assets where respondent_id=:id), (select count(*) from f1_othr_reg_liab where respondent_id=:id), (select count(*) from f1_overhead where respondent_id=:id), (select count(*) from f1_pccidica where respondent_id=:id), (select count(*) from f1_plant_in_srvce where respondent_id=:id), (select count(*) from f1_pumped_storage where respondent_id=:id), (select count(*) from f1_purchased_pwr where respondent_id=:id), (select count(*) from f1_reconrpt_netinc where respondent_id=:id), (select count(*) from f1_reg_comm_expn where respondent_id=:id), (select count(*) from f1_respdnt_control where respondent_id=:id), (select count(*) from f1_retained_erng where respondent_id=:id), (select count(*) from f1_r_d_demo_actvty where respondent_id=:id), (select count(*) from f1_sales_by_sched where respondent_id=:id), (select count(*) from f1_sale_for_resale where respondent_id=:id), (select count(*) from f1_sbsdry_totals where respondent_id=:id), (select count(*) from f1_schedules_list where respondent_id=:id), (select count(*) from f1_security_holder where respondent_id=:id), (select count(*) from f1_slry_wg_dstrbtn where respondent_id=:id), (select count(*) from f1_substations where respondent_id=:id), (select count(*) from f1_taxacc_ppchrgyr where respondent_id=:id), (select count(*) from f1_unrcvrd_cost where respondent_id=:id), (select count(*) from f1_utltyplnt_smmry where respondent_id=:id), (select count(*) from f1_work where respondent_id=:id), (select count(*) from f1_xmssn_adds where respondent_id=:id), (select count(*) from f1_xmssn_elc_bothr where respondent_id=:id), (select count(*) from f1_xmssn_elc_fothr where respondent_id=:id), (select count(*) from f1_xmssn_line where respondent_id=:id), (select count(*) from f1_xtraordnry_loss where respondent_id=:id), (select count(*) from f1_audit_log where respondent_id=:id), (select count(*) from f1_load_file_names where respondent_id=:id), (select count(*) from f1_privilege where respondent_id=:id), (select count(*) from f1_sys_error_log where respondent_id=:id), (select count(*) from f1_hydro where respondent_id=:id), (select count(*) from f1_ident_attsttn where respondent_id=:id), (select count(*) from f1_steam where respondent_id=:id), (select count(*) from f1_leased where respondent_id=:id), (select count(*) from f1_sbsdry_detail where respondent_id=:id), (select count(*) from f1_plant where respondent_id=:id), (select count(*) from f1_long_term_debt where respondent_id=:id), (select count(*) from f1_106_2009 where respondent_id=:id), (select count(*) from f1_106a_2009 where respondent_id=:id), (select count(*) from f1_106b_2009 where respondent_id=:id), (select count(*) from f1_208_elc_dep where respondent_id=:id), (select count(*) from f1_231_trn_stdycst where respondent_id=:id), (select count(*) from f1_324_elc_expns where respondent_id=:id), (select count(*) from f1_325_elc_cust where respondent_id=:id), (select count(*) from f1_331_transiso where respondent_id=:id), (select count(*) from f1_338_dep_depl where respondent_id=:id), (select count(*) from f1_397_isorto_stl where respondent_id=:id), (select count(*) from f1_398_ancl_ps where respondent_id=:id), (select count(*) from f1_399_mth_peak where respondent_id=:id), (select count(*) from f1_400_sys_peak where respondent_id=:id), (select count(*) from f1_400a_iso_peak where respondent_id=:id), (select count(*) from f1_429_trans_aff where respondent_id=:id), (select count(*) from f1_allowances_nox where respondent_id=:id), (select count(*) from f1_cmpinc_hedge_a where respondent_id=:id), (select count(*) from f1_cmpinc_hedge where respondent_id=:id), (select count(*) from f1_email where respondent_id=:id), (select count(*) from f1_rg_trn_srv_rev where respondent_id=:id), (select count(*) from f1_s0_filing_log where respondent_id=:id), (select count(*) from f1_security where respondent_id=:id)', {'id': '145'})`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1113/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 752995227,MDU6SXNzdWU3NTI5OTUyMjc=,1115,SpatiaLite error could suggest --load-extension=spatialite,9599,simonw,closed,0,,,,,1,2020-11-29T20:05:07Z,2022-01-20T21:29:42Z,2020-11-29T20:13:22Z,OWNER,,"https://github.com/simonw/datasette/blob/242bc89fdf2e775e340d69a4e851b3a9accb31c6/datasette/cli.py#L533-L548 This could use the `find_spatialite()` function and, if it finds something, suggest the user use `--load-extension=spatialite` https://github.com/simonw/datasette/blob/242bc89fdf2e775e340d69a4e851b3a9accb31c6/datasette/utils/__init__.py#L1015-L1019",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1115/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 753026003,MDU6SXNzdWU3NTMwMjYwMDM=,54,github-to-sqlite workflows command,9599,simonw,closed,0,,,,,3,2020-11-29T21:56:42Z,2020-11-29T22:08:46Z,2020-11-29T21:57:17Z,MEMBER,,"A command that fetches the YAML workflows for different repos, parses them and stores them in relational tables would be really useful for maintaining larger numbers of workflows.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 753026388,MDU6SXNzdWU3NTMwMjYzODg=,55,github-to-sqlite workflows does not correctly replace existing records,9599,simonw,closed,0,,,,,0,2020-11-29T21:58:43Z,2020-11-29T23:48:50Z,2020-11-29T23:48:50Z,MEMBER,,Following #54 - see this TODO: https://github.com/dogsheep/github-to-sqlite/blob/1b23ce11953f9f59c0161ea1f99188b55b5ea11c/github_to_sqlite/utils.py#L700,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/55/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 753122082,MDU6SXNzdWU3NTMxMjIwODI=,56,Link to example tables from the README,9599,simonw,closed,0,,,,,0,2020-11-30T04:01:51Z,2020-11-30T04:10:27Z,2020-11-30T04:10:27Z,MEMBER,,Would help demonstrate how the tool works.,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/56/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 753767911,MDExOlB1bGxSZXF1ZXN0NTI5NzgzMjc1,1117,Support for generated columns,9599,simonw,closed,0,,,,,6,2020-11-30T20:10:46Z,2020-11-30T22:23:19Z,2020-11-30T21:29:58Z,OWNER,simonw/datasette/pulls/1117,"Refs #1116. My first attempt at this worked on my laptop but broke in CI, so I'm going to iterate on it in a pull request instead.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1117/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 753788261,MDU6SXNzdWU3NTM3ODgyNjE=,1118,messagge_is_html typo,9599,simonw,closed,0,,,,,0,2020-11-30T20:43:22Z,2020-11-30T21:24:28Z,2020-11-30T21:24:28Z,OWNER,,https://ripgrep.datasette.io/-/ripgrep?pattern=messagge_is_html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1118/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 753876808,MDU6SXNzdWU3NTM4NzY4MDg=,1119,"Include generated columns in fixtures.db, if SQLite version supports it",9599,simonw,closed,0,,,,,1,2020-11-30T23:25:31Z,2020-12-01T00:41:14Z,2020-12-01T00:28:04Z,OWNER,,"In #1117 I added a one-off test that creates a DB with generated columns in it: https://github.com/simonw/datasette/blob/461670a0b87efa953141b449a9a261919864ceb3/tests/test_api.py#L1933-L1964 If this table was conditionally added to `fixtures.db` (if SQLite supports the feature) it would be available in the demo at https://latest.datasette.io/",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1119/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 753898359,MDExOlB1bGxSZXF1ZXN0NTI5ODg3ODYx,1120,generated_columns table in fixtures.py,9599,simonw,closed,0,,,,,1,2020-12-01T00:17:19Z,2020-12-01T00:28:03Z,2020-12-01T00:28:02Z,OWNER,simonw/datasette/pulls/1120,Refs #1119,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1120/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 755721275,MDU6SXNzdWU3NTU3MjEyNzU=,1123,"Table actions hook are order dependent, should not be",9599,simonw,closed,0,,,,,1,2020-12-03T00:47:12Z,2020-12-03T00:50:17Z,2020-12-03T00:50:17Z,OWNER,,"Got this error: https://github.com/simonw/datasette/runs/1489770800 ``` > assert get_table_actions_links(response_2.text) == [ {""label"": ""From async"", ""href"": ""/""}, {""label"": ""Database: fixtures"", ""href"": ""/""}, {""label"": f""Table: {table_or_view}"", ""href"": ""/""}, ] E AssertionError: assert [{'href': '/'...'From async'}] == [{'href': '/'...: facetable'}] E At index 0 diff: {'label': 'Database: fixtures', 'href': '/'} != {'label': 'From async', 'href': '/'} E Use -v to get the full diff ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1123/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 756439516,MDU6SXNzdWU3NTY0Mzk1MTY=,1124,Datasette on Amazon Linux on ARM returns 404 for static assets,9599,simonw,closed,0,,,,,9,2020-12-03T18:20:37Z,2020-12-03T21:42:02Z,2020-12-03T21:10:54Z,OWNER,,"Very weird bug this one. Steps to reproduce: ``` # I started a amzn2-ami-hvm-2.0.20201126.0-arm64-gp2 t4g.micro instance ec2 % ssh -i simonw-ec2.pem ec2-user@ec2-18-219-238-192.us-east-2.compute.amazonaws.com [ec2-user@ip-172-31-30-7 ~]$ sudo yum install python3 Loaded plugins: extras_suggestions, langpacks, priorities, update-motd ... Is this ok [y/d/N]: y Downloading packages: (1/4): python3-3.7.9-1.amzn2.0.1.aarch64.rpm | 72 kB 00:00:00 (2/4): python3-pip-9.0.3-1.amzn2.0.2.noarch.rpm | 1.9 MB 00:00:00 (3/4): python3-setuptools-38.4.0-3.amzn2.0.6.noarch.rpm | 617 kB 00:00:00 (4/4): python3-libs-3.7.9-1.amzn2.0.1.aarch64.rpm | 9.1 MB 00:00:00 ----------------------------------------------------------------------------------------------------------------------------------------------------------- Total 68 MB/s | 12 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : python3-pip-9.0.3-1.amzn2.0.2.noarch 1/4 Installing : python3-3.7.9-1.amzn2.0.1.aarch64 2/4 Installing : python3-setuptools-38.4.0-3.amzn2.0.6.noarch 3/4 Installing : python3-libs-3.7.9-1.amzn2.0.1.aarch64 4/4 Verifying : python3-setuptools-38.4.0-3.amzn2.0.6.noarch 1/4 Verifying : python3-pip-9.0.3-1.amzn2.0.2.noarch 2/4 Verifying : python3-3.7.9-1.amzn2.0.1.aarch64 3/4 Verifying : python3-libs-3.7.9-1.amzn2.0.1.aarch64 4/4 Installed: python3.aarch64 0:3.7.9-1.amzn2.0.1 Dependency Installed: python3-libs.aarch64 0:3.7.9-1.amzn2.0.1 python3-pip.noarch 0:9.0.3-1.amzn2.0.2 python3-setuptools.noarch 0:38.4.0-3.amzn2.0.6 Complete! [ec2-user@ip-172-31-30-7 ~]$ python3 -m pip install --user pipx Collecting pipx Downloading https://files.pythonhosted.org/packages/42/8a/8447ec14562c5d97afbee54ec390864718bccce9dfb0506c8c77852489d3/pipx-0.15.6.0-py3-none-any.whl (43kB) 100% |████████████████████████████████| 51kB 3.8MB/s Collecting packaging>=20.0 (from pipx) Downloading https://files.pythonhosted.org/packages/28/87/8edcf555adaf60d053ead881bc056079e29319b643ca710339ce84413136/packaging-20.7-py2.py3-none-any.whl Collecting argcomplete<2.0,>=1.9.4 (from pipx) Downloading https://files.pythonhosted.org/packages/e3/d0/ee7fc6ceac8957196def9bfa3b955d02163058defd3edd51ef7b1ff2769e/argcomplete-1.12.2-py2.py3-none-any.whl Collecting userpath>=1.4.1 (from pipx) Downloading https://files.pythonhosted.org/packages/fb/f1/faca8a3cc86bd2223aaf72edd222bc31d6ae2eea5feaf17a144634a92e6d/userpath-1.4.1-py2.py3-none-any.whl Collecting pyparsing>=2.0.2 (from packaging>=20.0->pipx) Downloading https://files.pythonhosted.org/packages/8a/bb/488841f56197b13700afd5658fc279a2025a39e22449b7cf29864669b15d/pyparsing-2.4.7-py2.py3-none-any.whl (67kB) 100% |████████████████████████████████| 71kB 8.8MB/s Collecting importlib-metadata<4,>=0.23; python_version == ""3.7"" (from argcomplete<2.0,>=1.9.4->pipx) Downloading https://files.pythonhosted.org/packages/99/c7/4ccf2baa455613aa9e61372365aba8594ff2806c82189c31a6c65e7c679e/importlib_metadata-3.1.1-py3-none-any.whl Collecting click (from userpath>=1.4.1->pipx) Downloading https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl (82kB) 100% |████████████████████████████████| 92kB 10.5MB/s Collecting distro; platform_system == ""Linux"" (from userpath>=1.4.1->pipx) Downloading https://files.pythonhosted.org/packages/25/b7/b3c4270a11414cb22c6352ebc7a83aaa3712043be29daa05018fd5a5c956/distro-1.5.0-py2.py3-none-any.whl Collecting zipp>=0.5 (from importlib-metadata<4,>=0.23; python_version == ""3.7""->argcomplete<2.0,>=1.9.4->pipx) Downloading https://files.pythonhosted.org/packages/41/ad/6a4f1a124b325618a7fb758b885b68ff7b058eec47d9220a12ab38d90b1f/zipp-3.4.0-py3-none-any.whl Installing collected packages: pyparsing, packaging, zipp, importlib-metadata, argcomplete, click, distro, userpath, pipx Successfully installed argcomplete-1.12.2 click-7.1.2 distro-1.5.0 importlib-metadata-3.1.1 packaging-20.7 pipx-0.15.6.0 pyparsing-2.4.7 userpath-1.4.1 zipp-3.4.0 [ec2-user@ip-172-31-30-7 ~]$ python3 -m pipx ensurepath /home/ec2-user/.local/bin is already in PATH. /home/ec2-user/.local/bin is already in PATH. All pipx binary directories have been added to PATH. If you are sure you want to proceed, try again with the '--force' flag. Otherwise pipx is ready to go! ✨ 🌟 ✨ [ec2-user@ip-172-31-30-7 ~]$ pipx install datasette installed package datasette 0.52.2, Python 3.7.9 These apps are now globally available - datasette done! ✨ 🌟 ✨ [ec2-user@ip-172-31-30-7 ~]$ datasette --version datasette, version 0.52.2 [ec2-user@ip-172-31-30-7 ~]$ datasette --get /-/static/app.css 404 ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1124/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 756622648,MDU6SXNzdWU3NTY2MjI2NDg=,1125,Show pysqlite3 version on /-/versions,9599,simonw,closed,0,,,,,5,2020-12-03T21:57:23Z,2020-12-04T04:16:57Z,2020-12-04T04:16:57Z,OWNER,,"This code can use `pysqlite3` if available. The version should be exposed on `/-/versions`. https://github.com/simonw/datasette/blob/4cce5516661b24afeddaf35bee84b00fbf5c7f89/datasette/utils/sqlite.py#L1-L4",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1125/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 756761963,MDU6SXNzdWU3NTY3NjE5NjM=,1126,Switch to google-github-actions/setup-gcloud for demo deploy,9599,simonw,closed,0,,,,,1,2020-12-04T03:11:24Z,2020-12-04T03:51:38Z,2020-12-04T03:51:38Z,OWNER,,"That workflow is showing warnings: https://github.com/simonw/datasette/actions/runs/399400077 > Thank you for using setup-gcloud Action. GoogleCloudPlatform/github-actions/setup-gcloud has been deprecated, please switch to google-github-actions/setup-gcloud. https://github.com/google-github-actions/setup-gcloud has a Cloud Run example here: https://github.com/google-github-actions/setup-gcloud/blob/master/example-workflows/cloud-run/.github/workflows/cloud-run.yml",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1126/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 757481949,MDU6SXNzdWU3NTc0ODE5NDk=,1131,"""datasette inspect"" outputs invalid JSON if an error is logged",9599,simonw,closed,0,,,,,3,2020-12-05T00:00:45Z,2020-12-05T20:48:34Z,2020-12-05T05:21:19Z,OWNER,,"See https://github.com/simonw/register-of-members-interests/issues/6: ``` % datasette inspect regmem.db ERROR: conn=, sql = 'select count(*) from [items_fts]', params = None: SQL logic error { ""regmem"": { ""hash"": ""6fde27e3dea80d6b65f2ac7f89cd8448980fee8c91b505ba29c311ba0393317f"", ""size"": 936198144, ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1131/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 758899581,MDU6SXNzdWU3NTg4OTk1ODE=,1132,New filter: array does not contain,9599,simonw,closed,0,,,,,1,2020-12-07T22:28:20Z,2020-12-07T22:50:36Z,2020-12-07T22:41:09Z,OWNER,,"I want to see all of my GitHub repos that are tagged `datasette-plugin` but are NOT tagged `datasette-io`: https://github-to-sqlite.dogsheep.net/github/repos?_facet=owner&owner=9599&_facet_array=topics&topics__arraycontains=datasette-plugin&topics__arraycontains=datasette-io#facet-topics",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 758944006,MDU6SXNzdWU3NTg5NDQwMDY=,57,--readme throws 404 error if README does not exist in repo,9599,simonw,closed,0,,,,,0,2020-12-07T23:58:49Z,2020-12-16T18:17:54Z,2020-12-16T18:17:54Z,MEMBER,,It should fail silently (populate the column with a null) instead.,207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/57/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 759695780,MDU6SXNzdWU3NTk2OTU3ODA=,1133,Option to omit header row in CSV export,9599,simonw,closed,0,,,,,2,2020-12-08T18:54:46Z,2021-06-17T18:12:31Z,2020-12-10T23:28:51Z,OWNER,,`?_header=off` - for symmetry with existing option `?_nl=on`.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1133/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 760605882,MDU6SXNzdWU3NjA2MDU4ODI=,1135,Feature: --create option to create database file if it does not yet exist,9599,simonw,closed,0,,,,,0,2020-12-09T19:23:58Z,2021-01-24T21:19:39Z,2020-12-09T19:45:52Z,OWNER,,"I'd like to be able to tell people to run the following in the Datasette documentation to get started: brew install datasette datasette install datasette-upload-csvs datasette data.db --create --root --open This would give them a local Datasette instance with the ability to drag-and-drop CSV files directly into it. Just one catch: I don't want to have to talk them through creating an empty SQLite database file. So I want to add a new `--create` option which means ""If any of the database files passed on the command-line do not yet exist, create them"".",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1135/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 760621356,MDU6SXNzdWU3NjA2MjEzNTY=,1136,Establish pattern for release branches to support bug fixes,9599,simonw,closed,0,,,,,8,2020-12-09T19:48:18Z,2020-12-09T20:17:02Z,2020-12-09T20:14:41Z,OWNER,,"I want to fix the bug in #1134 and ship it as Datasette 0.52.5 - but the `main` branch now has a feature in it (4c25b035b2370983c8dd5e0c8762e9154e379774 added `arraynotcontains`, #1132). I'm not ready for a feature release, so instead I want to release 0.52.5 with just that bug fix. This is the first time I will have shipped a release from a branch. I need to establish that pattern and add it to the documentation in https://docs.datasette.io/en/stable/contributing.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1136/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 761706858,MDU6SXNzdWU3NjE3MDY4NTg=,1137,Update README to reflect new datasette.io site,9599,simonw,closed,0,,,,,0,2020-12-10T23:22:06Z,2020-12-10T23:28:50Z,2020-12-10T23:28:50Z,OWNER,,Can finally close #659.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1137/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 761713079,MDU6SXNzdWU3NjE3MTMwNzk=,1138,"""Powered by Datasette"" should link to new datasette.io site",9599,simonw,closed,0,,,,,0,2020-12-10T23:33:41Z,2020-12-15T02:28:10Z,2020-12-10T23:37:14Z,OWNER,,https://datasette.io/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1138/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 761915790,MDU6SXNzdWU3NjE5MTU3OTA=,206,sqlite-utils should suggest --csv if JSON parsing fails,9599,simonw,closed,0,,,,,4,2020-12-11T05:17:56Z,2021-10-30T15:52:17Z,2021-01-03T18:42:22Z,OWNER,,"``` ~ % gsutil cat gs://ossf-criticality-score/python_top_200.csv | sqlite-utils insert /tmp/crit.db crit - ... File ""/usr/local/Cellar/python@3.9/3.9.0_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py"", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File ""/usr/local/Cellar/python@3.9/3.9.0_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py"", line 355, in raw_decode raise JSONDecodeError(""Expecting value"", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ``` A nicer error message here would be one that says the JSON is invalid but suggests that maybe you could try `--csv`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/206/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 763207948,MDU6SXNzdWU3NjMyMDc5NDg=,1141,Default styling for bullet point lists,9599,simonw,closed,0,,,,,0,2020-12-12T02:49:33Z,2021-03-29T00:14:05Z,2021-03-29T00:14:05Z,OWNER,,"I just noticed that https://datasette.io/content/recent_releases (which uses `datasette-render-markdown`) is missing its bullet points: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1141/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 763283616,MDU6SXNzdWU3NjMyODM2MTY=,207,sqlite-utils analyze-tables command,9599,simonw,closed,0,,,,,4,2020-12-12T04:33:12Z,2020-12-13T07:25:23Z,2020-12-13T07:20:13Z,OWNER,,"A command which analyzes a table (potentially taking quite a while if the table is large) and outputs information for each column - things like: - How many unique values does this column have? - How many null rows? - How many blank rows? (defined as empty string) - What are the 10 most common values? - What are the 10 least common values? The command can output this information to the terminal, but it should also provide an option for writing the information to a database table so it can be explored later.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/207/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 763320133,MDExOlB1bGxSZXF1ZXN0NTM3NzkxNjc1,208,sqlite-utils analyze-tables command and table.analyze_column() method,9599,simonw,closed,0,,,,,6,2020-12-12T05:27:49Z,2020-12-13T07:20:16Z,2020-12-13T07:20:12Z,OWNER,simonw/sqlite-utils/pulls/208,"Refs #207 - [x] Improve design of CLI output - [x] Truncate long values in least/most common - [x] Add a `-c` column selection option - [x] Tests - [x] Documentation",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/208/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 769150394,MDU6SXNzdWU3NjkxNTAzOTQ=,58,Readme HTML has broken internal links,9599,simonw,closed,0,,,,,2,2020-12-16T17:58:11Z,2020-12-16T19:20:14Z,2020-12-16T19:20:14Z,MEMBER,,"From https://github.com/simonw/datasette.io/issues/46 ```html
  • Filtering tables
    • ...

    Filtering tables

    ``` So this is a bug in GitHub's API, but we need to work around it.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 769282206,MDU6SXNzdWU3NjkyODIyMDY=,30,Upgrade to sqlite-utils 3.0 (tests are failing),9599,simonw,closed,0,,,,,0,2020-12-16T21:25:15Z,2020-12-16T21:27:11Z,2020-12-16T21:27:10Z,MEMBER,,"``` results = beta_db[""search_index""].search(""run"") if use_porter: assert results == [ ( ""dogs.db/dogs"", ""1"", ""Cleo"", ""2020-08-22 04:41:33"", 1, 0, ""running"", None, None, ) ] else: > assert results == [] E assert == [] E + E -[] E Full diff: E - [] E + ``` This was caused by a backwards incompatible change in sqlite-utils 3.0: https://sqlite-utils.readthedocs.io/en/stable/changelog.html#v3-0",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/30/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 771202454,MDU6SXNzdWU3NzEyMDI0NTQ=,1153,"Use YAML examples in documentation by default, not JSON",9599,simonw,closed,0,,,,,22,2020-12-18T22:20:15Z,2023-07-08T20:09:48Z,2023-07-08T20:08:13Z,OWNER,,"YAML configuration is much better for multi-line strings, and I'm increasingly adding configuration options to Datasette that benefit from that - fragments of HTML in `description_html` or SQL queries used to configure things like https://github.com/simonw/datasette-atom for example. Rather than confusing things by showing both in the documentation, I should switch all of the default examples to use YAML instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1153/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 771216293,MDU6SXNzdWU3NzEyMTYyOTM=,1155,Better internal database_name for _internal database,9599,simonw,closed,0,,,,,9,2020-12-18T22:47:27Z,2020-12-22T20:04:35Z,2020-12-22T20:04:35Z,OWNER,,"https://latest.datasette.io/login-as-root then https://latest.datasette.io/_internal That `database_name` is ugly. It should just be `_internal`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1155/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 771316301,MDU6SXNzdWU3NzEzMTYzMDE=,31,"Searching for ""github-to-sqlite"" throws an error",9599,simonw,closed,0,,,,,4,2020-12-19T06:07:20Z,2020-12-19T06:18:07Z,2020-12-19T06:18:07Z,MEMBER,,"https://datasette.io/-/beta?q=github-to-sqlite&sort=relevance&type=blog.db%2Fentries - ""no such column: to""",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777386465,MDU6SXNzdWU3NzczODY0NjU=,211,table.triggers_dict introspection property,9599,simonw,closed,0,,,,,0,2021-01-02T02:04:00Z,2021-01-02T02:10:10Z,2021-01-02T02:10:10Z,OWNER,,"`table.triggers` currently returns a list of `Trigger` values. A `table.triggers_dict` property could behave like `columns_dict`, returning a dictionary mapping trigger names to their SQL definitions for that table.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/211/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777392020,MDU6SXNzdWU3NzczOTIwMjA=,212,Mechanism for maintaining cache of table counts using triggers,9599,simonw,closed,0,,,,,1,2021-01-02T02:58:53Z,2021-01-02T21:40:27Z,2021-01-02T21:40:27Z,OWNER,,"Counting all of the rows in a large table is expensive - this is one of the main causes of performance problems in Datasette when running against large databases. Carefully constructed SQL triggers could be used to maintain accurate cached counts for a table, by incrementing and decrementing a counter every time a row is inserted or deleted. `sqlite-utils` already has a mechanism for creating triggers for FTS - the `table.enable_fts(..., create_triggers=True)` method. How about a similar mechanism for setting up triggers to maintain a count of table rows?",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/212/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777529979,MDU6SXNzdWU3Nzc1Mjk5Nzk=,213,db.enable_counts() method,9599,simonw,closed,0,,,,,2,2021-01-02T21:44:55Z,2021-01-02T22:04:02Z,2021-01-02T22:04:02Z,OWNER,,"Following #212 it would be useful if there was a utility method for enabling counts for ALL tables in a database: ```python db.enable_counts() ``` Open question: should this setup triggers for virtual tables such as FTS tables? Could that break things? ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/213/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777530107,MDU6SXNzdWU3Nzc1MzAxMDc=,214,sqlite-utils enable-counts command,9599,simonw,closed,0,,,,,0,2021-01-02T21:45:48Z,2021-01-03T04:26:44Z,2021-01-03T04:26:44Z,OWNER,,"The CLI version of #212 and #213. # Enable counts for all tables: sqlite-utils enable-counts data.db # Enable counts for specific tables: sqlite-utils enable-counts data.db table1 table2",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/214/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777535402,MDU6SXNzdWU3Nzc1MzU0MDI=,215,Use _counts to speed up counts,9599,simonw,closed,0,,,,,9,2021-01-02T22:30:17Z,2021-01-03T20:19:40Z,2021-01-03T20:19:40Z,OWNER,,"Utility mechanism for taking advantage of the new `_counts` table from #212 would be nice. These can trigger automatically if the `_counts` table exists, but since `sqlite-utils` needs to work against any existing database there should be a way of opting out of this optimization.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/215/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777540352,MDU6SXNzdWU3Nzc1NDAzNTI=,216,database.triggers_dict introspection property,9599,simonw,closed,0,,,,,1,2021-01-02T23:13:00Z,2021-01-03T04:27:14Z,2021-01-03T04:25:36Z,OWNER,,Following #211,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/216/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777543336,MDU6SXNzdWU3Nzc1NDMzMzY=,217,Rename .escape() to .quote(),9599,simonw,closed,0,,,,,2,2021-01-02T23:40:52Z,2021-01-03T04:27:38Z,2021-01-03T04:15:23Z,OWNER,,"`.quote()` is a better name because it reflects that the method adds quotes around the value. This method has never been documented so I'm going to rename it without a major version bump. ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/217/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777560474,MDU6SXNzdWU3Nzc1NjA0NzQ=,218,"""sqlite-utils triggers"" command",9599,simonw,closed,0,,,,,1,2021-01-03T02:34:50Z,2021-01-03T03:49:51Z,2021-01-03T03:03:35Z,OWNER,,"A command to list the triggers in the database. sqlite-utils triggers my.db Can optionally take one or more tables: sqlite-utils triggers my.db table1 table2",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/218/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 777707544,MDU6SXNzdWU3Nzc3MDc1NDQ=,219,reset_counts() method and command,9599,simonw,closed,0,,,,,4,2021-01-03T20:08:28Z,2021-01-03T20:59:37Z,2021-01-03T20:59:37Z,OWNER,,"> Thought: maybe there should be a `.reset_counts()` method too, for if the table gets out of date with the triggers. > > One way that could happen is if a table is dropped and recreated - the counts in the `_counts` table would likely no longer match the number of rows in that table. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/219/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 783910901,MDU6SXNzdWU3ODM5MTA5MDE=,221,.add_missing_columns() does not take case insensitivity into account,9599,simonw,closed,0,,,,,0,2021-01-12T05:01:00Z,2021-01-12T23:17:33Z,2021-01-12T23:17:33Z,OWNER,,SQLite columns are case insensitive - but the `.add_missing_columns()` method doesn't know that. This means that it can crash if it identifies a column that is a case-insensitive duplicate of an existing column. https://github.com/simonw/sqlite-utils/blob/4cc82fd0bccc9d2eeb3510beb4e691d7da099f84/sqlite_utils/db.py#L1974-L1980,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 787900412,MDU6SXNzdWU3ODc5MDA0MTI=,222,.m2m() should accept alter=True parameter,9599,simonw,closed,0,,,,,0,2021-01-18T04:15:43Z,2021-01-18T04:26:10Z,2021-01-18T04:26:10Z,OWNER,,Needed by https://github.com/dogsheep/swarm-to-sqlite/issues/11,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 788527932,MDU6SXNzdWU3ODg1Mjc5MzI=,223,--delimiter option for CSV import,9599,simonw,closed,0,,,,,2,2021-01-18T20:25:03Z,2021-02-06T01:39:47Z,2021-02-06T01:34:54Z,OWNER,,"https://bruxellesdata.opendatasoft.com/explore/dataset/dog-toilets/export/?location=12,50.85802,4.38054 says: > CSV uses semicolon (;) as a separator. Would be useful to be able to do this: sqlite-utils insert places.db places places.csv --delimiter ';' `--delimiter` could imply `--csv`",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/223/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 792625812,MDU6SXNzdWU3OTI2MjU4MTI=,1198,Plugin testing documentation on using pytest-httpx,9599,simonw,closed,0,,,,,1,2021-01-23T18:46:16Z,2021-01-24T20:40:38Z,2021-01-24T20:38:43Z,OWNER,,"I keep on having to figure this out: if you use the https://pypi.org/project/pytest-httpx/ fixture to write tests against mocked external APIs, they will fail because that module will break Datasette's own `datasette.client` testing mechanism. You can fix this using: ```python @pytest.fixture def non_mocked_hosts(): return [""localhost""] ``` See https://github.com/simonw/datasette-indieauth/blob/1.2/tests/test_indieauth.py I can add this tip to the https://docs.datasette.io/en/stable/testing_plugins.html page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1198/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 792958773,MDExOlB1bGxSZXF1ZXN0NTYwNzI1NzE0,1203,Easier way to run Prettier locally,9599,simonw,closed,0,,,,,0,2021-01-25T01:39:06Z,2021-01-25T01:41:46Z,2021-01-25T01:41:46Z,OWNER,simonw/datasette/pulls/1203,Refs #1167,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1203/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 793086333,MDExOlB1bGxSZXF1ZXN0NTYwODMxNjM4,1206,Release 0.54,9599,simonw,closed,0,,,,,3,2021-01-25T06:45:47Z,2021-01-25T17:33:30Z,2021-01-25T17:33:29Z,OWNER,simonw/datasette/pulls/1206,Refs #1201,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1206/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 793881756,MDU6SXNzdWU3OTM4ODE3NTY=,1207,"Document the Datasette(..., pdb=True) testing pattern",9599,simonw,closed,0,,,,,1,2021-01-26T02:48:10Z,2021-01-29T02:37:19Z,2021-01-29T02:12:34Z,OWNER,,"If you're writing tests for a Datasette plugin and you get a 500 error from inside Datasette, you can cause Datasette to open a PDB session within the application server code by doing this: ```python ds = Datasette([db_path], pdb=True) response = await ds.client.get(""/"") ``` You'll need to run `pytest -s` to interact with the debugger, otherwise you'll get an error.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1207/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 799693777,MDU6SXNzdWU3OTk2OTM3Nzc=,1214,Re-submitting filter form duplicates _x querystring arguments,9599,simonw,closed,0,,,,,3,2021-02-02T21:13:35Z,2021-02-02T21:28:53Z,2021-02-02T21:21:13Z,OWNER,,"Really nasty bug, caused by #1194 fix in 07e163561592c743e4117f72102fcd350a600909 Navigate to this page: https://github-to-sqlite.dogsheep.net/github/labels?_search=help&_sort=id Click ""Apply"" to submit the form and the resulting URL is https://github-to-sqlite.dogsheep.net/github/labels?_search=help&_sort=id&_search=help&_sort=id That's because the (truncated) HTML for the form looks like this: ```html ... ...
    ... ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1214/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 800669347,MDU6SXNzdWU4MDA2NjkzNDc=,1216,"/-/databases should reflect connection order, not alphabetical order",9599,simonw,closed,0,,,,,1,2021-02-03T20:20:23Z,2021-06-02T03:10:19Z,2021-06-02T03:10:19Z,OWNER,,"The order in which databases are attached to Datasette matters - it affects the homepage, and it's beginning to influence how certain plugins work (see https://github.com/simonw/datasette-tiles/issues/8). Two years ago in cccea85be6aaaeadb31f3b588ec7f732628815f5 I made `/-/databases` return things in alphabetical order, to fix a test failure in Python 3.5. Python 3.5 is no longer supported, so this is no longer necessary - and this behaviour should now be treated as a bug.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1216/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 802583450,MDU6SXNzdWU4MDI1ODM0NTA=,226,3.4 release is broken - includes a rogue line,9599,simonw,closed,0,,,,,0,2021-02-06T02:08:01Z,2021-02-06T02:10:26Z,2021-02-06T02:10:26Z,OWNER,,"I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3 That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199 I'm surprised the tests didn't catch this!",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 806849424,MDU6SXNzdWU4MDY4NDk0MjQ=,1221,Support SSL/TLS directly,9599,simonw,closed,0,,,,,4,2021-02-12T00:18:29Z,2022-12-18T02:39:04Z,2021-02-12T00:52:18Z,OWNER,,This should be pretty easy because Uvicorn supports them already. Need a good mechanism for testing it - https://pypi.org/project/trustme/ looks ideal.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1221/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 806861312,MDExOlB1bGxSZXF1ZXN0NTcyMjA5MjQz,1222,"--ssl-keyfile and --ssl-certfile, refs #1221",9599,simonw,closed,0,,,,,0,2021-02-12T00:45:58Z,2021-02-12T00:52:18Z,2021-02-12T00:52:17Z,OWNER,simonw/datasette/pulls/1222,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1222/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 807437089,MDU6SXNzdWU4MDc0MzcwODk=,228,--no-headers option for CSV and TSV,9599,simonw,closed,0,,,,,10,2021-02-12T17:56:51Z,2021-12-26T07:01:31Z,2021-02-14T22:25:17Z,OWNER,,"https://bl.iro.bl.uk/work/ns/3037474a-761c-456d-a00c-9ef3c6773f4c has a fascinating CSV file that doesn't have a header row - it starts like this: ```csv Computation and measurement of turbulent flow through idealized turbine blade passages,,""Loizou, Panos A."",https://isni.org/isni/0000000136122593,,University of Manchester,https://isni.org/isni/0000000121662407,1989,Thesis (Ph.D.),,Physical Sciences,,,https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.232781, ""Prolactin and growth hormone secretion in normal, hyperprolactinaemic and acromegalic man"",,""Prescott, R. W. G."",https://isni.org/isni/0000000134992122,,University of Newcastle upon Tyne,https://isni.org/isni/0000000104627212,1983,Thesis (Ph.D.),,Biological Sciences,,,https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.232784, ``` It would be useful if `sqlite-utils insert ... --csv` had a mechanism for importing files like this one.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/228/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 808008305,MDU6SXNzdWU4MDgwMDgzMDU=,230,--sniff option for sniffing delimiters,9599,simonw,closed,0,,,,,8,2021-02-14T17:43:54Z,2021-02-14T21:15:33Z,2021-02-14T19:24:32Z,OWNER,,"> I just spotted that `csv.Sniffer` in the Python standard library has a `.has_header(sample)` method which detects if the first row appears to be a header or not, which is interesting. https://docs.python.org/3/library/csv.html#csv.Sniffer _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778812050_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/230/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 808028757,MDU6SXNzdWU4MDgwMjg3NTc=,231,"limit=X, offset=Y parameters for more Python methods",9599,simonw,closed,0,,,,,2,2021-02-14T19:31:23Z,2021-02-14T20:03:08Z,2021-02-14T20:03:08Z,OWNER,,"> I'm going to add a `offset=` parameter to support this case. Thanks for the suggestion! _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/224#issuecomment-778828495_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 808036774,MDU6SXNzdWU4MDgwMzY3NzQ=,232,Run tests against Windows in GitHub Actions,9599,simonw,closed,0,,,,,0,2021-02-14T20:09:45Z,2021-02-14T20:39:55Z,2021-02-14T20:39:55Z,OWNER,,"> I'm going to try and get the test suite to run in Windows on GitHub Actions. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 808037010,MDExOlB1bGxSZXF1ZXN0NTczMTQ3MTY4,233,"Run tests against Ubuntu, macOS and Windows",9599,simonw,closed,0,,,,,0,2021-02-14T20:11:02Z,2021-02-14T20:39:54Z,2021-02-14T20:39:54Z,OWNER,simonw/sqlite-utils/pulls/233,Refs #232,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/233/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 808046597,MDU6SXNzdWU4MDgwNDY1OTc=,234,.insert_all() fails if subsequent chunks contain additional columns,9599,simonw,closed,0,,,,,1,2021-02-14T21:01:51Z,2021-02-14T21:03:40Z,2021-02-14T21:03:40Z,OWNER,,Reported by @nieuwenhoven in #225 along with a proposed fix.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/234/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 808843401,MDU6SXNzdWU4MDg4NDM0MDE=,1226,--port option should validate port is between 0 and 65535,9599,simonw,closed,0,,,,,4,2021-02-15T22:01:33Z,2021-02-18T18:41:27Z,2021-02-18T18:41:27Z,OWNER,,"Currently throws an ugly error message: ``` (datasette-graphql) datasette-graphql % datasette fivethirtyeight.db -p 80094 INFO: Started server process [45497] INFO: Waiting for application startup. INFO: Application startup complete. Traceback (most recent call last): File ""/Users/simon/.local/share/virtualenvs/datasette-graphql-n1OSJCS8/bin/datasette"", line 8, in sys.exit(cli()) ... server = await loop.create_server( File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py"", line 1461, in create_server sock.bind(sa) OverflowError: bind(): port must be 0-65535. ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1226/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 810394616,MDU6SXNzdWU4MTAzOTQ2MTY=,1227,Configure sphinx.ext.extlinks for issues,9599,simonw,closed,0,,,,,1,2021-02-17T17:38:02Z,2021-08-02T21:38:39Z,2021-02-18T01:20:33Z,OWNER,,"Spotted this in the aspw documentation: https://github.com/rogerbinns/apsw/blob/3.34.0-r1/doc/conf.py#L29-L36 ```python extlinks={ 'cvstrac': ('https://sqlite.org/cvstrac/tktview?tn=%s', 'SQLite ticket #'), 'sqliteapi': ('https://sqlite.org/c3ref/%s.html', 'XXYouShouldNotSeeThisXX'), 'issue': ('https://github.com/rogerbinns/apsw/issues/%s', 'APSW issue '), 'source': ('https://github.com/rogerbinns/apsw/blob/master/%s', ''), } ``` Which lets you link to issues like this: :issue:`268`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1227/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 811367257,MDU6SXNzdWU4MTEzNjcyNTc=,1231,Race condition errors in new refresh_schemas() mechanism,9599,simonw,closed,0,,,,,11,2021-02-18T18:49:54Z,2021-07-16T19:44:59Z,2021-07-16T19:44:59Z,OWNER,,I tried running a Locust load test against Datasette and hit an error message about a failure to create tables because they already existed. I think this means there are race conditions in the new `refresh_schemas()` mechanism added in #1150.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 811407131,MDExOlB1bGxSZXF1ZXN0NTc1OTQwMTkz,1232,--crossdb option for joining across databases,9599,simonw,closed,0,,,,,8,2021-02-18T19:48:50Z,2021-02-18T22:09:13Z,2021-02-18T22:09:12Z,OWNER,simonw/datasette/pulls/1232,"Refs #283. Still needs: - [x] Unit test for --crossdb queries - [x] Show warning on console if it truncates at ten databases (or on web interface) - [x] Show connected databases on the `/_memory` database page - [x] Documentation - [x] https://latest.datasette.io/ demo should demonstrate this feature",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1232/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 811589344,MDU6SXNzdWU4MTE1ODkzNDQ=,1235,Upgrade Python version used by official Datasette Docker image,9599,simonw,closed,0,,,,,2,2021-02-19T00:47:40Z,2021-02-19T01:48:31Z,2021-02-19T01:48:30Z,OWNER,,"Currently uses 3.7.2: https://github.com/simonw/datasette/blob/73bed175631a79e13a521eee82f8451dd0477eb3/Dockerfile#L1 There's a security fix for Python which it would be good to ship in this image (even though I'm reasonably confident it doesn't affect Datasette): https://bugs.python.org/issue42938",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1235/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 811680502,MDU6SXNzdWU4MTE2ODA1MDI=,236,--attach command line option for attaching extra databases,9599,simonw,closed,0,,,,,1,2021-02-19T04:38:30Z,2021-02-19T05:10:41Z,2021-02-19T05:08:43Z,OWNER,,"This will enable cross-database joins, as seen in https://github.com/simonw/datasette/issues/283 Also refs #113",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/236/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 812228314,MDU6SXNzdWU4MTIyMjgzMTQ=,1236,Ability to increase size of the SQL editor window,9599,simonw,closed,0,,,,,9,2021-02-19T18:09:27Z,2021-05-18T03:28:25Z,2021-02-22T21:05:21Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1236/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 813978858,MDU6SXNzdWU4MTM5Nzg4NTg=,1239,JSON filter fails if column contains spaces,9599,simonw,closed,0,,,,,1,2021-02-23T00:18:07Z,2021-02-23T00:22:53Z,2021-02-23T00:22:53Z,OWNER,,"Got this exception: `ERROR: conn=, sql = 'select Address, Affiliation, County, [Has Report], [Latest report notes], [Latest report yes], Latitude, [Location Type], Longitude, Name, id, [Appointment scheduling instructions], [Availability Info], [Latest report] from locations where rowid in (\n select locations.rowid from locations, json_each(locations.Availability Info) j\n where j.value = :p0\n ) and ""Latest report yes"" = :p1 order by id limit 101', params = {'p0': 'Yes: appointment required', 'p1': '1'}: near ""Info"": syntax error`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1239/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 816523763,MDU6SXNzdWU4MTY1MjM3NjM=,238,.add_foreign_key() corrupts database if column contains a space,9599,simonw,closed,0,,,,,1,2021-02-25T15:07:20Z,2021-02-25T16:54:02Z,2021-02-25T16:54:02Z,OWNER,,"I ran this: db[""Reports""].add_foreign_key(""Reported by ID"", ""Reporters"", ""id"") And got this: ``` ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in add_foreign_keys(self, foreign_keys) 616 # Have to VACUUM outside the transaction to ensure .foreign_keys property 617 # can see the newly created foreign key. --> 618 self.vacuum() 619 620 def index_foreign_keys(self): ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in vacuum(self) 629 630 def vacuum(self): --> 631 self.execute(""VACUUM;"") 632 633 ~/jupyter-venv/lib/python3.9/site-packages/sqlite_utils/db.py in execute(self, sql, parameters) 234 return self.conn.execute(sql, parameters) 235 else: --> 236 return self.conn.execute(sql) 237 238 def executescript(self, sql): DatabaseError: database disk image is malformed ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/238/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 816560819,MDU6SXNzdWU4MTY1NjA4MTk=,240,table.pks_and_rows_where() method returning primary keys along with the rows,9599,simonw,closed,0,,,,,7,2021-02-25T15:49:28Z,2021-02-25T16:39:23Z,2021-02-25T16:28:23Z,OWNER,,"*Original title: Easier way to update a row returned from .rows* Here's a surprisingly hard problem I ran into while trying to implement #239 - given a row returned by `db[table].rows` how can you update that row? The problem is that the `db[table].update(...)` method requires a primary key. But if you have a row from the `db[table].rows` iterator it might not even contain the primary key - provided the table is a `rowid` table. Instead, currently, you need to introspect the table and, if `rowid` is a primary key, explicitly include that in the `select=` argument to `table.rows_where(...)` - otherwise it will not be returned. A utility mechanism to make this easier would be very welcome.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/240/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 817528452,MDU6SXNzdWU4MTc1Mjg0NTI=,1244,Plugin tip: look at the examples linked from the hooks page,9599,simonw,closed,0,,,,,1,2021-02-26T17:18:27Z,2021-02-26T17:30:38Z,2021-02-26T17:27:15Z,OWNER,,"Someone asked ""what are good example plugins I can look at?"" and I realized that the answer is to look through the example links on https://docs.datasette.io/en/stable/plugin_hooks.html - but that tip should be written down somewhere on the https://docs.datasette.io/en/stable/writing_plugins.html page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1244/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 817597268,MDU6SXNzdWU4MTc1OTcyNjg=,1246,Suggest for ArrayFacet possibly confused by blank values,9599,simonw,closed,0,,,,,3,2021-02-26T19:11:52Z,2021-03-01T03:46:11Z,2021-03-01T03:46:11Z,OWNER,,I sometimes don't get the suggestion for facet-by-array for columns that contain arrays. I think it may be because they have empty spaces in them - or perhaps it's because the null detection doesn't actually work.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1246/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 818430405,MDU6SXNzdWU4MTg0MzA0MDU=,1247,datasette.add_memory_database() method,9599,simonw,closed,0,,,,,2,2021-03-01T03:48:38Z,2021-03-01T04:02:26Z,2021-03-01T04:02:26Z,OWNER,,"I just wrote this code: https://github.com/simonw/datasette/blob/47eb885cc2c3aafa03645c330c6f597bee9b3b25/tests/test_facets.py#L334-L335 It would be nice if you didn't have to separately instantiate a database object here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1247/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 824064069,MDU6SXNzdWU4MjQwNjQwNjk=,1249,Updated Dockerfile with SpatiaLite version 5.0,9599,simonw,closed,0,,,,,45,2021-03-08T00:17:36Z,2022-01-20T21:29:43Z,2021-03-29T00:57:13Z,OWNER,,"The version bundled in Datasette's Docker image right now is 4.4.0-RC0 https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/Dockerfile#L16-L17 5 has been out for a couple of months and has a bunch of big improvements, most notable stable KNN support.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 824067604,MDU6SXNzdWU4MjQwNjc2MDQ=,1250,Research: Plugin hook for alternative database connections,9599,simonw,closed,0,,,,,2,2021-03-08T00:28:15Z,2021-03-12T01:01:25Z,2021-03-12T01:01:17Z,OWNER,,"The `Database` class is a natural looking fit for a plugin hook to load custom database connections... potentially even databases other than SQLite. DuckDB (refs #968) could make for a great starting point, since it looks very compatible with the existing SQLite code. The real win would be if this could lead to running Datasette against PostgreSQL. I made some initial explorations in that direction a while ago in #670.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1250/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 828811618,MDU6SXNzdWU4Mjg4MTE2MTg=,1257,Table names containing single quotes break things,9599,simonw,closed,0,,,,,2,2021-03-11T06:29:38Z,2021-06-02T03:28:29Z,2021-06-02T03:28:29Z,OWNER,,"e.g. I found a table called `Yesterday's ELRs by County` It threw an error inside the `detect_fts()` function attempting to run this SQL query: ```sql select name from sqlite_master where rootpage = 0 and ( sql like '%VIRTUAL TABLE%USING FTS%content=""Yesterday's ELRs by County""%' or sql like '%VIRTUAL TABLE%USING FTS%content=[Yesterday's ELRs by County]%' or ( tbl_name = ""Yesterday's ELRs by County"" and sql like '%VIRTUAL TABLE%USING FTS%' ) ) ``` Here's the code at fault: https://github.com/simonw/datasette/blob/640ac7071b73111ba4423812cd683756e0e1936b/datasette/utils/__init__.py#L534-L548",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1257/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 836273891,MDU6SXNzdWU4MzYyNzM4OTE=,1266,Documentation for Response.asgi_send(send) method,9599,simonw,closed,0,,,,,1,2021-03-19T18:52:49Z,2021-03-20T21:35:00Z,2021-03-20T21:32:28Z,OWNER,,"I found myself wanting to use this method for https://github.com/simonw/datasette-auth-passwords/issues/15 - but it's not documented. It should be documented. https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/asgi.py#L320-L340",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 837308703,MDU6SXNzdWU4MzczMDg3MDM=,1268,Figure out why SpatiaLite 5.0 hangs the database page on Linux,9599,simonw,closed,0,,,,,18,2021-03-22T04:44:16Z,2022-01-20T21:29:43Z,2021-03-22T17:41:12Z,OWNER,,"See detailed notes in https://github.com/simonw/datasette/issues/1249 - for some reason SpatiaLite 5.0 hangs the `/dbname` page on Linux (inside Docker containers, both with a custom compiled SpatiaLite and one installed from the Ubuntu 20.10 package repository). This doesn't happen on macOS with SpatiaLite 5 installed using Homebrew.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1268/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 837348479,MDU6SXNzdWU4MzczNDg0Nzk=,1269,Don't attempt to run count(*) against virtual tables,9599,simonw,closed,0,,,,,2,2021-03-22T05:57:43Z,2021-03-22T17:40:42Z,2021-03-22T17:40:41Z,OWNER,,"Counting the rows in a virtual table doesn't seem very interesting to me, and it's the cause of at least one crashing bug with SpatiaLite 5.0 on Linux, see https://github.com/simonw/datasette/issues/1268",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1269/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 838148087,MDU6SXNzdWU4MzgxNDgwODc=,250,Handle byte order marks (BOMs) in CSV files,9599,simonw,closed,0,,,,,3,2021-03-22T22:13:18Z,2021-05-29T05:34:21Z,2021-05-29T05:34:21Z,OWNER,,I often find `sqlite-utils insert ... --csv` creates a first column with a weird character at the start of it - which it turns out is the UTF-8 BOM. Fix that.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/250/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 841377702,MDU6SXNzdWU4NDEzNzc3MDI=,251,"""sqlite-utils convert"" command to replace the separate ""sqlite-transform"" tool",9599,simonw,closed,0,,,,,15,2021-03-25T22:36:36Z,2021-08-02T22:39:46Z,2021-08-02T04:47:40Z,OWNER,,"See https://github.com/simonw/sqlite-transform/issues/11 - I built a separate `sqlite-transform` tool a while ago that uses the word ""transform"" to means something entirely different from `sqlite-utils transform` - I'd like to resolve this by merging the two tools.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/251/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 842212586,MDU6SXNzdWU4NDIyMTI1ODY=,1277,Facet by array breaks if table name contains a space,9599,simonw,closed,0,,,,,1,2021-03-26T18:38:19Z,2021-03-27T03:49:38Z,2021-03-27T03:49:34Z,OWNER,,It breaks when you try to select a filtered item.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1277/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 842416110,MDU6SXNzdWU4NDI0MTYxMTA=,1278,SpatiaLite timezones demo is broken,9599,simonw,closed,0,,,,,2,2021-03-27T04:45:27Z,2022-01-20T21:29:43Z,2021-03-27T16:17:13Z,OWNER,,https://github.com/simonw/datasette/blob/5fd02890650db790b2ffdb90eb9f78f8e0639c37/docs/spatialite.rst#L96,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1278/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 842881221,MDU6SXNzdWU4NDI4ODEyMjE=,1281,Latest Datasette tags missing from Docker Hub,9599,simonw,closed,0,,,,,7,2021-03-29T00:58:30Z,2021-03-29T01:41:48Z,2021-03-29T01:41:48Z,OWNER,,"Spotted this while testing https://github.com/simonw/datasette/issues/1249#issuecomment-808998719_ https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated isn't showing the tags for any version more recent than 0.54.1 - we are up to 0.56 now. But the `:latest` tag is for the new 0.56 release.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1281/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 849543502,MDU6SXNzdWU4NDk1NDM1MDI=,1289,Speed up tests with pytest-xdist,9599,simonw,closed,0,,,,,3,2021-04-03T00:47:39Z,2021-04-03T03:42:28Z,2021-04-03T03:42:28Z,OWNER,,"I think I can get this working for almost every test, then use the pattern in https://github.com/pytest-dev/pytest-xdist/issues/385#issuecomment-444545641 to opt specific tests out of being run in parallel.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1289/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 849568079,MDExOlB1bGxSZXF1ZXN0NjA4MzIzMDI4,1290,Use pytest-xdist to speed up tests,9599,simonw,closed,0,,,,,1,2021-04-03T03:34:36Z,2021-04-03T03:42:29Z,2021-04-03T03:42:28Z,OWNER,simonw/datasette/pulls/1290,"Closes #1289, refs #1212.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1290/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 858501079,MDU6SXNzdWU4NTg1MDEwNzk=,255,transform --help should tell you the available types,9599,simonw,closed,0,,,,,0,2021-04-15T05:24:48Z,2021-05-29T03:55:52Z,2021-05-29T03:55:52Z,OWNER,,"``` Usage: sqlite-utils transform [OPTIONS] PATH TABLE Transform a table beyond the capabilities of ALTER TABLE Options: --type ... Change column type to X ``` This should specify that the possible types are 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/255/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 866668415,MDU6SXNzdWU4NjY2Njg0MTU=,1308,"Columns named ""link"" display in bold",9599,simonw,closed,0,,,,,3,2021-04-24T05:58:11Z,2021-04-24T06:07:49Z,2021-04-24T06:07:49Z,OWNER,,Reported in office hours today.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1308/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 894948100,MDU6SXNzdWU4OTQ5NDgxMDA=,259,Suggest the --alter option if a new column cannot be added,9599,simonw,closed,0,,,,,1,2021-05-19T03:17:38Z,2021-05-19T03:27:33Z,2021-05-19T03:26:26Z,OWNER,,Refs #256.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/259/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 898904402,MDU6SXNzdWU4OTg5MDQ0MDI=,1337,"""More"" link for facets that shows _facet_size=max results",9599,simonw,closed,0,,,,,7,2021-05-23T00:08:51Z,2021-05-27T16:14:14Z,2021-05-27T16:01:03Z,OWNER,,"_Original title: ""More"" link for facets that shows the full set of results_ The simplest way to do this will be to have it link to a generated SQL query. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1332#issuecomment-846479062_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1337/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 899169307,MDU6SXNzdWU4OTkxNjkzMDc=,1338,Fix jinja2 warnings,9599,simonw,closed,0,,,,,0,2021-05-24T01:38:23Z,2021-05-24T01:41:55Z,2021-05-24T01:41:55Z,OWNER,,"Lots of these in the test suite now, after the Jinja upgrade in #1331: ``` tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:45: DeprecationWarning: 'jinja2.escape' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.escape' instead. label=jinja2.escape(data[""label""] or """") or "" "", tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:41: DeprecationWarning: 'jinja2.Markup' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.Markup' instead. return jinja2.Markup( ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1338/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 899851083,MDExOlB1bGxSZXF1ZXN0NjUxNDkyODg4,1339,?_col=/?_nocol= to show/hide columns on the table page,9599,simonw,closed,0,,,,,1,2021-05-24T17:15:20Z,2021-05-27T04:17:44Z,2021-05-27T04:17:43Z,OWNER,simonw/datasette/pulls/1339,"See #615. Still to do: - [x] Allow combination of `?_col=` and `?_nocol=` (`_nocol` wins) - [x] Deduplicate same column if passed in `?_col=` multiple times - [x] Validate that user did not try to remove a primary key - [x] Add tests - [x] Ensure this works correctly for SQL views - [x] Add documentation ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1339/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 903200328,MDU6SXNzdWU5MDMyMDAzMjg=,1341,"""Show all columns"" cog menu item should show if ?_col= is used",9599,simonw,closed,0,,,,,1,2021-05-27T04:28:17Z,2021-05-27T04:31:16Z,2021-05-27T04:31:16Z,OWNER,,"On https://latest.datasette.io/fixtures/sortable?_col=sortable the ""Show all columns"" item (from #615) is not shown (it should be): ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1341/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 903978133,MDU6SXNzdWU5MDM5NzgxMzM=,1343,Figure out how to publish alpha/beta releases to Docker Hub,9599,simonw,closed,0,,,,,4,2021-05-27T16:42:17Z,2021-05-27T16:46:37Z,2021-05-27T16:45:41Z,OWNER,,"> It looks like all I need to do to ship an alpha version to Docker Hub is NOT point the `latest` tag at it after it goes live: https://github.com/simonw/datasette/blob/1a8972f9c012cd22b088c6b70661a9c3d3847853/.github/workflows/publish.yml#L75-L77 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1319#issuecomment-849780481_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1343/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 904071938,MDU6SXNzdWU5MDQwNzE5Mzg=,1345,?_nocol= does not interact well with default facets,9599,simonw,closed,0,,,,,7,2021-05-27T18:39:55Z,2021-05-31T02:40:44Z,2021-05-31T02:31:21Z,OWNER,,"Clicking ""Hide this column"" on `fips` on https://covid-19.datasettes.com/covid/ny_times_us_counties shows this error: https://covid-19.datasettes.com/covid/ny_times_us_counties?_nocol=fips > ## Invalid SQL > no such column: fips The reason is that https://covid-19.datasettes.com/-/metadata sets up the following: ```json ""ny_times_us_counties"": { ""sort_desc"": ""date"", ""facets"": [ ""state"", ""county"", ""fips"" ], ``` It's setting `fips` as a default facet, which breaks if you attempt to remove the column using `?_nocol`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1345/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 904537568,MDExOlB1bGxSZXF1ZXN0NjU1Njg0NDc3,1346,Re-display user's query with an error message if an error occurs,9599,simonw,closed,0,,,,,3,2021-05-28T02:04:20Z,2021-06-02T03:46:21Z,2021-06-02T03:46:21Z,OWNER,simonw/datasette/pulls/1346,Refs #619,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1346/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 906330187,MDU6SXNzdWU5MDYzMzAxODc=,260,Support creating descending order indexes,9599,simonw,closed,0,,,,,12,2021-05-29T03:42:59Z,2021-05-29T05:01:39Z,2021-05-29T05:01:39Z,OWNER,,"SQLite lets you create indexes in reverse order, which can have a surprisingly big impact on performance, see https://github.com/simonw/covid-19-datasette/issues/27 I tried doing this using `sqlite-utils` like so, but it's didn't work: ```python db[""ny_times_us_counties""].create_index([""date desc""]) ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/260/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 906345899,MDU6SXNzdWU5MDYzNDU4OTk=,261,`table.xindexes` using `PRAGMA index_xinfo(table)`,9599,simonw,closed,0,,,,,5,2021-05-29T04:23:48Z,2021-06-03T03:54:14Z,2021-06-03T03:51:32Z,OWNER,,"> `PRAGMA index_xinfo(table)` DOES return that data: > ``` > (Pdb) [c[0] for c in fresh_db.execute(""PRAGMA > index_xinfo('idx_dogs_age_name')"").description] > ['seqno', 'cid', 'name', 'desc', 'coll', 'key'] > (Pdb) fresh_db.execute(""PRAGMA index_xinfo('idx_dogs_age_name')"").fetchall() > [(0, 2, 'age', 1, 'BINARY', 1), (1, 0, 'name', 0, 'BINARY', 1), (2, -1, None, 0, 'BINARY', 0)] > ``` > See https://sqlite.org/pragma.html#pragma_index_xinfo > > Example output: https://covid-19.datasettes.com/covid?sql=select+*+from+pragma_index_xinfo%28%27idx_ny_times_us_counties_date%27%29 _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850766552_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/261/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 906355849,MDExOlB1bGxSZXF1ZXN0NjU3MzczNzI2,262,Ability to add descending order indexes,9599,simonw,closed,0,,,,,0,2021-05-29T04:51:04Z,2021-05-29T05:01:42Z,2021-05-29T05:01:39Z,OWNER,simonw/sqlite-utils/pulls/262,Refs #260,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/262/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 906356331,MDU6SXNzdWU5MDYzNTYzMzE=,263,`sqlite-utils indexes` command,9599,simonw,closed,0,,,,,6,2021-05-29T04:52:34Z,2021-06-03T04:34:38Z,2021-06-03T04:34:38Z,OWNER,,"While working on #260 I realized there's no command to show indexes in a database, even though there is one for showing tables and one for triggers. I should implement #261 first.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/263/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 906385991,MDU6SXNzdWU5MDYzODU5OTE=,1349,CSV ?_stream=on redundantly calculates facets for every page,9599,simonw,closed,0,,,,,9,2021-05-29T06:11:23Z,2021-06-17T18:12:32Z,2021-06-01T15:52:53Z,OWNER,,"I'm trying to figure out why a full CSV export from https://covid-19.datasettes.com/covid/ny_times_us_counties runs unbearably slowly. It's because the streaming endpoint works by scrolling through every page, and it turns out every page calculates facets and suggested facets!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1349/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 906977719,MDU6SXNzdWU5MDY5Nzc3MTk=,1350,?_nofacets=1 query string argument for disabling facets and suggested facets,9599,simonw,closed,0,,,,,2,2021-05-31T02:22:29Z,2021-06-01T16:19:38Z,2021-05-31T02:39:18Z,OWNER,,"This is needed as an internal option for #1349. `datasette-graphql` can benefit from this too - maybe can even use it so that if you pass `?_shape=array` it gets automatically added, fixing #263.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1350/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 906993731,MDU6SXNzdWU5MDY5OTM3MzE=,1351,Get `?_trace=1` working with CSV and streaming CSVs,9599,simonw,closed,0,,,,,1,2021-05-31T03:02:15Z,2021-06-17T18:12:32Z,2021-06-01T15:50:09Z,OWNER,,"> I think it's worth getting `?_trace=1` to work with streaming CSV - this would have helped me spot this issue a long time ago. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1349#issuecomment-851133125_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1351/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 908446997,MDU6SXNzdWU5MDg0NDY5OTc=,1353,?_nocount=1 for opting out of table counts,9599,simonw,closed,0,,,,,2,2021-06-01T15:53:27Z,2021-06-01T16:18:54Z,2021-06-01T16:17:04Z,OWNER,,"Running a trace against a CSV streaming export with the new `_trace=1` feature from #1351 shows that the following code is executing a `select count(*) from table` for every page of results returned: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/table.py#L700-L705 This is inefficient - a new `?_nocount=1` option would let us disable this count in the same way as #1349: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/base.py#L264-L276 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1353/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 908465747,MDU6SXNzdWU5MDg0NjU3NDc=,1354,Update help in tests for latest Click,9599,simonw,closed,0,,,,,1,2021-06-01T16:14:31Z,2021-06-01T16:17:04Z,2021-06-01T16:17:04Z,OWNER,,"Now that Uvicorn 0.14 is out with an unpinned Click dependency - https://github.com/encode/uvicorn/pull/1033 - our test suite runs against Click 8.0 - which subtly changes the output of `--help` causing test failures: https://github.com/simonw/datasette/runs/2720383031?check_suite_focus=true ``` def test_help_includes(name, filename): expected = (docs_path / filename).read_text() runner = CliRunner() result = runner.invoke(cli, name.split() + [""--help""], terminal_width=88) actual = f""$ datasette {name} --help\n\n{result.output}"" # actual has ""Usage: cli package [OPTIONS] FILES"" # because it doesn't know that cli will be aliased to datasette expected = expected.replace(""Usage: datasette"", ""Usage: cli"") > assert expected == actual E AssertionError: assert '$ datasette ...e and exit.\n' == '$ datasette ...e and exit.\n' E Skipping 848 identical leading characters in diff, use -v to show E nt_id xxx E + E --version-note TEXT Additional note to show on /-/versions E --secret TEXT Secret used for signing secure values, such as signed E cookies E + E --title TEXT Title for metadata ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1354/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 912394511,MDExOlB1bGxSZXF1ZXN0NjYyNTU3MjQw,1357,Make custom pages compatible with base_url setting,9599,simonw,closed,0,,,,,1,2021-06-05T18:54:39Z,2021-06-05T18:59:54Z,2021-06-05T18:59:54Z,OWNER,simonw/datasette/pulls/1357,Refs #1238.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1357/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 912418094,MDU6SXNzdWU5MTI0MTgwOTQ=,1358,Release Datasette 0.57,9599,simonw,closed,0,,,,,3,2021-06-05T19:56:13Z,2021-06-05T22:20:07Z,2021-06-05T22:20:07Z,OWNER,,"Need release notes. Changes are here: https://github.com/simonw/datasette/compare/0.56...368aa5f1b16ca35f82d90ff747023b9a2bfa27c1 Partial release notes already exist for the two alphas, https://github.com/simonw/datasette/releases/tag/0.57a0 and https://github.com/simonw/datasette/releases/tag/0.57a1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1358/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 912419349,MDU6SXNzdWU5MTI0MTkzNDk=,1359,`?_trace=1` should only be available with a new `trace_debug` setting,9599,simonw,closed,0,,,,,0,2021-06-05T19:59:27Z,2021-06-05T20:18:46Z,2021-06-05T20:18:46Z,OWNER,,Just like template debug mode is controlled by this off-by-default setting: https://github.com/simonw/datasette/blob/368aa5f1b16ca35f82d90ff747023b9a2bfa27c1/datasette/app.py#L160-L164,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1359/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 912464443,MDU6SXNzdWU5MTI0NjQ0NDM=,1360,"Security flaw, to be fixed in 0.56.1 and 0.57",9599,simonw,closed,0,,,,,2,2021-06-05T21:53:51Z,2021-06-05T22:23:23Z,2021-06-05T22:22:06Z,OWNER,,"See security advisory here for details: https://github.com/simonw/datasette/security/advisories/GHSA-xw7c-jx9m-xh5g - the `?_trace=1` debugging option was not correctly escaping its JSON output, resulting in a [reflected cross-site scripting](https://owasp.org/www-community/attacks/xss/#reflected-xss-attacks) vulnerability.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1360/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 912485040,MDU6SXNzdWU5MTI0ODUwNDA=,1361,Intermittent CI failure: restore_working_directory FileNotFoundError,9599,simonw,closed,0,,,,,4,2021-06-05T22:48:13Z,2021-06-05T23:16:24Z,2021-06-05T23:16:24Z,OWNER,,"e.g. in https://github.com/simonw/datasette/runs/2754772233 - this is an intermittent error: ``` __________ ERROR at setup of test_hook_register_routes_render_message __________ [gw0] linux -- Python 3.8.10 /opt/hostedtoolcache/Python/3.8.10/x64/bin/python tmpdir = local('/tmp/pytest-of-runner/pytest-0/popen-gw0/test_hook_register_routes_rend0') request = > @pytest.fixture def restore_working_directory(tmpdir, request): > previous_cwd = os.getcwd() E FileNotFoundError: [Errno 2] No such file or directory ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1361/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 912959264,MDU6SXNzdWU5MTI5NTkyNjQ=,1364,Don't truncate columns on the list of databases,9599,simonw,closed,0,,,,,0,2021-06-06T22:01:56Z,2021-06-06T22:07:50Z,2021-06-06T22:07:50Z,OWNER,,"https://covid-19.datasettes.com/covid currently truncates at 9 database columns: Django SQL Dashboard showed me that this is a bad idea - having the full list of columns is actually really useful documentation for crafting custom SQL queries.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1364/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 913135723,MDU6SXNzdWU5MTMxMzU3MjM=,266,"Add some types, enforce with mypy",9599,simonw,closed,0,,,,,3,2021-06-07T06:05:56Z,2021-08-18T22:25:38Z,2021-08-18T22:25:38Z,OWNER,,"A good starting point would be adding type information to the members of these named tuples and the introspection methods that return them: https://github.com/simonw/sqlite-utils/blob/9dff7a38831d471b1dff16d40d89eb5c3b4e84d6/sqlite_utils/db.py#L51-L75",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 913823889,MDU6SXNzdWU5MTM4MjM4ODk=,1367,Navigation menu display bug,9599,simonw,closed,0,,,,,1,2021-06-07T18:18:08Z,2021-06-07T18:24:19Z,2021-06-07T18:24:19Z,OWNER,,"With Datasette 0.57 the navigation menu looks like this: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1367/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 915455228,MDU6SXNzdWU5MTU0NTUyMjg=,1371,Menu plugin hooks should include the request,9599,simonw,closed,0,,,,,1,2021-06-08T20:23:35Z,2021-06-10T04:46:01Z,2021-06-10T04:46:01Z,OWNER,,"https://docs.datasette.io/en/stable/plugin_hooks.html#menu-links-datasette-actor - `menu_links(datasette, actor)` - `table_actions(datasette, actor, database, table)` - `database_actions(datasette, actor, database)` All three of these should optionally also accept the `request` object. This would allow them to take into account additional cookies, `Authorization` headers or the current request URL (including the domain/subdomain) - or even access `request.scope` for extra context that might have been passed down from ASGI middleware.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1371/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 919181559,MDU6SXNzdWU5MTkxODE1NTk=,268,db.schema property and sqlite-utils schema command,9599,simonw,closed,0,,,,,4,2021-06-11T20:25:47Z,2021-06-11T20:51:56Z,2021-06-11T20:51:56Z,OWNER,,"`table.schema` returns the schema for a table. `db.schema` should return the schema for the whole databes. Can do this using `select sql from sqlite_master where sql is not null`: https://latest.datasette.io/fixtures?sql=select+sql+from+sqlite_master+where+sql+is+not+null",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/268/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 919702451,MDU6SXNzdWU5MTk3MDI0NTE=,271,table.upsert_all() fails if input has a single column that should be a primary key,9599,simonw,closed,0,,,,,1,2021-06-13T02:50:27Z,2021-06-13T02:57:29Z,2021-06-13T02:57:29Z,OWNER,,"This works: ```pycon >>> db['foo'].insert_all([{""name"": ""hello""}], pk=""name"")
    ``` But this fails: ``` >>> db['foo3'].upsert_all([{""name"": ""hello""}], pk=""name"") Traceback (most recent call last): File """", line 1, in File ""/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1837, in upsert_all return self.insert_all( File ""/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1778, in insert_all self.insert_chunk( File ""/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1588, in insert_chunk result = self.db.execute(query, params) File ""/Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py"", line 213, in execute return self.conn.execute(sql, parameters) sqlite3.OperationalError: near ""WHERE"": syntax error ``` With the debugger: ``` >>> import pdb; pdb.pm() > /Users/simon/.local/share/virtualenvs/datasette.io-TK86ygSO/lib/python3.9/site-packages/sqlite_utils/db.py(213)execute() -> return self.conn.execute(sql, parameters) (Pdb) print(sql, parameters) UPDATE [foo3] SET WHERE [name] = ? ['hello'] ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/271/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 919733213,MDU6SXNzdWU5MTk3MzMyMTM=,33,Searching for whitespace throws an error,9599,simonw,closed,0,,,,,0,2021-06-13T06:57:57Z,2021-06-13T14:36:39Z,2021-06-13T14:36:39Z,MEMBER,,"https://datasette.io/-/beta?q=+ returns a 500 > fts5: syntax error near """"",197431109,dogsheep-beta,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/33/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 920884085,MDU6SXNzdWU5MjA4ODQwODU=,1377,Mechanism for plugins to exclude certain paths from CSRF checks,9599,simonw,closed,0,,,,,3,2021-06-15T00:48:20Z,2021-06-23T22:51:33Z,2021-06-23T22:51:33Z,OWNER,,I need this for a plugin I'm building that offers a POST API.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1377/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 921878733,MDU6SXNzdWU5MjE4Nzg3MzM=,272,"Idea: import CSV to memory, run SQL, export in a single command",9599,simonw,closed,0,,,,,22,2021-06-15T23:02:48Z,2021-06-19T23:36:48Z,2021-06-18T15:05:03Z,OWNER,,"I quite often load a CSV file into a SQLite DB, then do stuff with it (like export results back out again as a new CSV) without any intention of keeping the CSV file around afterwards. What if `sqlite-utils` could do this for me? Something like this: sqlite-utils --csv blah.csv --csv baz.csv ""select * from blah join baz ..."" ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/272/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 922099793,MDExOlB1bGxSZXF1ZXN0NjcxMDE0NzUx,273,sqlite-utils memory command for directly querying CSV/JSON data,9599,simonw,closed,0,,,,,8,2021-06-16T05:04:58Z,2021-06-18T15:01:17Z,2021-06-18T15:00:52Z,OWNER,simonw/sqlite-utils/pulls/273,"Refs #272. Initial implementation only does CSV data, still needs: - [x] Implement `--save` - [x] Add `--dump` to the documentation - [x] Add `--attach` example to the documentation - [x] Replace `:memory:` in documentation",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/273/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 922832113,MDU6SXNzdWU5MjI4MzIxMTM=,274,sqlite-utils dump my.db command,9599,simonw,closed,0,,,,,0,2021-06-16T16:30:14Z,2021-06-16T23:51:54Z,2021-06-16T23:51:54Z,OWNER,,"Inspired by the `--dump` mechanism I added to `sqlite-utils memory` here: https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862018937 > Can use `.iterdump()` to implement this: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.iterdump > > Maybe instead (or as-well-as) offer `--dump` which dumps out the SQL from that.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/274/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 922955697,MDU6SXNzdWU5MjI5NTU2OTc=,275,Enable code coverage,9599,simonw,closed,0,,,,,1,2021-06-16T18:33:49Z,2021-06-17T00:12:12Z,2021-06-17T00:12:12Z,OWNER,,"https://app.codecov.io/gh/simonw/sqlite-utils Same mechanism as Datasette. Need to copy across the token from that page and add an equivalent of this workflow: https://github.com/simonw/datasette/blob/main/.github/workflows/test-coverage.yml",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/275/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 924990677,MDU6SXNzdWU5MjQ5OTA2Nzc=,279,sqlite-utils memory should handle TSV and JSON in addition to CSV,9599,simonw,closed,0,,,,,7,2021-06-18T15:02:54Z,2021-06-19T03:11:59Z,2021-06-19T03:11:59Z,OWNER,,"- Use sniff to detect CSV or TSV (if `:tsv` or `:csv` was not specified) and delimiters Follow-on from #272",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/279/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 924991194,MDU6SXNzdWU5MjQ5OTExOTQ=,280,Add --encoding option to sqlite-utils memory,9599,simonw,closed,0,,,,,0,2021-06-18T15:03:32Z,2021-06-18T15:29:46Z,2021-06-18T15:29:46Z,OWNER,,Follow-on from #272 - this will work like `--encoding` on `sqlite-utils insert` and will affect all CSV files processed by `sqlite-utils memory`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/280/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 924992318,MDU6SXNzdWU5MjQ5OTIzMTg=,281,Mechanism for explicitly stating CSV or JSON or TSV for sqlite-utils memory,9599,simonw,closed,0,,,,,1,2021-06-18T15:04:53Z,2021-06-19T03:11:59Z,2021-06-19T03:11:59Z,OWNER,,"- Implement `filename.json:json` and `-:nl` and suchlike options for specifying the format rather than guessing it - see https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861985944 Follows #272",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/281/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 925305186,MDU6SXNzdWU5MjUzMDUxODY=,282,Automatic type detection for CSV data,9599,simonw,closed,0,,,,,4,2021-06-19T03:33:21Z,2021-06-19T04:42:03Z,2021-06-19T04:38:00Z,OWNER,,"I've touched on this before in #179 - but now that I've added `sqlite-utils memory` this is much more important - because unlike with `sqlite-utils insert` the in-memory command doesn't give you the opportunity to fix any types you imported from CSV, so queries like `select * from stdin where age > 3` are never going to work correctly against these temporary in-memory tables. Teaching `sqlite-utils insert` to detect types for columns in a CSV file would be a backwards-compatibility breaking change. Teaching `sqlite-utils memory` that trick would not be, since it hasn't been included in a release yet. It's a little inconsistent, but I'm going to have `sqlite-utils memory` default to detecting types while `sqlite-utils insert` does not. In each case this can be controlled by a new command-line option: cat file.csv | sqlite-utils memory - --no-detect-types To opt-in for `sqlite-utils insert`: cat file.csv | sqlite-utils insert blah.db blah - --detect-types I'll have short options for these too: `-n` for `--no-detect-types` and `-d` for `--detect-types`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/282/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,completed 925319214,MDU6SXNzdWU5MjUzMTkyMTQ=,283,memory: Shouldn't detect types for JSON,9599,simonw,closed,0,,,,,1,2021-06-19T05:17:35Z,2021-06-19T14:52:48Z,2021-06-19T14:52:48Z,OWNER,,"https://github.com/simonw/sqlite-utils/blob/ec5174ed40fa283cb06f25ee0c0136297ec313ae/sqlite_utils/cli.py#L1244-L1251 This runs against JSON as well as CSV/TSV - which isn't necessary and In fact throws errors if there is any nested data.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/283/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 925320167,MDU6SXNzdWU5MjUzMjAxNjc=,284,.transform(types=) turns rowid into a concrete column,9599,simonw,closed,0,,,,,5,2021-06-19T05:25:27Z,2021-06-19T15:28:30Z,2021-06-19T15:28:30Z,OWNER,,"Noticed this in the tests for `sqlite-utils memory` in #282 - is it possible to fix this? https://github.com/simonw/sqlite-utils/commit/ec5174ed40fa283cb06f25ee0c0136297ec313ae",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/284/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 925410305,MDU6SXNzdWU5MjU0MTAzMDU=,285,Introspection property for telling if a table is a rowid table,9599,simonw,closed,0,,,,,7,2021-06-19T14:56:16Z,2021-06-19T15:12:33Z,2021-06-19T15:12:33Z,OWNER,,_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/284#issuecomment-864416785_,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/285/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 925487946,MDU6SXNzdWU5MjU0ODc5NDY=,286,Add installation instructions,9599,simonw,closed,0,,,,,1,2021-06-19T23:55:36Z,2021-06-20T18:47:13Z,2021-06-20T18:47:13Z,OWNER,,"`pip install sqlite-utils`, `pipx install sqlite-utils` and `brew install sqlite-utils`",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/286/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 925544070,MDU6SXNzdWU5MjU1NDQwNzA=,287,Update rowid examples in the docs,9599,simonw,closed,0,,,,,0,2021-06-20T08:03:00Z,2021-06-20T18:26:21Z,2021-06-20T18:26:21Z,OWNER,,Changed in #284 - a couple of examples need updating on https://github.com/simonw/sqlite-utils/blob/3.10/docs/cli.rst.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 925545468,MDU6SXNzdWU5MjU1NDU0Njg=,288,sqlite-utils memory blah.json --schema,9599,simonw,closed,0,,,,,0,2021-06-20T08:10:40Z,2021-06-20T18:26:21Z,2021-06-20T18:26:21Z,OWNER,,Like `--dump` but only outputs the schema - useful for understanding what you are about to run queries against.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/288/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 926777310,MDU6SXNzdWU5MjY3NzczMTA=,290,`db.query()` method (renamed `db.execute_returning_dicts()`),9599,simonw,closed,0,,,,,6,2021-06-22T03:03:54Z,2021-06-24T23:17:38Z,2021-06-24T22:54:43Z,OWNER,,"Most of this library deals with lists of Python dictionaries - `.insert_all()`, `.rows`, `.rows_where()`, `.search()`. The `db.execute()` method is the only thing that returns a `sqlite3` cursor. There is a clumsily named `db.execute_returning_dicts(sql)` method but it's not currently mentioned in the documentation. It needs a better name, and needs to be properly documented.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/290/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 927766296,MDU6SXNzdWU5Mjc3NjYyOTY=,291,Adopt flake8,9599,simonw,closed,0,,,,,2,2021-06-23T01:19:37Z,2021-06-24T17:50:27Z,2021-06-24T17:50:27Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/291/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 927789811,MDU6SXNzdWU5Mjc3ODk4MTE=,292,Add contributing documentation,9599,simonw,closed,0,,,,,0,2021-06-23T02:13:05Z,2021-06-25T17:53:51Z,2021-06-25T17:53:51Z,OWNER,,Like https://docs.datasette.io/en/latest/contributing.html (but simpler) - should cover how to run `black` and `flake8` and `mypy` and how to run the tests.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/292/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 929748885,MDExOlB1bGxSZXF1ZXN0Njc3NTU0OTI5,293,Test against Python 3.10-dev,9599,simonw,closed,0,,,,,3,2021-06-25T01:40:39Z,2021-10-13T21:49:33Z,2021-10-13T21:49:33Z,OWNER,simonw/sqlite-utils/pulls/293,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/293/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 931752773,MDU6SXNzdWU5MzE3NTI3NzM=,294,Add a `sqlite-utils memory` example to the README,9599,simonw,closed,0,,,,,0,2021-06-28T16:35:59Z,2021-08-18T21:40:03Z,2021-08-18T21:40:03Z,OWNER,,,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/294/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 935930820,MDU6SXNzdWU5MzU5MzA4MjA=,1387,absolute_url() behind a proxy assembles incorrect http://127.0.0.1:8001/ URLs,9599,simonw,closed,0,,,,,8,2021-07-02T16:58:25Z,2021-07-02T17:58:23Z,2021-07-02T17:33:05Z,OWNER,,"Reported in the wild on https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-3d4a4b7/bib?_facet=bib_level_callnumber - the ""next page"" link links to https://127.0.0.1:8010/collection-analysis/current_collection-3d4a4b7/bib?_facet=bib_level_callnumber&_next=100 That installation uses `""base_url"": ""/collection-analysis/""` Weirdly all of the other links on that page - to facet results, sort orders, row permalinks etc - work fine. It's JUST the `next_url` one that is broken. Also broken in their JSON: https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-3d4a4b7/bib.json?_size=1 returns ```json ""suggested_facets"": [], ""next"": ""1"", ""next_url"": ""https://127.0.0.1:8010/collection-analysis/current_collection-3d4a4b7/bib.json?_size=1&_next=1"", ""private"": false, ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1387/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 940077168,MDU6SXNzdWU5NDAwNzcxNjg=,1389,"""searchmode"": ""raw"" in table metadata",9599,simonw,closed,0,,,,,6,2021-07-08T17:32:10Z,2021-07-10T18:33:13Z,2021-07-10T18:33:13Z,OWNER,,"> http://localhost:8001/index/summary?_search=language%3Aeng&_sort=title&_searchmode=raw > > But I'm not able to manage it in the metadata file. Here is mine (note that the sort column is taken into account) > Here it is: > > ``` > { > ""databases"": { > ""index"": { > ""tables"": { > ""summary"": { > ""sort"": ""title"", > ""searchmode"": ""raw"" > } > } > } > } > } _Originally posted by @Krazybug in https://github.com/simonw/datasette/issues/759#issuecomment-624860451_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1389/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 940891698,MDU6SXNzdWU5NDA4OTE2OTg=,1390,Mention restarting systemd in documentation,9599,simonw,closed,0,,,,,2,2021-07-09T16:05:15Z,2021-07-09T16:32:57Z,2021-07-09T16:32:33Z,OWNER,,"https://docs.datasette.io/en/stable/deploying.html#running-datasette-using-systemd Need to clarify that if you add a new database or change metadata you need to restart systemd.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1390/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 941300946,MDU6SXNzdWU5NDEzMDA5NDY=,1391,Stop using generated columns in fixtures.db,9599,simonw,closed,0,,,,,5,2021-07-10T18:26:11Z,2021-07-10T19:26:58Z,2021-07-10T19:26:00Z,OWNER,,"Refs #1376 - but I also keep running into this myself, where I try to run something against `fixtures.db` and get this confusing error: sqlite3.DatabaseError: malformed database schema (generated_columns) - near ""AS"": syntax error I'm going to stop using generated columns in `fixtures.db` and instead dynamically generate the generated column table for the duration of the relevant test.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1391/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 944870799,MDU6SXNzdWU5NDQ4NzA3OTk=,1394,Big performance boost on faceting: skip the inner order by,9599,simonw,closed,0,,,,,4,2021-07-14T23:32:29Z,2021-07-16T02:23:32Z,2021-07-15T00:05:50Z,OWNER,,"I just noticed something that could make for a huge performance improvement in faceting. The default query used by Datasette when faceting looks like this: ```sql select country_long, count(*) from ( select * from [global-power-plants] order by rowid ) where country_long is not null group by country_long order by count(*) desc ``` Here it takes 53ms: https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D+order+by+rowid%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc Note that there's a `order by rowid` in there which isn't necessary - the order on that inner query doesn't matter since we're grouping and counting. I had assumed SQLite would optimize this away - but it turns out it doesn't! Consider this version of the query, with that pointless order by removed: ``` select country_long, count(*) from ( select * from [global-power-plants] ) where country_long is not null group by country_long order by count(*) desc ``` https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc runs in 7.2ms! I tried this optimization on a table with 2.5m rows in it - without the optimization it took 5 seconds, with the optimization it took 450ms. So this is a very significant improvement!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1394/reactions"", ""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 944903881,MDU6SXNzdWU5NDQ5MDM4ODE=,1396,"""invalid reference format"" publishing Docker image",9599,simonw,closed,0,,,,,9,2021-07-15T01:02:07Z,2021-10-19T08:10:26Z,2021-07-15T19:47:25Z,OWNER,,"Error ocurred at the end of the publish flow for Datasette 0.58: https://github.com/simonw/datasette/runs/3072216421 ``` Removing intermediate container cf32b9440907 ---> dfd6985b2afc Successfully built dfd6985b2afc Successfully tagged ***/datasette:0.58 invalid reference format Error: Process completed with exit code 1. ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1396/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 946553953,MDExOlB1bGxSZXF1ZXN0NjkxNzA3NDA5,1397,"Fix for race condition in refresh_schemas(), closes #1231",9599,simonw,closed,0,,,,,0,2021-07-16T19:44:43Z,2021-07-16T19:45:00Z,2021-07-16T19:44:58Z,OWNER,simonw/datasette/pulls/1397,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1397/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 952154468,MDU6SXNzdWU5NTIxNTQ0Njg=,299,Ability to see just specific table schemas with `sqlite-utils schema`,9599,simonw,closed,0,,,,,1,2021-07-24T22:00:05Z,2021-07-24T22:12:01Z,2021-07-24T22:08:46Z,OWNER,,"It currently accepts no arguments. Allowing for optional arguments specifying tables would be useful: sqlite-utils schema fixtures.db facetable searchable ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/299/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 953352015,MDU6SXNzdWU5NTMzNTIwMTU=,1404,`register_routes()` hook should take `datasette` argument,9599,simonw,closed,0,,,,,1,2021-07-26T23:00:33Z,2021-07-26T23:27:07Z,2021-07-26T23:26:00Z,OWNER,,Currently that plugin hook takes no arguments at all. This means it's not possible to conditionally register routes based on Datasette plugin configuration.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1404/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 955316250,MDU6SXNzdWU5NTUzMTYyNTA=,1405,utils.parse_metadata() should be a documented internal function,9599,simonw,closed,0,,,,,3,2021-07-28T23:51:39Z,2021-07-29T23:33:30Z,2021-07-29T23:30:24Z,OWNER,,Because it's used by this plugin: https://github.com/simonw/datasette-remote-metadata,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1405/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 956303470,MDU6SXNzdWU5NTYzMDM0NzA=,1406,Tests failing with FileNotFoundError in runner.isolated_filesystem,9599,simonw,closed,0,,,,,8,2021-07-30T00:39:00Z,2021-07-31T18:56:35Z,2021-07-31T18:56:35Z,OWNER,,"e.g. https://github.com/simonw/datasette/runs/3197141955 I've seen this error before, but I don't yet have a good workaround for it. ``` @contextlib.contextmanager def isolated_filesystem( self, temp_dir: t.Optional[t.Union[str, os.PathLike]] = None ) -> t.Iterator[str]: """"""A context manager that creates a temporary directory and changes the current working directory to it. This isolates tests that affect the contents of the CWD to prevent them from interfering with each other. :param temp_dir: Create the temporary directory under this directory. If given, the created directory is not removed when exiting. .. versionchanged:: 8.0 Added the ``temp_dir`` parameter. """""" > cwd = os.getcwd() E FileNotFoundError: [Errno 2] No such file or directory /opt/hostedtoolcache/Python/3.6.14/x64/lib/python3.6/site-packages/click/testing.py:466: FileNotFoundError =========================== short test summary info ============================ FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_apt_get_install FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[---setting force_https_urls on] FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[--setting base_url /foo---setting base_url /foo --setting force_https_urls on] FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[--setting force_https_urls off---setting force_https_urls off] FAILED tests/test_publish_heroku.py::test_publish_heroku_requires_heroku - Fi... FAILED tests/test_publish_heroku.py::test_publish_heroku_installs_plugin - Fi... FAILED tests/test_publish_heroku.py::test_publish_heroku - FileNotFoundError:... FAILED tests/test_publish_heroku.py::test_publish_heroku_plugin_secrets - Fil... ================== 8 failed, 920 passed in 188.22s (0:03:08) =================== ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1406/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 957298475,MDU6SXNzdWU5NTcyOTg0NzU=,1407,OSError: AF_UNIX path too long in ds_unix_domain_socket_server,9599,simonw,closed,0,,,,,2,2021-07-31T18:36:06Z,2021-07-31T19:03:44Z,2021-07-31T19:03:44Z,OWNER,,"Got this exception while working on #1406. ``` @pytest.fixture(scope=""session"") def ds_unix_domain_socket_server(tmp_path_factory): socket_folder = tmp_path_factory.mktemp(""uds"") uds = str(socket_folder / ""datasette.sock"") ds_proc = subprocess.Popen( [""datasette"", ""--memory"", ""--uds"", uds], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=tempfile.gettempdir(), ) # Give the server time to start time.sleep(1.5) # Check it started successfully > assert not ds_proc.poll(), ds_proc.stdout.read().decode(""utf-8"") E AssertionError: INFO: Started server process [48453] E INFO: Waiting for application startup. E INFO: Application startup complete. E Traceback (most recent call last): E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/datasette"", line 33, in E sys.exit(load_entry_point('datasette', 'console_scripts', 'datasette')()) E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__ E return self.main(*args, **kwargs) E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1062, in main E rv = self.invoke(ctx) E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke E return _process_result(sub_ctx.command.invoke(sub_ctx)) E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke E return ctx.invoke(self.callback, **ctx.params) E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 763, in invoke E return __callback(*args, **kwargs) E File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 583, in serve E uvicorn.run(ds.app(), **uvicorn_kwargs) E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/main.py"", line 393, in run E server.run() E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 50, in run E loop.run_until_complete(self.serve(sockets=sockets)) E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py"", line 616, in run_until_complete E return future.result() E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 67, in serve E await self.startup(sockets=sockets) E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 133, in startup E server = await asyncio.start_unix_server( E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/streams.py"", line 132, in start_unix_server E return await loop.create_unix_server(factory, path, **kwds) E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/unix_events.py"", line 296, in create_unix_server E sock.bind(path) E OSError: AF_UNIX path too long E E assert not 1 E + where 1 = >() E + where > = .poll ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1407/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 957345476,MDU6SXNzdWU5NTczNDU0NzY=,1411,Canned query ?sql= is pointlessly echoed in query string starting from hidden mode,9599,simonw,closed,0,,,,,1,2021-08-01T00:17:13Z,2021-08-01T03:27:30Z,2021-08-01T00:58:17Z,OWNER,,"Example: https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 Submitting that form again results in this: https://latest.datasette.io/fixtures/neighborhood_search?sql=%0D%0Aselect+neighborhood%2C+facet_cities.name%2C+state%0D%0Afrom+facetable%0D%0A++++join+facet_cities%0D%0A++++++++on+facetable.city_id+%3D+facet_cities.id%0D%0Awhere+neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0D%0Aorder+by+neighborhood%3B%0D%0A&_hide_sql=1&text=cork Because the HTML on https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 includes this: ```html

    Custom SQL query returning 1 row (show)

    ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1411/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 957383814,MDU6SXNzdWU5NTczODM4MTQ=,301,insert-files should get a --silent option,9599,simonw,closed,0,,,,,0,2021-08-01T04:11:03Z,2021-08-02T19:12:21Z,2021-08-02T19:12:21Z,OWNER,,"The new `sqlite-utils convert` command I'm adding in #251 will have a `--silent` option for turning off the progress bars. The only other command that has progress bars right now is `insert-files` so it should get this option too, for consistency.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/301/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 957529248,MDU6SXNzdWU5NTc1MjkyNDg=,302,Python library version of `sqlite-utils convert`,9599,simonw,closed,0,9599,simonw,,,1,2021-08-01T16:11:02Z,2021-08-02T04:47:40Z,2021-08-02T04:47:40Z,OWNER,,"Spin off from #251. The ability to execute Python functions to convert and split columns should be part of the library too, not just the CLI.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/302/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 957536983,MDExOlB1bGxSZXF1ZXN0NzAwOTQ0NjQ0,303,sqlite-utils convert command and db[table].convert(...) method,9599,simonw,closed,0,,,,,1,2021-08-01T16:52:42Z,2021-08-02T04:47:42Z,2021-08-02T04:47:39Z,OWNER,simonw/sqlite-utils/pulls/303,"Refs #251, #302. - [x] Get recipes working - [x] Document recipes - [x] Implement `db[table].convert(...)` method - [x] Add tests for recipes that use the new Python method - [x] Implement `db[table].convert(..., multi=True)` mechanism - [x] Documentation for `db[table].convert(...)` - [x] Refactor `sqlite-utils convert` to use the new method",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/303/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 957731178,MDU6SXNzdWU5NTc3MzExNzg=,304,"`table.convert(..., where=)` and `sqlite-utils convert ... --where=`",9599,simonw,closed,0,,,,,3,2021-08-02T04:27:23Z,2021-08-02T19:00:00Z,2021-08-02T18:58:10Z,OWNER,,"For applying the conversion to a subset of rows selected using the where clause. Should also take optional arguments, as seen in `db[""dogs""].delete_where(""age < ?"", [3])`. Follows #302 and #251. This was originally https://github.com/simonw/sqlite-transform/issues/9",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/304/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 957741820,MDU6SXNzdWU5NTc3NDE4MjA=,305,Python: need a way to execute a count with an extra where clause,9599,simonw,closed,0,,,,,1,2021-08-02T04:52:02Z,2021-08-02T05:08:22Z,2021-08-02T05:08:22Z,OWNER,,I need this for #304. I'll probably add this to the `.execute_count()` method as `where=` and `where_args=`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/305/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 958516743,MDU6SXNzdWU5NTg1MTY3NDM=,306,Configure sphinx.ext.extlinks for issues,9599,simonw,closed,0,,,,,2,2021-08-02T21:19:19Z,2021-08-02T21:39:34Z,2021-08-02T21:29:22Z,OWNER,,As seen in Datasette: https://github.com/simonw/datasette/issues/1227,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/306/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 959276629,MDU6SXNzdWU5NTkyNzY2Mjk=,1416,"Use rich to render tracebacks on errors, if available",9599,simonw,closed,0,,,,,0,2021-08-03T16:12:08Z,2021-08-03T16:12:51Z,2021-08-03T16:12:51Z,OWNER,,"> Now thinking I should try adding Rich as an optional dependency to Datasette - if it's there, show tracebacks using it. Could be really handy for development > https://twitter.com/simonw/status/1422576091055616003",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1416/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 959278472,MDU6SXNzdWU5NTkyNzg0NzI=,1417,Use codespell in CI to spot spelling errors,9599,simonw,closed,0,,,,,1,2021-08-03T16:14:15Z,2021-08-03T16:36:40Z,2021-08-03T16:36:40Z,OWNER,,"I noticed Rich is using this: https://github.com/willmcgugan/rich/commit/9c12a4537499797c43725fff5276ef0da62423ef#diff-ce84a1b2c9eb4ab3ea22f610cad7111cb9a2f66365c3b24679901376a2a73ab2 Ran it against the Datasette docs and found a bunch of obvious fixes, surprisingly with no false positives. ``` datasette % codespell docs/*.rst docs/authentication.rst:63: perfom ==> perform docs/authentication.rst:76: perfom ==> perform docs/changelog.rst:429: repsonse ==> response docs/changelog.rst:503: permissons ==> permissions docs/changelog.rst:717: compatibilty ==> compatibility docs/changelog.rst:1172: browseable ==> browsable docs/deploying.rst:191: similiar ==> similar docs/internals.rst:434: Respons ==> Response, respond docs/internals.rst:440: Respons ==> Response, respond docs/internals.rst:717: tha ==> than, that, the docs/performance.rst:42: databse ==> database docs/plugin_hooks.rst:667: utilites ==> utilities docs/publish.rst:168: countainer ==> container docs/settings.rst:352: inalid ==> invalid docs/sql_queries.rst:406: preceeded ==> preceded, proceeded ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1417/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 959284434,MDExOlB1bGxSZXF1ZXN0NzAyNDIyMjYz,1418,Spelling corrections plus CI job for codespell,9599,simonw,closed,0,,,,,2,2021-08-03T16:21:19Z,2021-08-03T16:36:39Z,2021-08-03T16:36:38Z,OWNER,simonw/datasette/pulls/1418,Refs #1417.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1418/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 959305209,MDU6SXNzdWU5NTkzMDUyMDk=,307,codespell to spell check documentation,9599,simonw,closed,0,,,,,0,2021-08-03T16:48:19Z,2021-08-03T16:48:53Z,2021-08-03T16:48:53Z,OWNER,,As seen in https://github.com/simonw/datasette/issues/1417 and https://til.simonwillison.net/python/codespell,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/307/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 959898166,MDU6SXNzdWU5NTk4OTgxNjY=,1420,`datasette publish cloudrun --cpu X` option,9599,simonw,closed,0,,,,,5,2021-08-04T05:04:31Z,2021-08-05T00:54:59Z,2021-08-04T05:33:48Z,OWNER,,"For setting the number of vCPUs - current valid values are 1, 2 or 4: https://cloud.google.com/run/docs/configuring/cpu Pass that through to `gcloud run deploy --image IMAGE_URL --cpu CPU`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1420/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 961367843,MDU6SXNzdWU5NjEzNjc4NDM=,1422,Ability to default to hiding the SQL for a canned query,9599,simonw,closed,0,,,,,4,2021-08-05T02:51:39Z,2021-08-07T05:32:29Z,2021-08-07T05:32:29Z,OWNER,,"I'm working on a project with some HUGE (400+ lines of SQL) canned queries right now. Any time you land on the canned query page you have to scroll down a long distance to get to the results! Would be useful to be able to default to https://latest.datasette.io/fixtures/magic_parameters?_hide_sql=1 without needing the parameter.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1422/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 962391325,MDU6SXNzdWU5NjIzOTEzMjU=,1423,Show count of facet values if ?_facet_size=max,9599,simonw,closed,0,,,,,9,2021-08-06T04:42:20Z,2021-12-15T17:48:40Z,2021-08-16T18:56:43Z,OWNER,,"I sometimes want to get a count of the values in a facet - if it's a facet of US states for example I want to know if all 50 are represented. Idea: if `?_facet_size=max` is present, add a count to the facet heading. So on: https://latest.datasette.io/fixtures/compound_three_primary_keys?_facet=content&_facet_size=max&_facet=pk1&_facet=pk2#facet-pk2 It could have something like this: Note that the first column shows >1000 - because in that case we've truncated the facet calculation since the maximum allowed returned rows is 1000.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1423/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 963528457,MDU6SXNzdWU5NjM1Mjg0NTc=,1425,render_cell() hook should support returning an awaitable,9599,simonw,closed,0,,,,,11,2021-08-08T22:32:29Z,2021-08-09T07:14:35Z,2021-08-09T03:00:37Z,OWNER,,"Many of the plugin hooks can return an awaitable - e.g. https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-extra-template-vars - but `render_cell()` doesn't support this. I recently found myself wanting to execute an additional SQL query from that hook, but it wasn't possible to do that since I couldn't use `await`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 964400482,MDU6SXNzdWU5NjQ0MDA0ODI=,310,`sqlite-utils insert --flatten` option to flatten nested JSON,9599,simonw,closed,0,,,,,3,2021-08-09T21:23:08Z,2021-10-16T13:54:56Z,2021-08-09T21:44:06Z,OWNER,,"I had to do this with a `jq` recipe today: https://til.simonwillison.net/cloudrun/tailing-cloud-run-request-logs ``` cat log.json | jq -c '[leaf_paths as $path | { ""key"": $path | join(""_""), ""value"": getpath($path) }] | from_entries' \ | sqlite-utils insert /tmp/logs.db logs - --nl --alter --batch-size 1 ``` That was to turn something like this: ```json { ""httpRequest"": { ""latency"": ""0.112114537s"", ""requestMethod"": ""GET"", ""requestSize"": ""534"", ""status"": 200, }, ""insertId"": ""6111722f000b5b4c4d4071e2"", ""labels"": { ""service"": ""datasette-io"" } } ``` Into this instead: ```json { ""httpRequest_latency"": ""0.112114537s"", ""httpRequest_requestMethod"": ""GET"", ""httpRequest_requestSize"": ""534"", ""httpRequest_status"": 200, ""insertId"": ""6111722f000b5b4c4d4071e2"", ""labels_service"": ""datasette-io"" } ``` I have to do this often enough that I think it should be an option, `--flatten` - so I can do this instead: ``` cat log.json | sqlite-utils insert /tmp/logs.db logs - --flatten ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/310/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 965102534,MDU6SXNzdWU5NjUxMDI1MzQ=,311,Add reference documentation generated from docstrings,9599,simonw,closed,0,,,,,4,2021-08-10T16:04:00Z,2021-08-11T12:03:50Z,2021-08-11T12:03:50Z,OWNER,,"Using https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html I'm not a big fan of this kind of documentation because it so often comes in place of narrative documentation - but the library has great narrative documentation now, so the reference documentation can link to it in places. This will also encourage me to add good docstrings everywhere, useful for IDEs and suchlike.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/311/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 965143346,MDExOlB1bGxSZXF1ZXN0NzA3NDkwNzg5,312,Add reference page to documentation using Sphinx autodoc,9599,simonw,closed,0,,,,,10,2021-08-10T16:59:17Z,2021-08-10T23:09:32Z,2021-08-10T23:09:28Z,OWNER,simonw/sqlite-utils/pulls/312,Refs #311.,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/312/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 965166058,MDU6SXNzdWU5NjUxNjYwNTg=,313,`.add_foreign_keys()` doesn't reject being called with a View,9599,simonw,closed,0,,,,,0,2021-08-10T17:22:17Z,2021-08-10T17:25:34Z,2021-08-10T17:25:34Z,OWNER,,"Spotted this bug using `mypy` while working on #311 / #312! ``` % mypy sqlite_utils sqlite_utils/db.py:725: error: Item ""View"" of ""Union[Table, View]"" has no attribute ""foreign_keys"" Found 1 error in 1 file (checked 5 source files) ``` Refers to this code: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L710-L720 It's a bug! We run some checks earlier but none of them ensure that it's a view: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L697-L709",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/313/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 965210966,MDU6SXNzdWU5NjUyMTA5NjY=,314,Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__`,9599,simonw,closed,0,,,,,2,2021-08-10T18:03:59Z,2021-08-18T22:25:21Z,2021-08-18T22:25:21Z,OWNER,,"> Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/312#issuecomment-896200682_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/314/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 965440017,MDU6SXNzdWU5NjU0NDAwMTc=,315,`.delete_where()` returns `[]` when it should return self,9599,simonw,closed,0,,,,,1,2021-08-10T21:54:55Z,2021-08-10T23:09:29Z,2021-08-10T23:09:29Z,OWNER,,"If the table doesn't exist it should still return `self`, not `[]`: https://github.com/simonw/sqlite-utils/blob/ee469e3122d6f5973ec2584c1580d930daca2e7c/sqlite_utils/db.py#L1676-L1683 Spotted with `mypy` while working on #312.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/315/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 969758038,MDExOlB1bGxSZXF1ZXN0NzExNzgzNjE2,1430,Column metadata,9599,simonw,closed,0,,,,,1,2021-08-12T23:34:39Z,2021-08-12T23:53:23Z,2021-08-12T23:53:23Z,OWNER,simonw/datasette/pulls/1430,"Refs #942 Still needs: - [x] Tests - [x] Documentation",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1430/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 969840302,MDU6SXNzdWU5Njk4NDAzMDI=,1431,`--help-config` should be called `--help-settings`,9599,simonw,closed,0,,,,,1,2021-08-13T00:46:48Z,2021-08-13T01:01:58Z,2021-08-13T01:01:58Z,OWNER,,Follow-on from #1105 rebranding exercise.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1431/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 970320615,MDU6SXNzdWU5NzAzMjA2MTU=,316,Fix visible backticks on reference page,9599,simonw,closed,0,,,,,1,2021-08-13T11:37:46Z,2021-08-14T05:12:23Z,2021-08-14T05:10:48Z,OWNER,,"https://sqlite-utils.datasette.io/en/latest/reference.html Search for backtick to reveal various minor markup bugs.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/316/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 972827346,MDU6SXNzdWU5NzI4MjczNDY=,317,Link to a better example on docs index,9599,simonw,closed,0,,,,,1,2021-08-17T15:43:40Z,2021-08-18T18:31:43Z,2021-08-18T18:31:43Z,OWNER,,https://github.com/simonw/sqlite-utils/blob/7a19822ac9ee24be2fbb4c2326a0bf2f3d2d9c4d/docs/index.rst#L39 Is a very old example,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/317/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 974987856,MDU6SXNzdWU5NzQ5ODc4NTY=,1442,Mechanism to cause specific branches to deploy their own demos,9599,simonw,closed,0,,,,,6,2021-08-19T19:41:39Z,2021-08-19T21:11:45Z,2021-08-19T21:09:40Z,OWNER,,"A useful capability would be if it was super-easy to say ""any pushes to branch X should be deployed to `latest-X.datasette.io`"". I'd like to use this for the column query information work in #1434",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1442/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 974995592,MDU6SXNzdWU5NzQ5OTU1OTI=,1443,datasette.databases should be a documented property,9599,simonw,closed,0,,,,,1,2021-08-19T19:53:04Z,2021-08-19T21:25:07Z,2021-08-19T21:23:47Z,OWNER,,"https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/app.py#L231 I want to use it in https://github.com/simonw/datasette-block-robots/issues/5",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1443/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 975049826,MDExOlB1bGxSZXF1ZXN0NzE2MjYyODI5,1444,Ability to deploy demos of branches,9599,simonw,closed,0,,,,,0,2021-08-19T21:08:04Z,2021-08-19T21:09:44Z,2021-08-19T21:09:39Z,OWNER,simonw/datasette/pulls/1444,See #1442.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1444/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 975158266,MDU6SXNzdWU5NzUxNTgyNjY=,19,table activity_summary has no column named appleMoveTime,9599,simonw,closed,0,,,,,0,2021-08-20T00:46:44Z,2021-08-20T00:54:34Z,2021-08-20T00:54:34Z,MEMBER,,"Got this error today against a fresh export: table activity_summary has no column named appleMoveTime ",197882382,healthkit-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/19/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 976405225,MDU6SXNzdWU5NzY0MDUyMjU=,320,sqlite-utils memory --analyze option,9599,simonw,closed,0,,,,,2,2021-08-22T15:37:10Z,2021-08-22T15:46:56Z,2021-08-22T15:44:29Z,OWNER,,To provide a way of running [analyze-tables](https://sqlite-utils.datasette.io/en/stable/cli.html#analyzing-tables) directly against JSON or CSV data.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/320/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 978357984,MDU6SXNzdWU5NzgzNTc5ODQ=,1446,Modify base.html template to support optional sticky footer,9599,simonw,closed,0,,,,,3,2021-08-24T18:11:12Z,2021-08-31T01:54:59Z,2021-08-24T20:32:47Z,OWNER,,"The neatest way to have the footer stick to the bottom of the browser window that I've found is to use the flexbox pattern from https://css-tricks.com/couple-takes-sticky-footer/ ```html
    content
    ``` ```css html, body { height: 100%; } body { display: flex; flex-direction: column; } .content { flex: 1 0 auto; } .footer { flex-shrink: 0; } ``` I tried this in a custom plugin but it ended up having to duplicate the entire `base.html` template just to get a wrapper around the not-footer content. I think Datasette's own `base.html` template should have this wrapper element instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1446/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 978537855,MDExOlB1bGxSZXF1ZXN0NzE5MTA5NzA5,321,"Ability to insert file contents as text, in addition to blob",9599,simonw,closed,0,,,,,5,2021-08-24T22:37:18Z,2021-08-24T23:31:17Z,2021-08-24T23:31:13Z,OWNER,simonw/sqlite-utils/pulls/321,Refs #319.,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/321/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 978743426,MDU6SXNzdWU5Nzg3NDM0MjY=,13,xml.etree.ElementTree.ParseError: not well-formed (invalid token),9599,simonw,closed,0,,,,,4,2021-08-25T05:48:21Z,2021-08-26T18:45:13Z,2021-08-26T18:45:13Z,MEMBER,,"Got this error today: ``` (evernote-to-sqlite) /tmp % evernote-to-sqlite enex evernote.db simonwillison\'s\ notebook.enex Importing from ENEX [######------------------------------] 17% Traceback (most recent call last): File ""/Users/simon/.local/bin/evernote-to-sqlite"", line 8, in sys.exit(cli()) File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1137, in __call__ return self.main(*args, **kwargs) File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1062, in main rv = self.invoke(ctx) File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1668, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py"", line 763, in invoke return __callback(*args, **kwargs) File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/cli.py"", line 31, in enex save_note(db, note) File ""/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/utils.py"", line 36, in save_note content = ET.tostring(ET.fromstring(content_xml)).decode(""utf-8"") File ""/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/xml/etree/ElementTree.py"", line 1347, in XML parser.feed(text) xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 2, column 132 ```",303218369,evernote-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 979627285,MDU6SXNzdWU5Nzk2MjcyODU=,323,`table.convert()` method should clean up after itself,9599,simonw,closed,0,,,,,1,2021-08-25T21:15:39Z,2021-08-25T21:25:26Z,2021-08-25T21:25:18Z,OWNER,,"It currently works like this: https://github.com/simonw/sqlite-utils/blob/77c240df56068341561e95e4a412cbfa24dc5bc7/sqlite_utils/db.py#L2177-L2195 It's registering a function called `convert_value()` and then failing to de-register that function once it has finished. It might even be possible for two queries running against the same connection to clobber each other's `convert_value()` functions, leading to incorrect behaviour. So two fixes: firstly it should register the function with a unique name (maybe add a random suffix). Secondly, it should de-register that function once it has finished.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/323/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 981676832,MDU6SXNzdWU5ODE2NzY4MzI=,1449,`register_commands()` plugin hook to register extra CLI commands,9599,simonw,closed,0,,,,,13,2021-08-28T00:26:21Z,2021-08-28T01:58:35Z,2021-08-28T01:43:11Z,OWNER,,"The `datasette` CLI tool currently has 7 subcommands: `serve`, `inspect`, `install`, `package`, `plugins`, `publish` and `uninstall`. A plugin hook could allow plugins to register extra subcommands. I've avoided this for quite a while because I didn't have good use-cases for it - but the existence of the `datasette install xxx` command for installing packages into the correct virtual environment means that actually there's a good reason to do this: it would allow plugins to provide additional command-line mechanisms without the user having to understand how virtual environments work in order to install those commands into the same environment as the rest of Datasette.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1449/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 981681138,MDU6SXNzdWU5ODE2ODExMzg=,1450,"Datasette --help should show something more useful than ""Datasette!""",9599,simonw,closed,0,,,,,1,2021-08-28T00:44:51Z,2021-08-28T00:49:07Z,2021-08-28T00:49:07Z,OWNER,,"https://github.com/simonw/datasette/blob/a1a33bb5822214be1cebd98cd858b2058d91a4aa/datasette/cli.py#L122-L127 _Originally spotted by @simonw in https://github.com/simonw/datasette/issues/1449#issuecomment-907539668_ ``` ~ % datasette --help Usage: datasette [OPTIONS] COMMAND [ARGS]... Datasette! Options: --version Show the version and exit. --help Show this message and exit. ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1450/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 989986586,MDU6SXNzdWU5ODk5ODY1ODY=,1461,Try blacken-docs,9599,simonw,closed,0,,,,,3,2021-09-07T13:28:50Z,2021-09-07T16:13:59Z,2021-09-07T16:13:59Z,OWNER,,https://github.com/asottile/blacken-docs,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1461/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 991237645,MDExOlB1bGxSZXF1ZXN0NzI5NzMxNDQx,326,Test against 3.10-dev,9599,simonw,closed,0,,,,,2,2021-09-08T15:01:15Z,2021-10-13T21:49:28Z,2021-10-13T21:49:28Z,OWNER,simonw/sqlite-utils/pulls/326,"This tests against the latest 3.10 RC, https://www.python.org/downloads/release/python-3100rc2/",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/326/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 994450961,MDU6SXNzdWU5OTQ0NTA5NjE=,1469,"Column cog shows ""facet by this"" when already default faceted",9599,simonw,closed,0,,,,,2,2021-09-13T04:51:26Z,2021-10-13T21:20:07Z,2021-10-13T21:20:07Z,OWNER,,"e.g. on https://covid-19.datasettes.com/covid/economist_excess_deaths But if you add `?_facet=country` to the URL that goes away: https://covid-19.datasettes.com/covid/economist_excess_deaths?_facet_size=5&_facet=country The logic that decides if the ""Facet by this"" item is shown does not take default `metadata.json` facets into account.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1469/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1005891028,I_kwDOCGYnMM479K3U,329,Rethink approach to [ and ] in column names (currently throws error),9599,simonw,closed,0,,,,,12,2021-09-23T22:14:24Z,2021-11-15T02:57:51Z,2021-11-15T02:57:51Z,OWNER,,"> I think it's best to still keep `[` and `]` out of column names though. Transforming them into `(` and `)` seems reasonable - but should that happen here or in `sqlite-utils`? I think in `sqlite-utils`. _Originally posted by @simonw in https://github.com/simonw/datasette-app/issues/121#issuecomment-926200398_ This is a rethinking of the solution to: - https://github.com/simonw/sqlite-utils/issues/86",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/329/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1020436713,PR_kwDOBm6k_c4s6bJm,1481,Fix compatibility with Python 3.10,9599,simonw,closed,0,,,,,13,2021-10-07T20:34:23Z,2021-10-24T22:19:55Z,2021-10-24T22:19:54Z,OWNER,simonw/datasette/pulls/1481,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1481/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1021550542,I_kwDOBm6k_c4845_O,1482,Support Python 3.10,9599,simonw,closed,0,,,,,2,2021-10-09T00:30:52Z,2021-10-24T22:21:40Z,2021-10-24T22:19:55Z,OWNER,,"I started work on this in #1481 where I found a Python 3.10 bug that needs a workaround in Janus, see: - https://github.com/aio-libs/janus/issues/358 This is a tracking issue for anything else that shows up. This is also needed for the Homebrew package to upgrade to 3.10: - https://github.com/Homebrew/homebrew-core/pull/86932",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1482/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1021849766,I_kwDOBm6k_c486DCm,1483,Running a search on page 2 of results should not preserve ?_next=,9599,simonw,closed,0,,,,,0,2021-10-10T01:18:12Z,2021-10-13T21:08:10Z,2021-10-13T21:08:10Z,OWNER,,Reported by @eigenfoo in https://github.com/simonw/datasette/issues/1470,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1483/reactions"", ""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1025726600,PR_kwDOCGYnMM4tKxHD,330,Test against Python 3.10,9599,simonw,closed,0,,,,,1,2021-10-13T21:50:22Z,2021-11-15T02:59:29Z,2021-10-13T22:25:05Z,OWNER,simonw/sqlite-utils/pulls/330,,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/330/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1025754125,I_kwDOBm6k_c49I8QN,1488,Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects'),9599,simonw,closed,0,,,,,5,2021-10-13T22:37:22Z,2021-10-14T18:03:45Z,2021-10-14T18:03:45Z,OWNER,,This is caused by a change made to `httpx` in https://github.com/encode/httpx/releases/tag/0.20.0,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1488/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1026664511,PR_kwDOBm6k_c4tNtoe,1490,Upgrade to httpx 0.20,9599,simonw,closed,0,,,,,0,2021-10-14T17:51:05Z,2021-10-14T18:03:45Z,2021-10-14T18:03:44Z,OWNER,simonw/datasette/pulls/1490,Refs #1488 ,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1490/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1033864602,I_kwDOBm6k_c49n4Wa,1496,Named parameters docs should include an example of a cast,9599,simonw,closed,0,,,,,1,2021-10-22T18:56:04Z,2021-10-22T19:38:23Z,2021-10-22T19:34:27Z,OWNER,,"https://docs.datasette.io/en/stable/sql_queries.html#named-parameters It's not obvious that the values from parameters are always SQLite strings, which means that you can't do e.g. integer comparisons on them without casting them first. The documentation here should include an example of this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1496/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1034535001,I_kwDOBm6k_c49qcBZ,1497,"Publish to Docker Hub failing with ""libcrypt.so.1: cannot open shared object file""",9599,simonw,closed,0,,,,,18,2021-10-24T22:57:07Z,2023-01-18T17:13:45Z,2021-10-24T23:36:55Z,OWNER,,"This means the Datasette 0.59.1 release has not been published to Docker Hub. Here's where that failed: https://github.com/simonw/datasette/runs/3991043374?check_suite_focus=true ``` Preparing to unpack .../libc6_2.32-4_amd64.deb ... debconf: unable to initialize frontend: Dialog debconf: (TERM is not set, so the dialog frontend is not usable.) debconf: falling back to frontend: Readline debconf: unable to initialize frontend: Readline debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.) debconf: falling back to frontend: Teletype Checking for services that may need to be restarted... Checking init scripts... Unpacking libc6:amd64 (2.32-4) over (2.28-10) ... Setting up libc6:amd64 (2.32-4) ... /usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory dpkg: error processing package libc6:amd64 (--configure): installed libc6:amd64 package post-installation script subprocess returned error exit status 127 Errors were encountered while processing: libc6:amd64 E: Sub-process /usr/bin/dpkg returned an error code (1) The command '/bin/sh -c apt-get update && apt-get -y --no-install-recommends install software-properties-common && add-apt-repository ""deb http://httpredir.debian.org/debian sid main"" && apt-get update && apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite && apt-get remove -y software-properties-common && apt clean && rm -rf /var/lib/apt && rm -rf /var/lib/dpkg/info/*' returned a non-zero code: 100 ``` Same problem when I attempted to publish using the ""Push specific Docker tag"" workflow: https://github.com/simonw/datasette/runs/3991059912?check_suite_focus=true",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1497/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1050163432,I_kwDOBm6k_c4-mDjo,1503,`?_nocol=` removes that column from the filter interface,9599,simonw,closed,0,,,,,1,2021-11-10T18:22:50Z,2021-11-14T05:08:27Z,2021-11-14T04:53:07Z,OWNER,,"e.g. on https://latest.datasette.io/fixtures/sortable?_nocol=sortable This causes weird behaviour when you e.g. facet by a hidden column, since selecting facets and then re-submitting the form will clear the selected filter. ![nocol-bug](https://user-images.githubusercontent.com/9599/141171135-aded71d1-a4cb-4b7f-a4ea-26828fa98906.gif) ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1503/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1052826038,I_kwDOBm6k_c4-wNm2,1506,Columns beginning with an underscore do not facet correctly,9599,simonw,closed,0,,,,,1,2021-11-14T02:20:32Z,2021-11-14T04:45:21Z,2021-11-14T04:45:21Z,OWNER,,"Datasette treats columns that start with an underscore as querystring parameters it should ignore! Discovered in https://github.com/simonw/git-history/issues/14#issuecomment-968192464",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1506/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1052851176,I_kwDOBm6k_c4-wTvo,1507,ReadTheDocs build failed for 0.59.2 release,9599,simonw,closed,0,,,,,6,2021-11-14T05:24:34Z,2021-11-14T05:41:55Z,2021-11-14T05:41:55Z,OWNER,,"I had to cancel the 0.59.2 release because ReadTheDocs was failing to build the documentation. https://readthedocs.org/projects/datasette/builds/15268454/ ``` /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m sphinx -T -b html -d _build/doctrees -D language=en . _build/html Running Sphinx v1.8.5 loading translations [en]... done making output directory... building [mo]: targets for 0 po files that are out of date building [html]: targets for 27 source files that are out of date updating environment: 27 added, 0 changed, 0 removed reading sources... [ 3%] authentication Traceback (most recent call last): File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/cmd/build.py"", line 304, in build_main app.build(args.force_all, filenames) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/application.py"", line 341, in build self.builder.build_update() File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 347, in build_update len(to_build)) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 360, in build updated_docnames = set(self.read()) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 468, in read self._read_serial(docnames) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 490, in _read_serial self.read_doc(docname) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 534, in read_doc doctree = read_doc(self.app, self.env, self.env.doc2path(docname)) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/io.py"", line 318, in read_doc pub.publish() File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py"", line 219, in publish self.apply_transforms() File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py"", line 200, in apply_transforms self.document.transformer.apply_transforms() File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py"", line 90, in apply_transforms Transformer.apply_transforms(self) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/transforms/__init__.py"", line 171, in apply_transforms transform.apply(**kwargs) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py"", line 245, in apply apply_source_workaround(n) File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py"", line 94, in apply_source_workaround for classifier in reversed(node.parent.traverse(nodes.classifier)): TypeError: argument to reversed() must be a sequence Exception occurred: File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py"", line 94, in apply_source_workaround for classifier in reversed(node.parent.traverse(nodes.classifier)): TypeError: argument to reversed() must be a sequence The full traceback has been saved in /tmp/sphinx-err-vkl0oE.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at . Thanks! ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1507/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1053087862,I_kwDOCGYnMM4-xNh2,338,"dict, list, tuple should all map to TEXT",9599,simonw,closed,0,,,,,0,2021-11-15T00:28:01Z,2021-11-15T00:36:03Z,2021-11-15T00:36:03Z,OWNER,,"> This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947 > > So the `COLUMN_TYPE_MAPPING` should include those too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188 _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/322#issuecomment-968401459_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/338/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1053122092,I_kwDOCGYnMM4-xV4s,339,`table.lookup()` option to populate additional columns when creating a record,9599,simonw,closed,0,,,,,4,2021-11-15T01:41:17Z,2021-11-15T02:02:34Z,2021-11-15T02:02:00Z,OWNER,,"> For the commits table I feel like I want a version of `table.lookup()` that can be passed additional columns to populate only if the record does not exist yet. _Originally posted by @simonw in https://github.com/simonw/git-history/issues/12#issuecomment-967455017_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/339/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1053136495,I_kwDOCGYnMM4-xZZv,341,`hash_id: Optional[Any]` should be `hash_id: Optional[str]`,9599,simonw,closed,0,,,,,0,2021-11-15T02:12:39Z,2021-11-15T02:19:31Z,2021-11-15T02:19:31Z,OWNER,,"In a few places: https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L642 https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L751 https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1049 https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1230 But it's correct here: https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L2470",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/341/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1055402144,PR_kwDOBm6k_c4unfnq,1512,New pattern for async view classes,9599,simonw,closed,0,,,,,7,2021-11-16T21:55:44Z,2021-11-17T01:39:54Z,2021-11-17T01:39:44Z,OWNER,simonw/datasette/pulls/1512,Refs #878 - starting out with the new `AsyncBase` class implementing a pytest-inspired `asyncio` parallel execution mechanism.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1512/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1, 1055469073,I_kwDOBm6k_c4-6S4R,1513,Research: CTEs and union all to calculate facets AND query at the same time,9599,simonw,closed,0,,,,,12,2021-11-16T22:26:45Z,2021-11-16T23:41:46Z,2021-11-16T23:41:46Z,OWNER,,"Consider this page: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_search=plant&_facet=owner&_facet=country_long&_facet=primary_fuel Datasette needs to run the main query for the rows on that page, a count query for the total query, then a separate query for each of those three specified facets. This is a `_search=` query, so it needs to execute the FTS code once for the rows, again for the count, and then three more times for each of the facets. Could running that query as a CTE and doing the other queries as part of the same large query produce significant speed improvements?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1513/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1058196641,I_kwDOCGYnMM4_Esyh,342,Extra options to `lookup()` which get passed to `insert()`,9599,simonw,closed,0,,,,,7,2021-11-19T06:53:03Z,2021-11-19T07:26:54Z,2021-11-19T07:26:54Z,OWNER,,"For https://github.com/simonw/git-history/issues/12 I found myself wanting to pass extra options to `lookup()` to set the column order, primary key etc.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/342/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1058815557,I_kwDOBm6k_c4_HD5F,1521,Docker configuration for exercising Datasette behind Apache mod_proxy,9599,simonw,closed,0,,,,,10,2021-11-19T18:46:18Z,2021-11-19T20:32:29Z,2021-11-19T20:32:29Z,OWNER,,"> Having a live demo running on Cloud Run that proxies through Apache and uses `base_url` would be incredibly useful for replicating and debugging this kind of thing. I wonder how hard it is to run Apache and `mod_proxy` in the same Docker container as Datasette? _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974310208_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1521/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1058896236,I_kwDOBm6k_c4_HXls,1522,Deploy a live instance of demos/apache-proxy,9599,simonw,closed,0,,,,,34,2021-11-19T20:32:55Z,2021-11-23T03:00:34Z,2021-11-20T18:51:56Z,OWNER,,"> I'll get this working on my laptop first, but then I want to get it up and running on Cloud Run - maybe with a GitHub Actions workflow in this repo that re-deploys it on manual execution. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1521#issuecomment-974322178_ I started by following https://ahmet.im/blog/cloud-run-multiple-processes-easy-way/ - see example in https://github.com/ahmetb/multi-process-container-lazy-solution",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1522/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1059219106,I_kwDOBm6k_c4_Imai,1524,"Improve Apache proxy documentation, link to demo",9599,simonw,closed,0,,,,,4,2021-11-20T20:03:14Z,2021-11-20T23:34:03Z,2021-11-20T23:34:03Z,OWNER,,"> The latest demo is now live at https://datasette-apache-proxy-demo.fly.dev/prefix/fixtures/sortable?_facet=pk2 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974697824_ I'm going to put out 0.59.3 bugfix release with this, but I'd like to first improve the documentation on https://docs.datasette.io/en/stable/deploying.html#apache-proxy-configuration to highlight the new demo.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1524/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1059509927,I_kwDOBm6k_c4_Jtan,1525,"""Links from other tables"" broken for columns starting with underscore",9599,simonw,closed,0,,,,,3,2021-11-21T22:55:08Z,2021-11-30T06:39:01Z,2021-11-30T06:34:35Z,OWNER,,"Same bug as #1506, this time it's this link or the row page: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1525/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1065432388,I_kwDOBm6k_c4_gTVE,1534,Maybe return JSON from HTML pages if `Accept: application/json` is sent,9599,simonw,closed,0,,,,,4,2021-11-28T20:48:09Z,2022-04-27T21:59:34Z,2022-02-02T23:39:33Z,OWNER,,"Relates to #1533 - and to the work I've been doing on the https://github.com/simonw/datasette-table Web Component. It would be useful to support users pasting in a URL to a Datasette table or query without first having to add the `.json` extension themselves - since then other systems could hit that URL with `Accept: application/json` to get back the JSON representation without first needing to read the `Link: ` header from #1533 to figure out what the URL to that JSON is. (There is weird logic deep in Datasette that says that you add `.json` to the path UNLESS the table name itself ends with `.json`, in which case you add `?_format=json` - this is super-confusing). [Update: I removed that confusing feature here: [https://simonwillison.net/2022/Mar/19/weeknotes/](https://simonwillison.net/2022/Mar/19/weeknotes/)]",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1534/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1066288689,I_kwDOBm6k_c4_jkYx,1538,Research pattern for re-registering existing Click tools with register_commands,9599,simonw,closed,0,,,,,3,2021-11-29T17:09:47Z,2021-11-29T17:32:44Z,2021-11-29T17:27:16Z,OWNER,,"Building a Datasette plugin that imports an existing Click CLI tool and re-registers it is proving hard - Click doesn't really want you to do that. I tried this: ```python from datasette import hookimpl from git_history.cli import file as git_history_file @hookimpl def register_commands(cli): cli.command(name=""git-history"")(git_history_file.callback) ``` But when I run this: ``` % datasette git-history --help Usage: datasette git-history [OPTIONS] Analyze the history of a specific file and write it to SQLite Options: --help Show this message and exit. ``` The options are all missing - which means that the command doesn't actually work. Will need to research this pattern separately. _Originally posted by @simonw in https://github.com/simonw/git-history/issues/21#issuecomment-981835305_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1538/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1066474200,I_kwDOCGYnMM4_kRrY,344,Support STRICT tables,9599,simonw,closed,0,,,,,14,2021-11-29T20:32:23Z,2023-12-08T05:22:39Z,2023-12-08T05:22:39Z,OWNER,,"New in SQLite 3.37.0, released a few days ago: https://www.sqlite.org/stricttables.html",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/344/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1066501534,I_kwDOCGYnMM4_kYWe,345,`table.strict` introspection boolean for identifying STRICT mode tables,9599,simonw,closed,0,,,,,2,2021-11-29T21:05:10Z,2021-11-29T22:45:26Z,2021-11-29T22:44:36Z,OWNER,,"> From the STRICT docs: >> The SQLite parser accepts a comma-separated list of table options after the final close parenthesis in a CREATE TABLE statement. As of this writing (2021-08-23) only two options are recognized: >> >> - STRICT >> - [WITHOUT ROWID](https://www.sqlite.org/withoutrowid.html) > > So I think I need to read the `CREATE TABLE` statement from the `sqlite_master` table, split on the last `)`, split those tokens on `,` and see if `create` is in there (case insensitive). _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982020757_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/345/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1072780607,I_kwDOCGYnMM4_8VU_,351,Support `--import xml.etree.ElementTree` in `sqlite-utils convert`,9599,simonw,closed,0,,,,,1,2021-12-07T00:40:29Z,2021-12-11T00:11:25Z,2021-12-11T00:11:25Z,OWNER,,"It's not possible to use a module that requires a nested import, such as `xml.etree.ElementTree`, at the moment. I found and fixed this bug in `git-history`, I should replicate that fix (and accompanying documentation) here: https://github.com/simonw/git-history/issues/39",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/351/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1073712378,I_kwDOBm6k_c4__4z6,1544,Code that detects the label column for a table is case-sensitive,9599,simonw,closed,0,,,,,2,2021-12-07T20:01:25Z,2021-12-07T20:03:43Z,2021-12-07T20:03:43Z,OWNER,,I just noticed that a column called `Name` is not being picked up as the label column for a table.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1544/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1077243232,I_kwDOCGYnMM5ANW1g,354,Test failure in test_rebuild_fts,9599,simonw,closed,0,,,,,7,2021-12-10T21:27:55Z,2021-12-11T01:08:46Z,2021-12-11T01:08:46Z,OWNER,,"Not sure why this has only just started failing, but I'm getting this: https://github.com/simonw/sqlite-utils/runs/4488687639 ``` E sqlite3.DatabaseError: database disk image is malformed sqlite_utils/db.py:425: DatabaseError _______________________ test_rebuild_fts[searchable_fts] _______________________ fresh_db = > table_to_fix = 'searchable_fts' @pytest.mark.parametrize(""table_to_fix"", [""searchable"", ""searchable_fts""]) def test_rebuild_fts(fresh_db, table_to_fix): table = fresh_db[""searchable""] table.insert(search_records[0]) table.enable_fts([""text"", ""country""]) # Run a search rows = list(table.search(""tanuki"")) assert len(rows) == 1 assert { ""rowid"": 1, ""text"": ""tanuki are running tricksters"", ""country"": ""Japan"", ""not_searchable"": ""foo"", }.items() <= rows[0].items() # Delete from searchable_fts_data fresh_db[""searchable_fts_data""].delete_where() # This should have broken the index with pytest.raises(sqlite3.DatabaseError): list(table.search(""tanuki"")) # Running rebuild_fts() should fix it > fresh_db[table_to_fix].rebuild_fts() ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/354/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1077322009,I_kwDOCGYnMM5ANqEZ,355,Allow users to pass a full convert() function definition,9599,simonw,closed,0,,,,,4,2021-12-10T23:59:58Z,2021-12-11T00:51:15Z,2021-12-11T00:49:31Z,OWNER,,"> I think the fix for this is to change the rules about what code is accepted in both the `-` mode and the literal code string mode: you can pass in a Python expression, OR a fragment that gets turned into a function, OR code that implements its own `def convert(value)` function. So this would work too: > ```sh > sqlite-utils convert my.db mytable col1 ' > def convert(value): > return value.upper() > ' > ``` _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991381679_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/355/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1077431957,I_kwDOCGYnMM5AOE6V,356,`sqlite-utils insert --convert` option,9599,simonw,closed,0,,,,,11,2021-12-11T07:24:48Z,2022-01-06T06:30:13Z,2022-01-06T06:28:53Z,OWNER,,"Idea come to me while re-reading this: https://simonwillison.net/2021/Aug/6/sqlite-utils-convert/ This is a bit of a hack: ``` cat /tmp/log.txt | \ jq --raw-input '{line: .}' --compact-output | \ sqlite-utils insert /tmp/logs.db log - --nl ``` Would be great if you could pipe lines to `insert` and transform them on the way in. A `--convert python-code` option, modeled after `sqlite-utils convert`, could do this.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/356/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1079129258,PR_kwDOBm6k_c4vynly,1554,TableView refactor,9599,simonw,closed,0,,,,,3,2021-12-13T23:16:04Z,2021-12-20T23:52:11Z,2021-12-20T23:52:04Z,OWNER,simonw/datasette/pulls/1554,"I'm starting a PR with almost nothing in it so I can use the GitHub code commenting feature to add a bunch of comments to the code I intend to refactor. Related issues: - #617 - #715 - #870 - #1518",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1554/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1082743068,PR_kwDOBm6k_c4v-izc,1559,"filters_from_request plugin hook, now used in TableView",9599,simonw,closed,0,,,,,6,2021-12-16T23:59:33Z,2021-12-17T23:09:41Z,2021-12-17T19:02:15Z,OWNER,simonw/datasette/pulls/1559,"New plugin hook, refs #473 Used it to extract the logic from TableView that handles _search and _through and _where - refs #1518",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1559/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1084257842,I_kwDOBm6k_c5AoHYy,1575,__call__() got an unexpected keyword argument 'specname',9599,simonw,closed,0,,,,,1,2021-12-20T01:24:04Z,2021-12-20T01:48:03Z,2021-12-20T01:47:57Z,OWNER,,"> I've installed the alpha version but get an error when starting up Datasette: ``` Traceback (most recent call last): File ""/Users/tim/.pyenv/versions/stock-exchange/bin/datasette"", line 5, in from datasette.cli import cli File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py"", line 15, in from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py"", line 31, in from .views.database import DatabaseDownload, DatabaseView File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py"", line 25, in from datasette.plugins import pm File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py"", line 29, in mod = importlib.import_module(plugin) File ""/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py"", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py"", line 9, in @hookimpl(specname=""filters_from_request"") TypeError: __call__() got an unexpected keyword argument 'specname' ``` _Originally posted by @wragge in https://github.com/simonw/datasette/issues/1547#issuecomment-997511968_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1575/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1094890366,PR_kwDOCGYnMM4wlm3B,361,--lines and --text and --convert and --import,9599,simonw,closed,0,,,,,15,2022-01-06T01:49:44Z,2022-01-06T06:37:03Z,2022-01-06T06:24:54Z,OWNER,simonw/sqlite-utils/pulls/361,"Refs #356 Still TODO: - [x] Get `--lines` working, with tests - [x] Get `--text` working, with tests - [x] Get regular JSON import working with `--convert` with tests - [x] Get `--lines` working with `--convert` with tests - [x] Get `--text` working with `--convert` with tests - [x] Get `--csv` and `--tsv` import working with `--convert` with tests - [x] Get `--nl` working with `--convert` with tests - [x] Documentation for all of the above",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/361/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1094974713,I_kwDOCGYnMM5BQ_z5,362,upsert --detect-types is broken,9599,simonw,closed,0,,,,,0,2022-01-06T05:12:10Z,2022-01-06T06:54:45Z,2022-01-06T06:28:34Z,OWNER,,"Noticed this thanks to syntax highlighting in VS Code showing an unused variable - need to fix it and add a test. ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/362/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1094981339,I_kwDOCGYnMM5BRBbb,363,Better error message if `--convert` code fails to return a dict,9599,simonw,closed,0,,,,,4,2022-01-06T05:26:28Z,2022-02-03T22:52:30Z,2022-02-03T22:51:30Z,OWNER,,"Here's the traceback if your `--convert` function doesn't return a dict right now: ``` % sqlite-utils insert /tmp/all.db blah /tmp/log.log --convert 'all.upper()' --all Traceback (most recent call last): File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils"", line 33, in sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')()) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__ return self.main(*args, **kwargs) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1062, in main rv = self.invoke(ctx) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 763, in invoke return __callback(*args, **kwargs) File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 949, in insert insert_upsert_implementation( File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 834, in insert_upsert_implementation db[table].insert_all( File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py"", line 2602, in insert_all first_record = next(records) File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py"", line 3044, in fix_square_braces for record in records: File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 831, in docs = (decode_base64_values(doc) for doc in docs) File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/utils.py"", line 86, in decode_base64_values to_fix = [ File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/utils.py"", line 89, in if isinstance(doc[k], dict) TypeError: string indices must be integers ``` It would be nicer if that returned a more useful error message. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/361#issuecomment-1006295276_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/363/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1097040427,I_kwDOBm6k_c5BY4Ir,1587,Add `sqlite_stat1`(-4) tables to hidden table list,9599,simonw,closed,0,,,,,2,2022-01-08T21:28:20Z,2022-01-20T04:12:59Z,2022-01-20T04:12:59Z,OWNER,,"> Running `ANALYZE` creates a new visible table called `sqlite_stat1`: https://www.sqlite.org/fileformat.html#the_sqlite_stat1_table > > This should be added to the default list of hidden tables in Datasette.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1587/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1097091527,I_kwDOCGYnMM5BZEnH,369,Research how much of a difference analyze / sqlite_stat1 makes,9599,simonw,closed,0,,,,,11,2022-01-09T03:03:36Z,2022-02-03T21:07:41Z,2022-02-03T21:07:35Z,OWNER,,"> Is there a downside to having a `sqlite_stat1` table if it has wildly incorrect statistics in it? _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008163050_ More generally: how much of a difference does the `sqlite_stat1` table created by `ANALYZE` make to queries? I'm particularly interested in `group by` / `count *` queries since Datasette uses those for faceting.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/369/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1099584685,I_kwDOCGYnMM5BilSt,381,`sqlite-utils rows` options `--limit` and `--offset`,9599,simonw,closed,0,,,,,2,2022-01-11T20:23:12Z,2022-01-11T23:33:37Z,2022-01-11T23:19:36Z,OWNER,,Because I often want to use it just to preview a few rows from the database. Piping through `| head -n 20` works for JSON and CSV (they stream) but not for `--table`.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/381/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1099585611,I_kwDOCGYnMM5BilhL,382,`--where` option for `sqlite-rows`,9599,simonw,closed,0,,,,,1,2022-01-11T20:24:23Z,2022-01-11T23:33:14Z,2022-01-11T23:32:47Z,OWNER,,CLI equivalent of `table.rows_where()` - should accept parameters too. Work on this at the same time as #381.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/382/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1099586786,I_kwDOCGYnMM5Bilzi,383,Add documentation page with the output of `--help`,9599,simonw,closed,0,,,,,4,2022-01-11T20:25:58Z,2022-01-11T22:55:05Z,2022-01-11T21:44:05Z,OWNER,,"Can be maintained using `cog` from #373. Similar in purpose to the API reference page, but this is for the CLI.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/383/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1099897648,I_kwDOCGYnMM5Bjxsw,384,Add examples to every `--help`,9599,simonw,closed,0,,,,,0,2022-01-12T05:31:25Z,2022-01-26T03:15:02Z,2022-01-26T03:15:02Z,OWNER,,Everything on https://sqlite-utils.datasette.io/en/stable/cli-reference.html would benefit from an example.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/384/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1102612922,I_kwDOBm6k_c5BuIm6,1597,"""datasette inspect"" has no help summary",9599,simonw,closed,0,,,,,1,2022-01-14T00:02:16Z,2022-01-14T00:07:36Z,2022-01-14T00:07:36Z,OWNER,,"Made obvious by the new CLI reference page added in #1594. https://docs.datasette.io/en/latest/cli-reference.html#datasette-inspect-help ``` Commands: serve* Serve up specified SQLite database files with a web UI inspect install Install Python packages - e.g. ``` ``` Usage: datasette inspect [OPTIONS] [FILES]... Options: --inspect-file TEXT --load-extension TEXT Path to a SQLite extension to load --help Show this message and exit. ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1597/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1102637351,I_kwDOBm6k_c5BuOkn,1598,Replace update-docs-help.py script with cog,9599,simonw,closed,0,,,,,1,2022-01-14T00:33:27Z,2022-01-14T00:47:57Z,2022-01-14T00:47:57Z,OWNER,,"I introduced `cog` in #1594 - I can use this to replace the older `update-docs-help.py` mechanism: https://github.com/simonw/datasette/blob/76d66d5b2bf10249c0beaac0999b93ac8d757f48/tests/test_docs.py#L36-L53",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1598/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1104691662,I_kwDOBm6k_c5B2EHO,1600,plugins --all example should use cog,9599,simonw,closed,0,,,,,1,2022-01-15T11:47:49Z,2022-01-20T05:06:21Z,2022-01-20T05:04:16Z,OWNER,,The example output for `datasette plugins --all`on this page has got out of date: https://docs.datasette.io/en/stable/plugins.html#seeing-what-plugins-are-installed,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1600/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1107557831,I_kwDOCGYnMM5CA_3H,386,"Better ""contributing"" documentation",9599,simonw,closed,0,,,,,0,2022-01-19T02:11:48Z,2022-01-19T02:15:21Z,2022-01-19T02:15:21Z,OWNER,,"This page jumps straight into running the tests: https://sqlite-utils.datasette.io/en/latest/contributing.html It should add a little more about expected collaboration styles - opening an issue before filing a pull request - and probably link to https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/386/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1108846067,I_kwDOBm6k_c5CF6Xz,1606,Tests failing against Python 3.6,9599,simonw,closed,0,,,,,3,2022-01-20T04:22:44Z,2022-01-20T04:36:42Z,2022-01-20T04:36:42Z,OWNER,,"https://github.com/simonw/datasette/runs/4877484366 ``` E File ""/opt/hostedtoolcache/Python/3.6.15/x64/lib/python3.6/site-packages/uvicorn/server.py"", line 67, in run E return asyncio.run(self.serve(sockets=sockets)) E AttributeError: module 'asyncio' has no attribute 'run' ``` I think this may mean `uvicorn` has dropped support for Python 3.6.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1109808154,I_kwDOBm6k_c5CJlQa,1608,Documentation should clarify /stable/ vs /latest/,9599,simonw,closed,0,,,,,15,2022-01-20T22:02:59Z,2023-03-26T23:41:12Z,2022-01-20T22:53:17Z,OWNER,,"It's not currently clear what the difference between https://docs.datasette.io/en/latest/ and https://docs.datasette.io/en/stable/ is - I should fix that. On Twitter: https://twitter.com/simonw/status/1484285006243528705",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1608/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1109884720,I_kwDOBm6k_c5CJ38w,1609,"Ensure ""pip install datasette"" still works with Python 3.6",9599,simonw,closed,0,,,,,12,2022-01-21T00:08:10Z,2022-01-24T19:20:09Z,2022-01-21T02:24:13Z,OWNER,,"## Original title: Can I keep ""pip install datasette"" working on Python 3.6? I dropped support for 3.6 in: - #1577 I'm getting reports that `pip3 install datasette` throws an error on that Python, even though I haven't made that new release yet - presumably due to lack of pinning of Uvicorn: https://twitter.com/ldodds/status/1484289475195080706 Is it possible to get `pip` on that version of Python to install the highest possible version of the packages that are still known to support Python 3.6? If so, how?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1609/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1111293050,I_kwDOCGYnMM5CPPx6,387,Python library docs should start with a self contained example,9599,simonw,closed,0,,,,,1,2022-01-22T06:23:56Z,2022-01-26T01:37:17Z,2022-01-26T01:35:30Z,OWNER,,You have to read a lot of stuff in a lot of different places to get started with the Python library. Add a getting started introduction to https://sqlite-utils.datasette.io/en/stable/python-api.html,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/387/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1114543475,I_kwDOCGYnMM5CbpVz,388,Link to stable docs from older versions,9599,simonw,closed,0,,,,,7,2022-01-26T01:55:46Z,2023-03-26T23:43:12Z,2022-01-26T02:00:22Z,OWNER,,"https://sqlite-utils.datasette.io/en/2.14.1/ isn't showing a link to the stable release right now. I should also apply the same fix I used for Datasette in: - https://github.com/simonw/datasette/issues/1608 TIL: https://til.simonwillison.net/readthedocs/link-from-latest-to-stable",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/388/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1114544727,I_kwDOCGYnMM5CbppX,389,Plausible analytics for documentation,9599,simonw,closed,0,,,,,2,2022-01-26T01:58:35Z,2022-01-26T02:07:41Z,2022-01-26T02:07:41Z,OWNER,,"```html ``` _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/388#issuecomment-1021785268_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/389/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1114557284,I_kwDOCGYnMM5Cbstk,390,`sqlite-utils upsert` should require `--pk` more elegantly,9599,simonw,closed,0,,,,,1,2022-01-26T02:20:31Z,2022-01-26T03:20:25Z,2022-01-26T03:19:43Z,OWNER,,"Currently throws an ugly traceback: ``` % echo '[ {""id"": 1, ""name"": ""Lila""}, {""id"": 1, ""name"": ""Lila""} ]' | sqlite-utils upsert data.db chickens - Traceback (most recent call last): File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils"", line 33, in sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')()) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__ return self.main(*args, **kwargs) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1062, in main rv = self.invoke(ctx) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File ""/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py"", line 763, in invoke return __callback(*args, **kwargs) File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 1104, in upsert insert_upsert_implementation( File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py"", line 906, in insert_upsert_implementation db[table].insert_all( File ""/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py"", line 2615, in insert_all raise PrimaryKeyRequired(""upsert() requires a pk"") sqlite_utils.db.PrimaryKeyRequired: upsert() requires a pk ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/390/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1114638930,I_kwDOCGYnMM5CcApS,391,`sqlite-utils bulk` progress bar,9599,simonw,closed,0,,,,,2,2022-01-26T05:14:49Z,2022-01-26T05:17:20Z,2022-01-26T05:16:51Z,OWNER,,"It can easily have a progress bar because it works by looping through an iterator: https://github.com/simonw/sqlite-utils/blob/a9fca7efa4184fbb2a65ca1275c326950ed9d3c1/sqlite_utils/cli.py#L1014-L1018 Should also support the `--silent` option if I add this.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/391/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1114640101,I_kwDOCGYnMM5CcA7l,392,`sqlite-utils bulk --batch-size` option,9599,simonw,closed,0,,,,,4,2022-01-26T05:17:11Z,2022-01-26T18:17:59Z,2022-01-26T18:17:59Z,OWNER,,"> Could add support for `--batch-size` as seen in `insert`/`upsert` too - causing it to break the list up into batches and commit for each one. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/391#issuecomment-1021876055_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/392/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1118585417,I_kwDOCGYnMM5CrEJJ,393,Better documentation for insert-replace,9599,simonw,closed,0,,,,,1,2022-01-30T15:40:23Z,2022-02-03T22:13:24Z,2022-02-03T22:13:24Z,OWNER,,"Currently: https://sqlite-utils.datasette.io/en/stable/python-api.html#insert-replacing-data > If you want to insert a record or replace an existing record with the same primary key, using the replace=True argument to .insert() or .insert_all(): Should describe the exception you get first, then how to use replace to avoid it.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/393/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1122414274,PR_kwDOBm6k_c4x_evE,1622,Test against Python 3.11-dev,9599,simonw,closed,0,,,,,1,2022-02-02T21:39:38Z,2022-02-02T21:58:53Z,2022-02-02T21:58:53Z,OWNER,simonw/datasette/pulls/1622,Refs #1621,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1622/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1123849278,I_kwDOCGYnMM5C_JQ-,395,"""apt-get: command not found"" error on macOS",9599,simonw,closed,0,,,,,1,2022-02-04T06:03:42Z,2022-02-04T06:10:58Z,2022-02-04T06:10:58Z,OWNER,,"Yeah, `apt-get` isn't a thing on macOS so 4a2a3e2fd0d5534f446b3f1fee34cb165e4d86d2 (to test #79 against real SpatiaLite) broke.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/395/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1123851690,I_kwDOCGYnMM5C_J2q,396,"mypy failure, sqlite_utils/utils.py:56",9599,simonw,closed,0,,,,,0,2022-02-04T06:08:09Z,2022-02-04T06:10:33Z,2022-02-04T06:10:33Z,OWNER,,"https://github.com/simonw/sqlite-utils/runs/5062725880?check_suite_focus=true > `sqlite_utils/utils.py:56: error: Incompatible return value type (got ""None"", expected ""str"")`",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/396/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1125077063,I_kwDOCGYnMM5DD1BH,400,`sqlite-utils create-table` ... `--if-not-exists`,9599,simonw,closed,0,,,,,1,2022-02-06T01:32:53Z,2022-02-06T01:34:53Z,2022-02-06T01:34:46Z,OWNER,,"Inspired by: - #397 To match the option on `create-index`: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-index ``` --if-not-exists Ignore if index already exists ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/400/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1125081640,I_kwDOCGYnMM5DD2Io,401,Update SpatiaLite example in the documentation,9599,simonw,closed,0,,,,,2,2022-02-06T02:02:07Z,2022-02-06T02:05:03Z,2022-02-06T02:03:24Z,OWNER,,"This one here: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions It should take advantage of the new methods from: - #79",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/401/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1128120451,I_kwDOCGYnMM5DPcCD,404,Add example of `--convert` to the help for `sqlite-utils insert`,9599,simonw,closed,0,,,,,2,2022-02-09T06:49:09Z,2022-02-09T06:56:35Z,2022-02-09T06:55:16Z,OWNER,,"https://sqlite-utils.datasette.io/en/3.23/cli-reference.html#insert would be more useful if it included an example of `--convert` in action. I can maybe use an example from https://simonwillison.net/2022/Jan/11/sqlite-utils/",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/404/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1128139375,I_kwDOCGYnMM5DPgpv,405,"`Database(memory_name=""name"")` constructor argument",9599,simonw,closed,0,,,,,2,2022-02-09T07:15:03Z,2022-02-16T01:23:16Z,2022-02-16T01:23:16Z,OWNER,,"SQLite in-memory databases can be named, in which case multiple connections can be opened to a shared in-memory database running within the same process. Datasette supports this - SQLite could support it too. https://docs.datasette.io/en/0.60.2/internals.html#database-ds-path-none-is-mutable-false-is-memory-false-memory-name-none",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/405/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1152072027,I_kwDOBm6k_c5Eqzlb,1642,Dependency issue with asgiref and uvicorn,9599,simonw,closed,0,,,,,1,2022-02-26T18:00:35Z,2022-03-05T01:11:27Z,2022-03-05T01:11:17Z,OWNER,,"``` ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts. We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default. datasette 0.60.2 requires asgiref<3.5.0,>=3.2.10, but you'll have asgiref 3.5.0 which is incompatible. ``` That's after I forced an upgrade of `uvicorn` due to this warning: ``` ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts. We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default. uvicorn 0.13.1 requires click==7.*, but you'll have click 8.0.4 which is incompatible. ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1642/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1160407071,I_kwDOBm6k_c5FKmgf,1647,Test failures with SQLite 3.37.0+ due to column affinity case,9599,simonw,closed,0,,,,,5,2022-03-05T17:37:46Z,2022-03-05T19:56:28Z,2022-03-05T19:47:04Z,OWNER,,"These three tests are failing on my local machine: ``` FAILED tests/test_internals_database.py::test_table_column_details[facetable-expected0] - AssertionError: assert [Column(cid=0, name='pk', type='INTEGER', no... FAILED tests/test_internals_database.py::test_table_column_details[sortable-expected1] - AssertionError: assert [Column(cid=0, name='pk1', type='varchar(30)'... FAILED tests/test_table_html.py::test_sort_links - AssertionError: assert [{'a_href': None,\n 'attrs': {'class': ['col-Link'],\n 'data-column': '... ``` I ran `pytest --lf -vv` and the output had things like this in it: ``` E - Column(cid=1, name='created', type='text', notnull=0, default_value=None, is_pk=0, hidden=0), E ? ^^^^ E + Column(cid=1, name='created', type='TEXT', notnull=0, default_value=None, is_pk=0, hidden=0), ... E {'a_href': '/fixtures/sortable?_sort=sortable_with_nulls_2', E 'attrs': {'class': ['col-sortable_with_nulls_2'], E 'data-column': 'sortable_with_nulls_2', E 'data-column-not-null': '0', E - 'data-column-type': 'real', E ? ^^^^ E + 'data-column-type': 'REAL', E ? ^^^^ ``` Something is causing column types to come back in uppercase where previously they were lowercase.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1647/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1160432941,PR_kwDOBm6k_c4z_p6S,1648,Use dash encoding for table names and row primary keys in URLs,9599,simonw,closed,0,,,,,7,2022-03-05T19:50:45Z,2022-03-07T15:38:30Z,2022-03-07T15:38:30Z,OWNER,simonw/datasette/pulls/1648,"Refs #1439. - [x] Build `dash_encode` / `dash_decode` functions - [x] Use dash encoding for row primary keys - [x] Use dash encoding for `?_next=` pagination tokens - [x] Use dash encoding for table names in URLs - [x] Use dash encoding for database name - ~~Implement redirects from previous `%` URLs that replace those with `-`~~ - separate issue: #1650",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1648/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1161969891,I_kwDOBm6k_c5FQkDj,1654,Adopt a code of conduct,9599,simonw,closed,0,,,,,5,2022-03-07T22:00:24Z,2022-03-07T22:19:35Z,2022-03-07T22:19:35Z,OWNER,,"This is long overdue, especially given the size of the project now.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1654/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1166587040,I_kwDOCGYnMM5FiLSg,413,Display autodoc type information more legibly,9599,simonw,closed,0,,,,,5,2022-03-11T15:58:20Z,2022-03-11T18:07:10Z,2022-03-11T18:07:10Z,OWNER,,"https://sqlite-utils.datasette.io/en/3.25/reference.html#sqlite_utils.db.Table.insert looks like this at the moment: ",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/413/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1166731361,I_kwDOCGYnMM5Fiuhh,414,I forgot to include the changelog in the 3.25.1 release,9599,simonw,closed,0,,,,,7,2022-03-11T18:32:36Z,2022-03-11T18:40:39Z,2022-03-11T18:40:39Z,OWNER,,"I pushed a release for https://github.com/simonw/sqlite-utils/releases/tag/3.25.1 but forgot to include the release notes in `docs/changelog.rst` This means https://sqlite-utils.datasette.io/en/stable/changelog.html isn't showing them.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/414/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1169895600,PR_kwDOBm6k_c40eW7C,1659,Tilde encoding,9599,simonw,closed,0,,,,,1,2022-03-15T16:19:07Z,2022-03-15T18:01:58Z,2022-03-15T18:01:57Z,OWNER,simonw/datasette/pulls/1659,Refs #1657,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1659/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1173017980,PR_kwDOBm6k_c40oRq-,1664,Remove hashed URL mode,9599,simonw,closed,0,,,,,7,2022-03-17T23:19:10Z,2022-03-19T00:12:04Z,2022-03-19T00:12:04Z,OWNER,simonw/datasette/pulls/1664,Refs #1661.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1664/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1174404647,I_kwDOBm6k_c5F__4n,1669,Release 0.61 alpha,9599,simonw,closed,0,,,,,2,2022-03-20T00:35:35Z,2022-03-20T01:24:36Z,2022-03-20T01:24:36Z,OWNER,,"> I'm going to release this as a 0.61 alpha so I can more easily depend on it from `datasette-hashed-urls`. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1669/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1174423568,I_kwDOBm6k_c5GAEgQ,1670,Ship Datasette 0.61,9599,simonw,closed,0,,,,,6,2022-03-20T02:47:54Z,2022-03-23T18:32:32Z,2022-03-23T18:32:03Z,OWNER,,"Let the alpha bake for a while, since #1668 is a big last-minute change. After shipping, release a new `datasette-hashed-urls` that depends on it, also this: - https://github.com/simonw/datasette-hashed-urls/issues/11",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1670/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1175648453,I_kwDOBm6k_c5GEvjF,1675,Extract out `check_permissions()` from `BaseView,9599,simonw,closed,0,,,,,7,2022-03-21T16:39:46Z,2022-03-21T17:14:31Z,2022-03-21T17:13:21Z,OWNER,,"> I'm going to refactor this stuff out and document it so it can be easily used by plugins: https://github.com/simonw/datasette/blob/4a4164b81191dec35e423486a208b05a9edc65e4/datasette/views/base.py#L69-L103 _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1074136176_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1675/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1178521513,I_kwDOBm6k_c5GPs-p,1682,SQL queries against databases with different routes are broken,9599,simonw,closed,0,,,,,1,2022-03-23T18:42:57Z,2022-03-23T18:48:16Z,2022-03-23T18:48:16Z,OWNER,,"500 error on https://datasette-hashed-urls-preview.vercel.app/fixtures-09f8f95?sql=select+*+from+facetable Here's the trace: ``` File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py"", line 54, in data return await QueryView(self.ds).data( File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py"", line 232, in data self.ds.get_database(database), sql File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/app.py"", line 401, in get_database return self.databases[name] KeyError: 'fixtures-aa7318b' ``` It looks like this is a Datasette bug, which is frustrating because I just shipped Datasette 0.61 five minutes ago! _Originally posted by @simonw in https://github.com/simonw/datasette-hashed-urls/issues/13#issuecomment-1076693667_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1682/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1179928510,I_kwDOBm6k_c5GVEe-,1683,allow_facet: False should be respected by column cog menu,9599,simonw,closed,0,,,,,0,2022-03-24T19:05:06Z,2022-03-24T19:16:36Z,2022-03-24T19:16:36Z,OWNER,,"The column cog menu currently shows ""Facet by this"" even if faceting is disabled for the Datasette instance.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1683/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1182065616,I_kwDOBm6k_c5GdOPQ,1689,datasette.add_message() documentation is incorrect,9599,simonw,closed,0,,,,,1,2022-03-26T20:49:42Z,2022-03-26T21:35:57Z,2022-03-26T20:51:21Z,OWNER,,"https://docs.datasette.io/en/0.61.1/internals.html#add-message-request-message-message-type-datasette-info says: `.add_message(request, message, message_type=datasette.INFO)` But in the code it's: https://github.com/simonw/datasette/blob/6b99e4a66ba0ed8fca8ee41ceb7206928b60d5d1/datasette/app.py#L582",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1689/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1182143895,I_kwDOBm6k_c5GdhWX,1691,Bug in pytest-httpx example,9599,simonw,closed,0,,,,,0,2022-03-26T22:45:30Z,2022-03-26T22:46:09Z,2022-03-26T22:46:09Z,OWNER,,"https://docs.datasette.io/en/0.61.1/testing_plugins.html#testing-outbound-http-calls-with-pytest-httpx says: ```python async def test_outbound_http_call(httpx_mock): httpx_mock.add_response( url='https://www.example.com/', data='Hello world', ) ``` That's wrong - `data=` should be `text=`. https://github.com/Colin-b/pytest_httpx/blob/v0.20.0/README.md#reply-with-custom-body",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1691/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1190828163,I_kwDOBm6k_c5G-piD,1698,Add a warning about bots and Cloud Run,9599,simonw,closed,0,,,,,1,2022-04-03T05:57:17Z,2022-04-03T06:10:24Z,2022-04-03T06:10:24Z,OWNER,,Recommend the https://github.com/simonw/datasette-block-robots plugin if you are going to run a large database in Cloud Run (one with a lot of rows).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1698/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1200649889,I_kwDOBm6k_c5HkHah,1710,Guide for plugin authors to upgrade their plugins for 1.0,9599,simonw,closed,0,,,,,1,2022-04-11T22:58:25Z,2022-04-11T23:04:01Z,2022-04-11T23:03:25Z,OWNER,,I'll also encourage testing against both Datasette 0.x and Datasette 1.0 using a GitHub Actions matrix.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1710/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1200866134,I_kwDOCGYnMM5Hk8NW,424,Better error message if you try to create a table with no columns,9599,simonw,closed,0,,,,,1,2022-04-12T02:43:20Z,2022-04-13T22:40:15Z,2022-04-13T22:40:10Z,OWNER,,"Seen here: - https://github.com/simonw/geojson-to-sqlite/issues/30 Attempting to create a table with no columns produced this confusing error: ``` File ""/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/geojson_to_sqlite/utils.py"", line 69, in import_features db[table].create(column_types, pk=pk) File ""/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py"", line 863, in create self.db.create_table( File ""/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py"", line 517, in create_table self.execute(sql) File ""/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py"", line 236, in execute return self.conn.execute(sql) sqlite3.OperationalError: near "")"": syntax error ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/424/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1202227104,I_kwDOBm6k_c5HqIeg,1712,"Make """" easier to read",9599,simonw,closed,0,,,,,3,2022-04-12T18:17:07Z,2022-04-12T19:12:22Z,2022-04-12T18:44:20Z,OWNER,,"`Binary: 2,427,344 bytes` would be nicer - even better, include a tooltip showing that size translated using this function: https://github.com/simonw/datasette/blob/138e4d9a53e3982137294ba383303c3a848cfca4/datasette/utils/__init__.py#L837-L846 ![CleanShot 2022-04-12 at 11 15 04@2x](https://user-images.githubusercontent.com/9599/163027324-b0b6092e-6e11-438b-8077-789025d0bb37.png) ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1712/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1203842656,I_kwDOCGYnMM5HwS5g,425,`sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher,9599,simonw,closed,0,,,,,5,2022-04-13T22:16:53Z,2023-04-15T20:14:58Z,2022-04-13T22:48:57Z,OWNER,,"Got this error while investigating: - #421 Even though I was using the `LD_PRELOAD` trick from https://til.simonwillison.net/sqlite/ld-preload to use a newer version of SQLite. _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1098531354_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1205687423,I_kwDOCGYnMM5H3VR_,426,CLI docs should link to Python docs and vice versa,9599,simonw,closed,0,9599,simonw,,,1,2022-04-15T16:05:15Z,2023-07-22T22:13:22Z,2023-07-22T22:13:22Z,OWNER,,"For every command/API method there should be a link to the equivalent in the other form factor. Maybe also link to the API and CLI reference pages too.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/426/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1212823665,I_kwDOBm6k_c5ISjhx,1715,Refactor TableView to use asyncinject,9599,simonw,closed,0,,,,,13,2022-04-22T21:43:39Z,2022-12-01T21:15:18Z,2022-04-28T22:26:56Z,OWNER,,"I've been working on a dependency injection mechanism in a separate library: - https://github.com/simonw/asyncinject I think it's ready to try out with Datasette to see if it's a pattern that will work here. I'm going to attempt to refactor `TableView` to use it. There are two overall goals here: - Use `asyncinject` to add parallel execution of some aspects of the table page - most notably I want to be able to execute the `count(*)` query, the `select ...` query, the various faceting queries and the facet suggestion queries in parallel - and measure if doing so is good for performance. - Use it to execute different output formats (possibly with some changes to the existing `register_output_renderer()` plugin hook). I want CSV and JSON to use the same mechanism that plugins use. Stretch goal is to get this working with streaming data too, see: - #1101",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1715/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1212838949,I_kwDOBm6k_c5ISnQl,1716,Configure git blame to ignore Black commit,9599,simonw,closed,0,,,,,1,2022-04-22T21:56:37Z,2022-04-22T22:02:19Z,2022-04-22T22:02:19Z,OWNER,,"GitHub can support this in blame views now too: https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1716/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1213683988,I_kwDOBm6k_c5IV1kU,1718,Code examples in the documentation should be formatted with Black,9599,simonw,closed,0,,,,,12,2022-04-24T15:22:50Z,2022-04-24T16:24:14Z,2022-04-24T16:18:03Z,OWNER,,"For example on this page: https://docs.datasette.io/en/stable/writing_plugins.html#packaging-a-plugin I wonder if there's an easy way for me to enforce this for Sphinx documentation?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1718/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1214859703,I_kwDOBm6k_c5IaUm3,1719,Refactor `RowView` and remove `RowTableShared`,9599,simonw,closed,0,,,,,3,2022-04-25T18:06:24Z,2022-12-01T21:15:19Z,2022-04-25T18:33:44Z,OWNER,,"> The `RowTableShared` class is making this a whole lot more complicated. > > I'm going to split the `RowView` view out into an entirely separate `views/row.py` module. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1715#issuecomment-1108875068_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1719/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1215174094,I_kwDOBm6k_c5IbhXO,1720,Design plugin hook for extras,9599,simonw,closed,0,,,,,14,2022-04-26T00:08:10Z,2022-12-01T21:15:19Z,2022-04-26T20:20:27Z,OWNER,,"Refs: - #262 - #1709 I realized that this is a really natural plugin hook - and if I design it as a hook I can implement Datasette's core extras as default plugins.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1720/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1216508080,I_kwDOBm6k_c5IgnCw,1723,Research running SQL in table view in parallel using `asyncio.gather()`,9599,simonw,closed,0,,,,,5,2022-04-26T21:42:48Z,2022-04-27T18:53:44Z,2022-04-26T22:19:09Z,OWNER,,"Spun off from: - #1715",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1723/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1216619276,I_kwDOBm6k_c5IhCMM,1724,?_trace=1 doesn't work on Global Power Plants demo,9599,simonw,closed,0,,,,,3,2022-04-27T00:15:02Z,2022-04-27T06:15:14Z,2022-04-27T00:18:30Z,OWNER,,"https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_trace=1 is not showing the trace JSON at the bottom of the page. Confirmed that `trace_debug` is `true` on https://global-power-plants.datasettes.com/-/settings Possibly related: - https://github.com/simonw/datasette-total-page-time/issues/1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1724/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1223177069,PR_kwDOCGYnMM43LrKB,429,Depend on click-default-group-wheel,9599,simonw,closed,0,,,,,2,2022-05-02T18:03:10Z,2022-05-02T18:52:42Z,2022-05-02T18:05:00Z,OWNER,simonw/sqlite-utils/pulls/429,"Trying to get this to work with Pyodide. Refs: https://github.com/simonw/click-default-group-wheel/issues/3",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/429/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1223234932,I_kwDOBm6k_c5I6RV0,1733,Get Datasette compatible with Pyodide,9599,simonw,closed,0,,,,,9,2022-05-02T19:01:58Z,2022-05-04T15:14:01Z,2022-05-02T20:15:27Z,OWNER,,"I've already got this working as a prototype. Here are the changes I had to make: - Replace the two dependencies that don't publish pure Python wheels to PyPI: `click-default-group` and `python-baseconv` - Get Datasette to work without threading - which it turns out is exclusively used for database connections - Make the `uvicorn` dependency optional (only needed when Datasette runs in the CLI) TODO: - [x] Switch to `click-default-group-wheel` - [x] https://github.com/simonw/datasette/issues/1734 - [x] Work around `uvicorn` import error - [x] https://github.com/simonw/datasette/issues/1735 - [x] #1737 Goal is to be able to do the following directly in https://pyodide.org/en/stable/console.html ```python import micropip await micropip.install(""datasette"") from datasette.app import Datasette ds = Datasette() await ds.client.get(""/.json"") ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1733/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1223241647,I_kwDOBm6k_c5I6S-v,1734,Remove python-baseconv dependency,9599,simonw,closed,0,,,,,3,2022-05-02T19:08:37Z,2022-05-02T23:25:49Z,2022-05-02T19:39:20Z,OWNER,,"> I was going to vendor `baseconv.py`, but then I reconsidered - what if there are plugins out there that expect `import baseconv` to work because they have depended on Datasette? > > I used https://cs.github.com/ and as far as I can tell there aren't any! > > So I'm going to remove that dependency and work out a smarter way to do this - probably by providing a utility function within Datasette itself. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115258737_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1734/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1223263540,I_kwDOBm6k_c5I6YU0,1735,Datasette setting to disable threading (for Pyodide),9599,simonw,closed,0,,,,,1,2022-05-02T19:31:08Z,2022-05-02T23:25:49Z,2022-05-02T20:13:52Z,OWNER,,"> I'm going to add a Datasette setting to disable threading entirely, designed for usage in this particular case. > > I thought about adding a new setting, then I noticed this: > > datasette mydatabase.db --setting num_sql_threads 10 > > I'm going to let users set that to `0` to disable threaded execution of SQL queries. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115278325_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1735/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1223312279,PR_kwDOBm6k_c43MIU0,1736,Clean up compatibility with Pyodide,9599,simonw,closed,0,,,,,0,2022-05-02T20:14:38Z,2022-05-02T20:15:28Z,2022-05-02T20:15:27Z,OWNER,simonw/datasette/pulls/1736,"Closes #1735, closes #1733",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1736/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1223459734,I_kwDOBm6k_c5I7IOW,1737,Automated test for Pyodide compatibility,9599,simonw,closed,0,,,,,4,2022-05-02T23:24:25Z,2022-05-02T23:40:50Z,2022-05-02T23:40:50Z,OWNER,,"Refs: - #1733 Need something in the test suite such that if Datasette breaks against Pyodide in the future we hear about it. I'm thinking this is an opportunity to use [shot-scraper javascript](https://github.com/simonw/shot-scraper#scraping-pages-using-javascript).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1737/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1223699280,I_kwDOBm6k_c5I8CtQ,1739,.db downloads should be served with an ETag,9599,simonw,closed,0,,,,,6,2022-05-03T05:11:21Z,2022-05-04T18:21:18Z,2022-05-03T14:59:51Z,OWNER,,"I noticed that my Pyodide Datasette prototype is downloading the same database file every single time rather than browser caching it: ![image](https://user-images.githubusercontent.com/9599/166407074-dee19587-0667-4424-9e88-d3b5b90fd819.png) ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1739/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1239008850,I_kwDOBm6k_c5J2cZS,1744,`--nolock` feature for opening locked databases,9599,simonw,closed,0,,,,,7,2022-05-17T18:25:16Z,2022-05-17T19:46:38Z,2022-05-17T19:40:30Z,OWNER,,"The getting started docs currently suggest you try this to browse your Chrome history: datasette ~/Library/Application\ Support/Google/Chrome/Default/History But if Chrome is running you will likely get this error: sqlite3.OperationalError: database is locked Turns out there's a workaround for this which I just spotted [on the SQLite forum](https://sqlite.org/forum/forumpost/86a67f6995): > You can do this using a [URI filename](https://sqlite.org/uri.html): > ``` > sqlite3 'file:places.sqlite?mode=ro&nolock=1' > ``` > That opens the file `places.sqlite` in read-only mode with locking disabled. This isn't safe, in that changes to the database made by other corrections are likely to cause this connection to return incorrect results or crash. Read-only mode should at least mean that you don't corrupt the database in the process.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1744/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1239080102,I_kwDOBm6k_c5J2tym,1745,Documentation on running cog,9599,simonw,closed,0,,,,,1,2022-05-17T19:41:06Z,2022-05-17T19:45:51Z,2022-05-17T19:43:45Z,OWNER,,Noticed that `cog -r docs/*.rst` isn't documented in https://docs.datasette.io/en/latest/contributing.html#editing-and-building-the-documentation,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1745/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1243498298,I_kwDOBm6k_c5KHkc6,1746,Switch documentation theme to Furo,9599,simonw,closed,0,,,,,21,2022-05-20T18:42:17Z,2022-05-20T21:28:29Z,2022-05-20T21:28:29Z,OWNER,,"https://github.com/pradyunsg/furo I just did this for `shot-scraper` and I really like it: https://shot-scraper.datasette.io/en/latest/ - https://github.com/simonw/shot-scraper/issues/77",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1746/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1243512344,I_kwDOBm6k_c5KHn4Y,1747,Add tutorials to the getting started guide,9599,simonw,closed,0,,,,,1,2022-05-20T19:01:52Z,2022-05-20T19:12:30Z,2022-05-20T19:05:34Z,OWNER,,On https://docs.datasette.io/en/stable/getting_started.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1747/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1243517592,I_kwDOBm6k_c5KHpKY,1748,Add copy buttons next to code examples in the documentation,9599,simonw,closed,0,,,,,2,2022-05-20T19:09:00Z,2022-05-20T19:15:00Z,2022-05-20T19:11:32Z,OWNER,,Similar to the ones in `datasette-copyable` which are implemented here: https://github.com/executablebooks/sphinx-copybutton/tree/f84c001a0507f8ec46779d0701b079a265564583,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1748/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1243704847,I_kwDOCGYnMM5KIW4P,435,Switch to Furo documentation theme,9599,simonw,closed,0,,,,,2,2022-05-20T21:46:39Z,2022-05-20T21:56:10Z,2022-05-20T21:54:43Z,OWNER,,"As seen in: - https://github.com/simonw/datasette/issues/1746 - https://github.com/simonw/shot-scraper/issues/77",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/435/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1243715381,I_kwDOCGYnMM5KIZc1,436,"Add ""copy to clipboard"" button to code examples in documentation",9599,simonw,closed,0,,,,,0,2022-05-20T21:53:23Z,2022-05-20T21:57:53Z,2022-05-20T21:57:53Z,OWNER,,"Follows: - #435 Imitates: - https://github.com/simonw/datasette/issues/1748 I'll use https://github.com/executablebooks/sphinx-copybutton - here's the Datasette commit: https://github.com/simonw/datasette/commit/1465fea4798599eccfe7e8f012bd8d9adfac3039",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/436/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1269886084,I_kwDOCGYnMM5LsOyE,442,`maximize_csv_field_size_limit()` utility function,9599,simonw,closed,0,,,,,2,2022-06-13T19:54:54Z,2022-06-14T21:55:15Z,2022-06-14T21:31:49Z,OWNER,,"This code here runs only if `cli.py` is imported: https://github.com/simonw/sqlite-utils/blob/7ddf5300886a32d6daf60cf1d71efe492b65c87e/sqlite_utils/cli.py#L50-L59 I found myself needing the same fix in another library: - https://github.com/simonw/datasette-socrata/issues/13 It should be a documented utility function.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/442/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1269998342,I_kwDOCGYnMM5LsqMG,443,Make `utils.rows_from_file()` a documented API,9599,simonw,closed,0,,,,,2,2022-06-13T21:53:24Z,2022-06-20T19:49:37Z,2022-06-14T20:12:46Z,OWNER,,"> `rows_from_file()` isn't part of the documented API but maybe it should be! _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1154385916_",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/443/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1277295119,I_kwDOCGYnMM5MIfoP,445,`sqlite_utils.utils.TypeTracker` should be a documented API,9599,simonw,closed,0,,,,,3,2022-06-20T19:08:28Z,2022-06-20T19:49:02Z,2022-06-20T19:46:58Z,OWNER,,"I've used it in a couple of external places now: - https://github.com/simonw/datasette-socrata/blob/32fb256a461bf0e790eca10bdc7dd9d96c20f7c4/datasette_socrata/__init__.py#L264-L280 - https://github.com/simonw/datasette-lite/blob/caa8eade10f0321c64f9f65c4561186f02d57c5b/webworker.js#L55-L64 Refs: - https://github.com/simonw/datasette-lite/issues/32",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/445/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1277328147,I_kwDOCGYnMM5MInsT,446,Use Just to automate running tests and linters locally,9599,simonw,closed,0,,,,,2,2022-06-20T19:51:09Z,2022-06-21T19:28:35Z,2022-06-20T19:54:50Z,OWNER,,I keep committing code that fails additional tests like `mypy` and `flake8` and `black`. Automate those using Just.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/446/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1278571700,I_kwDOCGYnMM5MNXS0,447,Incorrect syntax highlighting in docs CLI reference,9599,simonw,closed,0,,,,,3,2022-06-21T14:53:10Z,2022-06-21T18:48:47Z,2022-06-21T18:48:46Z,OWNER,,"https://sqlite-utils.datasette.io/en/stable/cli-reference.html#insert ![CE020DDA-27FB-49C3-9EA6-37457DC4C321](https://user-images.githubusercontent.com/9599/174830380-06530537-b870-41c0-a8af-03c7fa720c6f.jpeg) It looks like Python keywords are being incorrectly highlighted here.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/447/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1292060682,I_kwDOCGYnMM5NA0gK,450,Add --ignore option to more commands,9599,simonw,closed,0,,,,,9,2022-07-02T13:52:02Z,2022-07-15T22:39:09Z,2022-07-15T22:37:45Z,OWNER,,"As seen in https://sqlite-utils.datasette.io/en/stable/cli-reference.html#add-foreign-key Could make this TIL trick unnecessary: https://til.simonwillison.net/bash/ignore-errors",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/450/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1292370469,I_kwDOBm6k_c5NCAIl,1765,Document plugins providing new plugin hook-,9599,simonw,closed,0,,,,,1,2022-07-03T17:05:14Z,2023-08-31T23:08:24Z,2023-08-31T23:06:31Z,OWNER,,I've used this pattern twice now: https://til.simonwillison.net/datasette/register-new-plugin-hooks - in `datasette-graphql` and `datasette-low-disk-space-hook`. I should describe the pattern on https://docs.datasette.io/en/stable/writing_plugins.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1765/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1298531653,I_kwDOCGYnMM5NZgVF,451,Make sqlite_utils.utils.chunks a documented function,9599,simonw,closed,0,,,,,2,2022-07-08T06:01:04Z,2022-07-15T22:09:34Z,2022-07-15T21:59:33Z,OWNER,,I want to use it in another project: https://github.com/simonw/sqlite-utils/blob/8a9fe6498faf783a1fdeb1793e661ad194a05267/sqlite_utils/utils.py#L471-L474,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/451/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1306548397,I_kwDOCGYnMM5N4Fit,454,CLI command for duplicating tables,9599,simonw,closed,0,,,,,1,2022-07-15T21:31:27Z,2022-07-15T21:48:23Z,2022-07-15T21:45:51Z,OWNER,,"CLI equivalent of: - #449",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/454/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1308461063,I_kwDODFdgUs5N_YgH,74,500 error in github-to-sqlite demo,9599,simonw,closed,0,,,,,5,2022-07-18T19:39:32Z,2022-07-18T21:16:18Z,2022-07-18T21:14:22Z,MEMBER,,"https://github-to-sqlite.dogsheep.net/github/issue_comments throws a 500: > `cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)` https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/metrics?project=datasette-222320 suggests this started happening 3 days ago.",207052882,github-to-sqlite,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1326087800,PR_kwDOCGYnMM48hI-_,460,Cross-link CLI to Python docs,9599,simonw,closed,0,,,,,4,2022-08-02T16:18:28Z,2022-08-18T21:58:10Z,2022-08-18T21:58:07Z,OWNER,simonw/sqlite-utils/pulls/460,"Work in progress, partly to test the ReadTheDocs preview link action. Refs: - #426 ---- :books: Documentation preview :books:: https://readthedocs-preview--460.org.readthedocs.build/en/460/ ",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/460/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1326391841,PR_kwDOCGYnMM48iLGF,462,Discord badge,9599,simonw,closed,0,,,,,2,2022-08-02T20:56:04Z,2022-08-02T21:15:57Z,2022-08-02T21:15:52Z,OWNER,simonw/sqlite-utils/pulls/462,"Also testing fix for: - https://github.com/readthedocs/readthedocs-preview/issues/10 ---- :books: Documentation preview :books:: https://sqlite-utils--462.org.readthedocs.build/en/462/ ",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/462/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1338001039,I_kwDOCGYnMM5PwEaP,464,Link from documentation to source code,9599,simonw,closed,0,,,,,5,2022-08-13T16:19:57Z,2022-08-17T23:38:03Z,2022-08-17T23:38:03Z,OWNER,,Twitter conversation asking for ways to automate this here: https://twitter.com/simonw/status/1558260492015046656,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/464/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1342357149,PR_kwDOCGYnMM49Wsnq,465,beanbag-docutils>=2.0,9599,simonw,closed,0,,,,,2,2022-08-17T22:41:39Z,2022-08-17T23:38:07Z,2022-08-17T23:38:02Z,OWNER,simonw/sqlite-utils/pulls/465,Refs #464,140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/465/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1342374388,PR_kwDOCGYnMM49Wv9T,466,Use Read the Docs action v1 (#463),9599,simonw,closed,0,,,,,0,2022-08-17T23:11:50Z,2022-08-17T23:11:54Z,2022-08-17T23:11:54Z,OWNER,simonw/sqlite-utils/pulls/466,"Read the Docs repository was renamed from `readthedocs/readthedocs-preview` to `readthedocs/actions/`. Now, the `preview` action is under `readthedocs/actions/preview` and is tagged as `v1`",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/466/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1342430983,I_kwDOBm6k_c5QA98H,1786,Adjust height of textarea for no JS case,9599,simonw,closed,0,,,,,4,2022-08-18T01:15:15Z,2022-10-27T21:50:12Z,2022-08-18T16:06:09Z,OWNER,,"Datasette Lite: https://lite.datasette.io/?sql=https://gist.githubusercontent.com/simonw/1f8a91123ccefd8844187225b1832d7a/raw/5069075b86aa79358fbab3d4482d1d269077d632/recipes.sql#/data?sql=select+id%2C+name%2C+ingredients%2C+%28%0A++select+json_group_array%28value%29+from+json_each%28ingredients%29%0A++where+value+in+%28select+value+from+json_each%28%3Ap0%29%29%0A%29+as+matching_ingredients%0Afrom+recipes%0Awhere+json_array_length%28matching_ingredients%29+%3E+0%0Aorder+by+json_array_length%28matching_ingredients%29+desc&p0=%5B%22sugar%22%2C+%22cheese%22%5D ![46F8101E-8CE3-4F61-B200-F865E6B5DBCC](https://user-images.githubusercontent.com/9599/185270723-f55513b0-b561-434d-9d7c-4fe5be9756e0.jpeg) ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1786/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1343422749,I_kwDOBm6k_c5QEwEd,1787,"Move ""datasette --get"" from Getting Started to CLI Reference",9599,simonw,closed,0,,,,,5,2022-08-18T17:53:39Z,2022-08-18T21:57:09Z,2022-08-18T21:56:21Z,OWNER,,It really shouldn't be here: https://docs.datasette.io/en/0.62/getting_started.html#datasette-get,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1787/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1343732788,I_kwDOBm6k_c5QF7w0,1788,Make it more obvious that Datasette publish can publish multiple databases,9599,simonw,closed,0,,,,,0,2022-08-18T22:57:51Z,2022-08-18T23:06:16Z,2022-08-18T23:06:16Z,OWNER,,Feedback initially for `datasette-publish-fly` but it applies to the others too.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1788/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1348394901,PR_kwDOBm6k_c49qmC2,1792,Test `--load-extension` in GitHub Actions,9599,simonw,closed,0,,,,,3,2022-08-23T18:43:29Z,2022-08-24T00:11:46Z,2022-08-24T00:11:45Z,OWNER,simonw/datasette/pulls/1792,"Refs: - #1789 ---- :books: Documentation preview :books:: https://datasette--1792.org.readthedocs.build/en/1792/ ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1792/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1352953535,PR_kwDOCGYnMM4950Az,473,Support entrypoints for `--load-extension`,9599,simonw,closed,0,,,,,1,2022-08-27T05:53:59Z,2022-08-27T05:55:52Z,2022-08-27T05:55:47Z,OWNER,simonw/sqlite-utils/pulls/473,"Refs #470 ---- :books: Documentation preview :books:: https://sqlite-utils--473.org.readthedocs.build/en/473/ ",140912432,sqlite-utils,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/473/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1355148385,I_kwDOBm6k_c5Qxexh,1796,Research an upgrade to CodeMirror 6,9599,simonw,closed,0,,,,,4,2022-08-30T04:27:46Z,2023-07-03T04:58:21Z,2023-07-03T04:58:21Z,OWNER,,"There are still a bunch of bugs in CodeMirror 5 that affect various mobile browsers - see Datasette Discord report here: https://discord.com/channels/823971286308356157/823971286941302908/1013878624992108645 https://user-images.githubusercontent.com/9599/187349269-7b7c0c8c-3894-4810-82f0-de7c1eb940b3.mp4 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1796/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1362363685,I_kwDOBm6k_c5RNAUl,1800,Remove upper bound dependencies as a default policy,9599,simonw,closed,0,,,,,3,2022-09-05T18:23:45Z,2022-09-05T18:39:52Z,2022-09-05T18:35:41Z,OWNER,,"https://iscinumpy.dev/post/bound-version-constraints/ has convinced me not to use upper bound dependencies unless I'm certain they are needed. Relevant PR: - https://github.com/simonw/datasette/pull/1799 Also: https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L45-L46 https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L48-L49 https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L51-L55 https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L57-L59 https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L75-L78 https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L81-L82 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1800/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1362367821,PR_kwDOBm6k_c4-ZGW6,1801,"Don't use upper bound dependencies, refs #1800",9599,simonw,closed,0,,,,,1,2022-09-05T18:29:28Z,2022-09-05T18:35:41Z,2022-09-05T18:35:41Z,OWNER,simonw/datasette/pulls/1801,"See https://iscinumpy.dev/post/bound-version-constraints/ ---- :books: Documentation preview :books:: https://datasette--1801.org.readthedocs.build/en/1801/ ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1801/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1362402998,I_kwDOBm6k_c5RNJ62,1802,Tests reliably failing on Python 3.7,9599,simonw,closed,0,,,,,15,2022-09-05T19:21:16Z,2022-09-06T00:40:20Z,2022-09-06T00:40:20Z,OWNER,," https://github.com/simonw/datasette/runs/8194907739?check_suite_focus=true I thought this might be an intermittent failure but attempts to re-run the tests have not made it pass. End of that trace is: ``` /home/runner/work/datasette/datasette/datasette/app.py:234: in __init__ self._refresh_schemas_lock = asyncio.Lock() /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/locks.py:161: in __init__ self._loop = events.get_event_loop() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_event_loop(self): """"""Get the event loop for the current context. Returns an instance of EventLoop or raises an exception. """""" if (self._local._loop is None and not self._local._set_called and isinstance(threading.current_thread(), threading._MainThread)): self.set_event_loop(self.new_event_loop()) if self._local._loop is None: raise RuntimeError('There is no current event loop in thread %r.' > % threading.current_thread().name) E RuntimeError: There is no current event loop in thread 'MainThread'. ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1802/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1362567197,PR_kwDOBm6k_c4-ZxWD,1803,Workaround for test failure: RuntimeError: There is no current event loop,9599,simonw,closed,0,,,,,1,2022-09-06T00:31:06Z,2022-09-06T00:40:19Z,2022-09-06T00:40:19Z,OWNER,simonw/datasette/pulls/1803,"Closes #1802 ---- :books: Documentation preview :books:: https://datasette--1803.org.readthedocs.build/en/1803/ ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1803/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1363440999,I_kwDOBm6k_c5RRHVn,1804,Ability to set a custom facet_size per table,9599,simonw,closed,0,,,,,6,2022-09-06T15:11:40Z,2022-09-07T00:21:56Z,2022-09-06T18:06:53Z,OWNER,,"Suggestion from Discord: https://discord.com/channels/823971286308356157/823971286941302908/1016725586351247430 > Is it possible to limit the facet size per database or even per table? This is a really good idea, it could be done in `metadata.yml`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1804/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1363765916,I_kwDOCGYnMM5RSWqc,483,`sqlite-utils install` command,9599,simonw,closed,0,,,,,2,2022-09-06T20:13:55Z,2022-09-26T19:04:43Z,2022-09-26T18:57:15Z,OWNER,,"With the addition of `--functions` in: - #471 In addition to the existing `convert` command, there are now very good reasons to want to install additional packages into the same virtual environment as `sqlite-utils` itself, to allow them to be used with those features. This isn't easy if you installed the tool with `pipx` or `brew install sqlite-utils`. Datasette solved this problem with the `datasette install` command: - https://github.com/simonw/datasette/issues/925 `sqlite-utils` could benefit from the same idea.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/483/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1368030952,I_kwDOBm6k_c5Rin7o,1808,Database() constructor currently defaults is_mutable to False,9599,simonw,closed,0,,,,,5,2022-09-09T16:02:41Z,2022-09-09T16:37:57Z,2022-09-09T16:19:25Z,OWNER,,"This is surprising. It caused a bug in `datasette-upload-dbs` because I didn't expect it to do that. > I think this is an API design flaw in Datasette itself, but I can fix it here first. _Originally posted by @simonw in https://github.com/simonw/datasette-upload-dbs/issues/6#issuecomment-1242150394_ Code in question: https://github.com/simonw/datasette/blob/bf8d84af5422606597be893cedd375020cb2b369/datasette/database.py#L29-L32",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1808/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1373595927,I_kwDOBm6k_c5R32kX,1809,`prepare_jinja2_environment()` hook should take `datasette` argument,9599,simonw,closed,0,,,,,11,2022-09-14T21:15:46Z,2022-09-17T03:39:05Z,2022-09-17T03:38:33Z,OWNER,,"That plugin hook's current signature is: https://github.com/simonw/datasette/blob/610425460b519e9c16d386cb81aa081c9d730ef0/datasette/hookspecs.py#L28-L30 As a result in the first alpha release of `datasette-edit-templates` I had to include this horrific hack: https://github.com/simonw/datasette-edit-templates/blob/087f6a6cabc20020f2b0524f11aa3a7836320848/datasette_edit_templates/__init__.py#L72-L75 ```python @hookimpl def prepare_jinja2_environment(env): # TODO: This should ideally take datasette, but that's not an argument yet datasette = inspect.currentframe().f_back.f_back.f_back.f_back.f_locals[""self""] ```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1809/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1378640768,I_kwDOBm6k_c5SLGOA,1816,Validate settings.json on startup in configuration directory mode,9599,simonw,closed,0,,,,,2,2022-09-19T23:35:18Z,2022-09-20T01:15:48Z,2022-09-20T01:15:48Z,OWNER,,"> It might have been useful for Datasette to show an error when started against a `settings.json` file that contains an invalid setting though. _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1814#issuecomment-1251677554_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1816/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1386593843,I_kwDOCGYnMM5Spb4z,494,Document how to use Just,9599,simonw,closed,0,,,,,2,2022-09-26T19:25:12Z,2022-09-26T19:32:36Z,2022-09-26T19:26:39Z,OWNER,,"I'm using `just` a lot know, based on this file - I should add that to https://sqlite-utils.datasette.io/en/latest/contributing.html https://github.com/simonw/sqlite-utils/blob/afbd2b2cba45cccb305c3d4638d18db4dd3d4bbd/Justfile#L1-L24",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/494/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1386734383,I_kwDOBm6k_c5Sp-Mv,1821,Release Datasette 0.63a0,9599,simonw,closed,0,,,,,1,2022-09-26T21:15:27Z,2022-09-26T22:06:39Z,2022-09-26T22:06:39Z,OWNER,,"> - The [prepare_jinja2_environment(env, datasette)](https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-hook-prepare-jinja2-environment) plugin hook now accepts an optional `datasette` argument. Hook implementations can also now return an `async` function which will be awaited automatically. ([#1809](https://github.com/simonw/datasette/issues/1809)) > - `--load-extension` option now supports entrypoints. Thanks, Alex Garcia. ([#1789](https://github.com/simonw/datasette/pull/1789)) > - New tutorial: [Cleaning data with sqlite-utils and Datasette](https://datasette.io/tutorials/clean-data). > - Facet size can now be set per-table with the new `facet_size` table metadata option. ([#1804](https://github.com/simonw/datasette/issues/1804)) > - `truncate_cells_html` setting now also affects long URLs in columns. ([#1805](https://github.com/simonw/datasette/issues/1805)) > - `Database(is_mutable=)` now defaults to `True`. ([#1808](https://github.com/simonw/datasette/issues/1808)) > - Non-JavaScript textarea now increases height to fit the SQL query. ([#1786](https://github.com/simonw/datasette/issues/1786)) > - More detailed command descriptions on the [CLI reference](https://docs.datasette.io/en/latest/cli-reference.html#cli-reference) page. ([#1787](https://github.com/simonw/datasette/issues/1787)) > - Datasette no longer enforces upper bounds on its depenedencies. ([#1800](https://github.com/simonw/datasette/issues/1800)) > - Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ([#1794](https://github.com/simonw/datasette/pull/1794)) > - The `settings.json` file used in [Configuration directory mode](https://docs.datasette.io/en/latest/settings.html#config-dir) is now validated on startup. ([#1816](https://github.com/simonw/datasette/issues/1816))",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1821/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1393903845,I_kwDOBm6k_c5TFUjl,1828,word-wrap: anywhere resulting in weird display,9599,simonw,closed,0,,,,,2,2022-10-02T21:25:03Z,2022-10-02T23:01:17Z,2022-10-02T23:01:17Z,OWNER,,"e.g. on https://github-to-sqlite.dogsheep.net/github/commits This is from a change introduced here: https://github.com/simonw/datasette/commit/bf8d84af5422606597be893cedd375020cb2b369 in #1805 https://github.com/simonw/datasette/blob/bf8d84af5422606597be893cedd375020cb2b369/datasette/static/app.css#L447-L450",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1828/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1396948693,I_kwDOBm6k_c5TQ77V,1829,Table/database that is private due to inherited permissions does not show padlock,9599,simonw,closed,0,,,,,8,2022-10-04T23:14:16Z,2022-10-24T02:23:46Z,2022-10-24T02:11:37Z,OWNER,,"I noticed that a table page that is private because the database or instance is private, e.g. this one: Is not displaying the padlock icon that indicates the table is not visible to the public. Same issue for the database page too, which in this case is private due to `view-instance`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1829/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1397084281,I_kwDOBm6k_c5TRdB5,1831,If user can see table but NOT database/instance nav links should not display,9599,simonw,closed,0,,,,,10,2022-10-05T02:16:31Z,2022-10-13T21:52:04Z,2022-10-13T21:52:04Z,OWNER,,"Spotted this bug while building this plugin: - https://github.com/simonw/datasette-public This is a public table, but the two links in the nav go to forbidden pages: Those nav links shouldn't be shown at all.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1831/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1397193691,I_kwDOBm6k_c5TR3vb,1832,__bool__ method on Results,9599,simonw,closed,0,,,,,2,2022-10-05T04:18:12Z,2022-10-05T04:32:33Z,2022-10-05T04:32:33Z,OWNER,,"Wrote this code today: https://github.com/simonw/datasette-public/blob/1401bfae50e71c1dfd2bfb6954f2e86d5a7ab21b/datasette_public/__init__.py#L41 ```python results = await db.execute( ""select 1 from _public_tables where table_name = ?"", [table_name] ) if len(results): return True ``` Would be nice if I could use `if results` there instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1832/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1408561039,PR_kwDOBm6k_c5Axrpb,1842,check_visibility can now take multiple permissions into account,9599,simonw,closed,0,,,,,3,2022-10-14T00:06:04Z,2022-10-24T02:11:36Z,2022-10-24T02:11:36Z,OWNER,simonw/datasette/pulls/1842,"Refs #1829 - [x] Fix table page - [x] Fix database page - [x] Fix query page - [x] Fix row page - [x] Tests - [x] Updated documentation for `check_visibility` method, to cover the new `permissions=` keyword argument Also this fix is currently only applied on the table page - needs to be applied on database, row and query pages too. ---- :books: Documentation preview :books:: https://datasette--1842.org.readthedocs.build/en/1842/ ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1842/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0, 1409679008,I_kwDOBm6k_c5UBf6g,1844,Update screenshots in documentation to match latest designs,9599,simonw,closed,0,,,,,18,2022-10-14T18:01:18Z,2022-10-14T23:51:46Z,2022-10-14T19:57:17Z,OWNER,,"https://docs.datasette.io/en/0.62/full_text_search.html has this out-of-date screenshot: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1844/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1413610718,I_kwDOCGYnMM5UQfze,500,Turn --flatten into a documented utility function,9599,simonw,closed,0,,,,,4,2022-10-18T17:43:36Z,2022-10-18T18:02:10Z,2022-10-18T18:00:40Z,OWNER,,The `--flatten` implementation isn't currently available to Python code - people have to roll their own implementation. Feedback from a conversation at DjangoCon.,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/500/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1413641049,I_kwDOCGYnMM5UQnNZ,501,Tests failing due to updated tabulate library,9599,simonw,closed,0,,,,,4,2022-10-18T18:07:52Z,2022-10-18T18:23:40Z,2022-10-18T18:23:40Z,OWNER,,"Failure here: https://github.com/simonw/sqlite-utils/actions/runs/3275786702/jobs/5391063221 I figured out the problem: ```diff diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst index b88e38a..82b4b6c 100644 --- a/docs/cli-reference.rst +++ b/docs/cli-reference.rst @@ -112,11 +112,15 @@ See :ref:`cli_query`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, - simple, textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, + latex, latex_booktabs, latex_longtable, latex_raw, + mediawiki, mixed_grid, mixed_outline, moinmoin, + orgtbl, outline, pipe, plain, presto, pretty, + psql, rounded_grid, rounded_outline, rst, simple, + simple_grid, simple_outline, textile, tsv, + unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row @@ -176,11 +180,15 @@ See :ref:`cli_memory`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, - simple, textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, + latex, latex_booktabs, latex_longtable, latex_raw, + mediawiki, mixed_grid, mixed_outline, moinmoin, + orgtbl, outline, pipe, plain, presto, pretty, + psql, rounded_grid, rounded_outline, rst, simple, + simple_grid, simple_outline, textile, tsv, + unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings -r, --raw Raw output, first column of first row @@ -401,11 +409,14 @@ See :ref:`cli_search`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, simple, - textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, latex, + latex_booktabs, latex_longtable, latex_raw, mediawiki, + mixed_grid, mixed_outline, moinmoin, orgtbl, outline, + pipe, plain, presto, pretty, psql, rounded_grid, + rounded_outline, rst, simple, simple_grid, + simple_outline, textile, tsv, unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings --load-extension TEXT Path to SQLite extension, with optional :entrypoint @@ -651,11 +662,14 @@ See :ref:`cli_tables`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, simple, - textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, latex, + latex_booktabs, latex_longtable, latex_raw, mediawiki, + mixed_grid, mixed_outline, moinmoin, orgtbl, outline, + pipe, plain, presto, pretty, psql, rounded_grid, + rounded_outline, rst, simple, simple_grid, + simple_outline, textile, tsv, unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings --columns Include list of columns for each table @@ -689,11 +703,14 @@ See :ref:`cli_views`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, simple, - textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, latex, + latex_booktabs, latex_longtable, latex_raw, mediawiki, + mixed_grid, mixed_outline, moinmoin, orgtbl, outline, + pipe, plain, presto, pretty, psql, rounded_grid, + rounded_outline, rst, simple, simple_grid, + simple_outline, textile, tsv, unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings --columns Include list of columns for each view @@ -732,11 +749,15 @@ See :ref:`cli_rows`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, - simple, textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, + latex, latex_booktabs, latex_longtable, latex_raw, + mediawiki, mixed_grid, mixed_outline, moinmoin, + orgtbl, outline, pipe, plain, presto, pretty, + psql, rounded_grid, rounded_outline, rst, simple, + simple_grid, simple_outline, textile, tsv, + unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings --load-extension TEXT Path to SQLite extension, with optional @@ -768,11 +789,14 @@ See :ref:`cli_triggers`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, simple, - textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, latex, + latex_booktabs, latex_longtable, latex_raw, mediawiki, + mixed_grid, mixed_outline, moinmoin, orgtbl, outline, + pipe, plain, presto, pretty, psql, rounded_grid, + rounded_outline, rst, simple, simple_grid, + simple_outline, textile, tsv, unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings --load-extension TEXT Path to SQLite extension, with optional :entrypoint @@ -804,11 +828,14 @@ See :ref:`cli_indexes`. --tsv Output TSV --no-headers Omit CSV headers -t, --table Output as a formatted table - --fmt TEXT Table format - one of fancy_grid, fancy_outline, - github, grid, html, jira, latex, latex_booktabs, - latex_longtable, latex_raw, mediawiki, moinmoin, - orgtbl, pipe, plain, presto, pretty, psql, rst, simple, - textile, tsv, unsafehtml, youtrack + --fmt TEXT Table format - one of asciidoc, double_grid, + double_outline, fancy_grid, fancy_outline, github, + grid, heavy_grid, heavy_outline, html, jira, latex, + latex_booktabs, latex_longtable, latex_raw, mediawiki, + mixed_grid, mixed_outline, moinmoin, orgtbl, outline, + pipe, plain, presto, pretty, psql, rounded_grid, + rounded_outline, rst, simple, simple_grid, + simple_outline, textile, tsv, unsafehtml, youtrack --json-cols Detect JSON cols and output them as JSON, not escaped strings --load-extension TEXT Path to SQLite extension, with optional :entrypoint diff --git a/docs/cli.rst b/docs/cli.rst index 8bc4176..1d67e88 100644 --- a/docs/cli.rst +++ b/docs/cli.rst @@ -187,10 +187,15 @@ Available ``--fmt`` options are: cog.out(""\n"" + ""\n"".join('- ``{}``'.format(t) for t in tabulate.tabulate_formats) + ""\n\n"") .. ]]] +- ``asciidoc`` +- ``double_grid`` +- ``double_outline`` - ``fancy_grid`` - ``fancy_outline`` - ``github`` - ``grid`` +- ``heavy_grid`` +- ``heavy_outline`` - ``html`` - ``jira`` - ``latex`` @@ -198,15 +203,22 @@ Available ``--fmt`` options are: - ``latex_longtable`` - ``latex_raw`` - ``mediawiki`` +- ``mixed_grid`` +- ``mixed_outline`` - ``moinmoin`` - ``orgtbl`` +- ``outline`` - ``pipe`` - ``plain`` - ``presto`` - ``pretty`` - ``psql`` +- ``rounded_grid`` +- ``rounded_outline`` - ``rst`` - ``simple`` +- ``simple_grid`` +- ``simple_outline`` - ``textile`` - ``tsv`` - ``unsafehtml`` ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/501/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1420055377,I_kwDOBm6k_c5UpFNR,1847,Both _local_metadata and _metadata_local?,9599,simonw,closed,0,,,,,2,2022-10-24T01:43:08Z,2022-10-24T01:53:13Z,2022-10-24T01:53:13Z,OWNER,,"Spotted this in the debugger against the `datasette` object while running tests (`pytest -k test_permissions_cascade` to be exact): ``` (Pdb) [p for p in dir(self) if p.startswith('_') and '__' not in p] ['_actor', '_asset_urls', '_connected_databases', '_crumb_items', '_local_metadata', '_metadata', '_metadata_local', '_metadata_recursive_update', '_permission_checks', '_plugins', '_prepare_connection', '_refresh_schemas', '_refresh_schemas_lock', '_register_custom_units', '_register_renderers', '_root_token', '_routes', '_secret', '_settings', '_show_messages', '_startup_hook_calculation', '_startup_hook_fired', '_startup_invoked', '_threads', '_versions', '_write_messages_to_response'] ``` ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1847/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1420090659,I_kwDOBm6k_c5UpN0j,1848,Private database page should show padlock on every table,9599,simonw,closed,0,,,,,3,2022-10-24T02:28:38Z,2022-10-24T02:50:29Z,2022-10-24T02:42:34Z,OWNER,,"Following: - #1829 https://latest.datasette.io/_internal looks like this: But those queries and tables are private too, and should also show the padlock icon.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1848/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1420174670,I_kwDOBm6k_c5UpiVO,1849,NoneType' object has no attribute 'actor',9599,simonw,closed,0,,,,,5,2022-10-24T04:02:15Z,2022-10-26T21:13:40Z,2022-10-26T21:13:40Z,OWNER,,"``` File ""/usr/local/lib/python3.10/site-packages/datasette/templates/_crumbs.html"", line 3, in template {% set items=crumb_items(request=request, database=database, table=table) %} File ""jinja2/async_utils.py"", line 65, in auto_await return await t.cast(""t.Awaitable[V]"", value) File ""datasette/app.py"", line 638, in _crumb_items actor=request.actor, action=""view-instance"", default=True ``` From Sentry.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1849/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1422915587,I_kwDOBm6k_c5Uz_gD,1853,Upgrade Datasette Docker to Python 3.11,9599,simonw,closed,0,,,,,7,2022-10-25T18:44:31Z,2022-10-25T19:28:56Z,2022-10-25T19:05:16Z,OWNER,,"Related: - #1768 I think this base image looks right: [3.11.0-slim-bullseye](https://hub.docker.com/layers/library/python/3.11.0-slim-bullseye/images/sha256-244c0b0e6e7608a16f87382fc8a5ef3c330d042113a9a7b6fc15a95360181651?context=explore)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1853/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1422954582,I_kwDOCGYnMM5U0JBW,502,Fix tests for Python 3.11,9599,simonw,closed,0,,,,,1,2022-10-25T19:20:31Z,2022-10-25T19:23:47Z,2022-10-25T19:23:47Z,OWNER,,"The way errors are represented has changed: https://github.com/simonw/sqlite-utils/actions/runs/3323588047/jobs/5494127154 ``` _________________________ test_query_invalid_function __________________________ db_path = '/tmp/pytest-of-runner/pytest-0/test_query_invalid_function0/test.db' def test_query_invalid_function(db_path): result = CliRunner().invoke( cli.cli, [db_path, ""select bad()"", ""--functions"", ""def invalid_python""] ) assert result.exit_code == 1 > assert ( result.output.strip() == ""Error: Error in functions definition: invalid syntax (, line 1)"" ) E AssertionError: assert 'Error: Error...ing>, line 1)' == 'Error: Error...ing>, line 1)' E - Error: Error in functions definition: invalid syntax (, line 1) E ? ^^^^^^ ^^^^^^ E + Error: Error in functions definition: expected '(' (, line 1) E ? ^^^^^^^ ^^^ ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/502/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1422973111,I_kwDOBm6k_c5U0Ni3,1854,Flaky test: test_serve_localhost_http,9599,simonw,closed,0,,,,,3,2022-10-25T19:37:35Z,2022-10-25T19:53:02Z,2022-10-25T19:53:02Z,OWNER,,Failing on Python 3.10 at the moment: https://github.com/simonw/datasette/actions/runs/3323629947/jobs/5494340302,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1854/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1423000702,I_kwDOCGYnMM5U0UR-,503,test_recreate failing on Windows Python 3.11,9599,simonw,closed,0,,,,,10,2022-10-25T20:01:41Z,2022-10-25T20:47:34Z,2022-10-25T20:45:43Z,OWNER,,"https://github.com/simonw/sqlite-utils/actions/runs/3323672128/jobs/5494726927 Related: - #502 ``` FAILED tests/test_recreate.py::test_recreate[True-True] - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_recreate_True_True_0\\data.db' FAILED tests/test_recreate.py::test_recreate[False-True] - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_recreate_False_True_0\\data.db' ```",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/503/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1423069384,I_kwDOCGYnMM5U0lDI,504,"db.close() method, calling db.conn.close()",9599,simonw,closed,0,,,,,1,2022-10-25T20:50:50Z,2022-10-25T21:00:29Z,2022-10-25T20:57:47Z,OWNER,,"I ended up needing to use `db.conn.close()` to fix this issue: - #503 I think `.close()` should be a method on `Database` itself.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/504/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1423182778,I_kwDOCGYnMM5U1Au6,505,Release sqlite-utils 3.30,9599,simonw,closed,0,,,,,2,2022-10-25T22:20:05Z,2022-10-25T22:41:26Z,2022-10-25T22:41:16Z,OWNER,,https://github.com/simonw/sqlite-utils/compare/3.29...defa2974c6d3abc19be28d6b319649b8028dc966,140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/505/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1425682079,I_kwDOBm6k_c5U-i6f,1865,Stop syncing main to master,9599,simonw,closed,0,,,,,1,2022-10-27T13:55:38Z,2022-10-27T13:58:27Z,2022-10-27T13:56:13Z,OWNER,,"I think it's been long enough now that I can drop the code that syncs the main branch to master. I originally added this for people who might be using `datasette publish ... --branch master` - which might only have been me anyway!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1865/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1426253476,I_kwDOBm6k_c5VAuak,1869,Release 0.63,9599,simonw,closed,0,,,,,3,2022-10-27T20:53:01Z,2022-10-27T22:24:38Z,2022-10-27T22:11:33Z,OWNER,,"Most of the release notes are already written: - https://github.com/simonw/datasette/releases/tag/0.63a0 - https://github.com/simonw/datasette/releases/tag/0.63a1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1869/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1429029604,I_kwDOCGYnMM5VLULk,506,Make `cursor.rowcount` accessible (wontfix),9599,simonw,closed,0,,,,,3,2022-10-30T21:51:55Z,2022-11-01T17:37:47Z,2022-11-01T17:37:13Z,OWNER,,"In building this Datasette feature on top of `sqlite-utils` I thought it might be useful to expose the number of rows that had been affected by a bulk insert or update - the `cursor.rowcount`: - https://github.com/simonw/datasette/issues/1866 This isn't currently exposed by `sqlite-utils`.",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/506/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1431786951,I_kwDOBm6k_c5VV1XH,1876,SQL query should wrap on SQL interrupted screen,9599,simonw,closed,0,,,,,2,2022-11-01T17:14:01Z,2022-11-01T17:22:33Z,2022-11-01T17:22:33Z,OWNER,,"Just saw this: ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1876/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1432012302,I_kwDOBm6k_c5VWsYO,1877,Refactor and tidy up final write API code,9599,simonw,closed,0,,,,,1,2022-11-01T20:00:11Z,2022-11-29T19:44:16Z,2022-11-29T19:44:07Z,OWNER,,"- `views/table.py` has got a bit too big - I think the write classes should be pulled out into a separate module. - [x] There's duplicate logic for deciding if the table and database exist and checking permissions",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1877/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1434094365,I_kwDOBm6k_c5Veosd,1881,Tool for simulating permission checks against actors,9599,simonw,closed,0,,,,,9,2022-11-03T04:43:20Z,2022-12-09T01:38:21Z,2022-11-04T00:13:05Z,OWNER,,"In working on this issue: - #1855 I realized that if I'm going to make actors more complicated (the proposed `_r` key for additional restricted permissions) I really need an interactive tool for simulating these checks, similar to the https://latest.datasette.io/-/allow-debug tool.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1881/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1450952393,I_kwDOCGYnMM5We8bJ,512,mypy failures in CI,9599,simonw,closed,0,,,,,3,2022-11-16T06:22:48Z,2022-11-16T07:49:51Z,2022-11-16T07:49:50Z,OWNER,,"https://github.com/simonw/sqlite-utils/actions/runs/3472012235 failed on Python 3.11: Truncated output: ``` sqlite_utils/db.py:2467: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True sqlite_utils/db.py:2467: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase sqlite_utils/db.py:2530: error: Incompatible default for argument ""where"" (default has type ""None"", argument has type ""str"") [assignment] sqlite_utils/db.py:2530: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True sqlite_utils/db.py:2530: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase sqlite_utils/db.py:2658: error: Argument 1 to ""count_where"" of ""Queryable"" has incompatible type ""Optional[str]""; expected ""str"" [arg-type] Found 23 errors in 1 file (checked 51 source files) ``` Best look at https://github.com/hauntsaninja/no_implicit_optional",140912432,sqlite-utils,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/512/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed 1452457263,I_kwDOBm6k_c5Wkr0v,1897,Serve schema JSON to the SQL editor to enable autocomplete,9599,simonw,closed,0,,,,,9,2022-11-16T23:47:45Z,2022-11-18T05:33:20Z,2022-11-18T02:54:43Z,OWNER,,"See: - https://github.com/simonw/datasette/issues/1893#issuecomment-1317831555 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1897/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed