{"id": 268470572, "node_id": "MDU6SXNzdWUyNjg0NzA1NzI=", "number": 40, "title": "Implement command-line tool interface", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2857392, "label": "Ship first public release"}, "comments": 11, "created_at": "2017-10-25T16:47:15Z", "updated_at": "2017-11-11T07:27:33Z", "closed_at": "2017-11-11T07:27:33Z", "author_association": "OWNER", "pull_request": null, "body": "The first version needs to take one or more file names or URLs, then generate and deploy an app to Now. It will assume you already have the now command installed and configured.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/40/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 316444720, "node_id": "MDU6SXNzdWUzMTY0NDQ3MjA=", "number": 233, "title": "Option to expose expanded foreign keys in JSON/CSV", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 11, "created_at": "2018-04-21T00:18:25Z", "updated_at": "2018-06-16T22:26:21Z", "closed_at": "2018-06-16T22:20:14Z", "author_association": "OWNER", "pull_request": null, "body": "https://datasette-cluster-map-demo.datasettes.com/sf-trees-02c8ef1/Street_Tree_List?qCareAssistant=1\r\n\r\n![f36b87c0-478e-4d55-9a5f-ad37df0b47cb](https://user-images.githubusercontent.com/9599/39078411-bb3e4f88-44be-11e8-9d0c-d22324793c77.png)\r\n\r\nIt would be nice if the info bubbles there could expose more than just the IDs, and if the title showed the expanded name of the selected qCareAssistant.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/233/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 565518772, "node_id": "MDU6SXNzdWU1NjU1MTg3NzI=", "number": 673, "title": "Mechanism for checking if a SQLite database file is safe to open", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 11, "created_at": "2020-02-14T19:36:04Z", "updated_at": "2020-02-14T20:13:59Z", "closed_at": "2020-02-14T20:13:59Z", "author_association": "OWNER", "pull_request": null, "body": "Opening a SpatiaLite database file without SpatiaLite will result in errors later on. Same for database files which use custom extensions, like the Apple Photos database.\r\n\r\nI've figured out how to tell if a database is safe to open or not:\r\n```sql\r\nselect sql from sqlite_master where sql like 'CREATE VIRTUAL TABLE%';\r\n```\r\nThis returns the SQL definitions for virtual tables. The bit after `using` tells you what they need.\r\n\r\nRun this against a SpatiaLite database and you get the following:\r\n```sql\r\nCREATE VIRTUAL TABLE SpatialIndex USING VirtualSpatialIndex()\r\nCREATE VIRTUAL TABLE ElementaryGeometries USING VirtualElementary()\r\n```\r\nRun it against an Apple Photos `photos.db` file (found with `find ~/Library | grep photos.db`) and you get this (partial list):\r\n```sql\r\nCREATE VIRTUAL TABLE RidList_VirtualReader using RidList_VirtualReaderModule\r\nCREATE VIRTUAL TABLE Array_VirtualReader using Array_VirtualReaderModule\r\nCREATE VIRTUAL TABLE LiGlobals_VirtualBufferReader using VirtualBufferReaderModule\r\nCREATE VIRTUAL TABLE RKPlace_RTree using rtree (modelId,minLongitude,maxLongitude,minLatitude,maxLatitude)\r\n```\r\nFor a database with FTS4 you get:\r\n```sql\r\nCREATE VIRTUAL TABLE \"docs_fts\" USING FTS4 (\r\n [title], [content], content=\"docs\"\r\n)\r\n```\r\nFTS5:\r\n```sql\r\nCREATE VIRTUAL TABLE [FARA_All_Registrants_fts] USING FTS5 (\r\n [Name], [Address_1], [Address_2],\r\n content=[FARA_All_Registrants]\r\n )\r\n```\r\nSo I can use this to figure out all of the `using` pieces and then compare them to a list of known support ones.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/pull/672#issuecomment-586441484_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/673/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 590666760, "node_id": "MDU6SXNzdWU1OTA2NjY3NjA=", "number": 39, "title": "--since feature can be confused by retweets", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 11, "created_at": "2020-03-30T23:25:33Z", "updated_at": "2020-04-01T03:45:16Z", "closed_at": "2020-04-01T03:45:16Z", "author_association": "MEMBER", "pull_request": null, "body": "If you run `twitter-to-sqlite user-timeline ... --since` it's supposed to fetch Tweets those specific users tweeted since last time the command was run.\r\n\r\nIt does this by seeking out the max ID of their previous tweets:\r\n\r\nhttps://github.com/dogsheep/twitter-to-sqlite/blob/810cb2af5a175837204389fd7f4b5721f8b325ab/twitter_to_sqlite/cli.py#L305-L311\r\n\r\nBUT... this has a nasty flaw: if another account had retweeted one of their recent tweets the retweeted-tweet will have been loaded into the database - so we may treat that as the most recent since ID and miss a bunch of their tweets!", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 628025100, "node_id": "MDU6SXNzdWU2MjgwMjUxMDA=", "number": 785, "title": "Datasette secret mechanism - initially for signed cookies", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 11, "created_at": "2020-05-31T19:14:52Z", "updated_at": "2020-06-06T00:43:40Z", "closed_at": "2020-06-01T00:18:40Z", "author_association": "OWNER", "pull_request": null, "body": "See comment in https://github.com/simonw/datasette/issues/784#issuecomment-636514974\r\n\r\nDatasette needs to be able to set signed cookies - which means it needs a mechanism for safely handling a signing secret.\r\n\r\nSince Datasette is a long-running process the default behaviour here can be to create a random secret on startup. This means that if the server restarts any signed cookies will be invalidated.\r\n\r\nIf the user wants a persistent secret they'll have to generate it themselves - maybe by setting an environment variable?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/785/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 572896293, "node_id": "MDU6SXNzdWU1NzI4OTYyOTM=", "number": 687, "title": "Expand plugins documentation to multiple pages", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5533512, "label": "Datasette 0.45"}, "comments": 11, "created_at": "2020-02-28T17:26:21Z", "updated_at": "2020-06-22T03:55:20Z", "closed_at": "2020-06-22T03:53:54Z", "author_association": "OWNER", "pull_request": null, "body": "I think the plugins docs need to extend beyond a single page now. I want to add a whole section on writing tests for plugins, showing how `httpx` can be used as seen in https://github.com/simonw/datasette-atom/issues/3 and suchlike.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/687/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 632724154, "node_id": "MDU6SXNzdWU2MzI3MjQxNTQ=", "number": 805, "title": "Writable canned queries live demo on Glitch", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 11, "created_at": "2020-06-06T20:52:13Z", "updated_at": "2020-07-01T22:44:01Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Needs to run somewhere with a mutable disk drive, so not Cloud Run or Heroku or Vercel.\r\n\r\nI think I'll put it on Glitch.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/805/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 648637666, "node_id": "MDU6SXNzdWU2NDg2Mzc2NjY=", "number": 880, "title": "POST to /db/canned-query that returns JSON should be supported (for API clients)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5818042, "label": "Datasette 0.49"}, "comments": 11, "created_at": "2020-07-01T03:14:43Z", "updated_at": "2020-09-14T21:28:21Z", "closed_at": "2020-09-14T21:25:01Z", "author_association": "OWNER", "pull_request": null, "body": "Now that CSRF is solved for API requests (#835) it would be good to support API requests to the `.json` extension.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/880/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 702069429, "node_id": "MDU6SXNzdWU3MDIwNjk0Mjk=", "number": 967, "title": "Writable canned queries with magic parameters fail if POST body is empty", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 11, "created_at": "2020-09-15T16:14:43Z", "updated_at": "2020-09-15T20:13:10Z", "closed_at": "2020-09-15T20:13:10Z", "author_association": "OWNER", "pull_request": null, "body": "When I try to use the new `?_json=1` feature from #880 with magic parameters from #842 I get this error:\r\n\r\n> Incorrect number of bindings supplied. The current statement uses 1, and there are 0 supplied", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/967/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 663228985, "node_id": "MDU6SXNzdWU2NjMyMjg5ODU=", "number": 904, "title": "datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 11, "created_at": "2020-07-21T18:42:52Z", "updated_at": "2020-10-23T19:44:05Z", "closed_at": "2020-10-20T00:51:51Z", "author_association": "OWNER", "pull_request": null, "body": "I tried using this block of template in a plugin and got an error:\r\n```html\r\n{% block nav %}\r\n
\r\n home /\r\n {{ database }} /\r\n {{ table }}\r\n
\r\n {{ super() }}\r\n{% endblock %}\r\n```\r\nError: `'database_url' is undefined`\r\n\r\nThat's because `database_url` is only made available by the BaseView template here:\r\n\r\nhttps://github.com/simonw/datasette/blob/d6e03b04302a0852e7133dc030eab50177c37be7/datasette/views/base.py#L110-L125", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/904/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 573755726, "node_id": "MDU6SXNzdWU1NzM3NTU3MjY=", "number": 690, "title": "Mechanism for plugins to add action menu items for various things", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 11, "created_at": "2020-03-02T06:48:36Z", "updated_at": "2020-10-30T05:20:43Z", "closed_at": "2020-10-30T05:20:42Z", "author_association": "OWNER", "pull_request": null, "body": "Now that we have support for plugins that can write I'm seeing all sorts of places where a plugin might need to add UI to the table page.\r\n\r\nSome examples:\r\n\r\n- `datasette-configure-fts` needs to add a \"configure search for this table\" link\r\n- a plugin that lets you render or delete tables needs to add a link or button somewhere\r\n- existing plugins like `datasette-vega` and `datasette-cluster-map` already do this with JavaScript\r\n\r\nThe challenge here is that multiple plugins may want to do this, so simply overriding templates and populating names blocks doesn't entirely work as templates may override each other.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/690/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 644582921, "node_id": "MDU6SXNzdWU2NDQ1ODI5MjE=", "number": 865, "title": "base_url doesn't seem to work when adding criteria and clicking \"apply\"", "user": {"value": 6739646, "label": "tballison"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 6026070, "label": "0.51"}, "comments": 11, "created_at": "2020-06-24T12:39:57Z", "updated_at": "2020-11-12T23:49:24Z", "closed_at": "2020-10-20T05:22:59Z", "author_association": "NONE", "pull_request": null, "body": "Over on Apache Tika, we're using datasette to allow users to make sense of the metadata for our file regression testing corpus.\r\n\r\nThis could be user error in how I've set up the reverse proxy!\r\n\r\nI started datasette like so:\r\n`docker run -d -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/corpora-metadata.db --config sql_time_limit_ms:60000 --config base_url:/datasette/`\r\n\r\nI then reverse proxied like so:\r\n\r\n ProxyPreserveHost On\r\n ProxyPass /datasette http://x.y.z.q:xxxx\r\n ProxyPassReverse /datasette http://x.y.z.q:xxx\r\n\r\nRegular sql works perfectly:\r\nhttps://corpora.tika.apache.org/datasette/corpora-metadata?sql=select+mime_string%2C+count%281%29+as+cnt%0D%0Afrom+profiles+p%0D%0Ajoin+mimes+m+on+p.mime_id%3Dm.mime_id%0D%0Agroup+by+mime_string%0D%0Aorder+by+cnt+desc\r\n\r\n\r\nHowever, adding criteria and clicking 'Apply' \r\nhttps://corpora.tika.apache.org/datasette/corpora-metadata/tika_1_24_1_mimes?_sort=file&mime__exact=text%2Fplain\r\n\r\nbounces back to:\r\nhttps://corpora.tika.apache.org/corpora-metadata/tika_1_24_1_mimes?_sort=file&file__contains=bug&mime__exact=text%2Fplain", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/865/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 615474990, "node_id": "MDU6SXNzdWU2MTU0NzQ5OTA=", "number": 21, "title": "bpylist.archiver.CircularReference: archive has a cycle with uid(13)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 11, "created_at": "2020-05-10T20:58:06Z", "updated_at": "2020-12-19T07:44:49Z", "closed_at": "2020-05-10T21:57:13Z", "author_association": "MEMBER", "pull_request": null, "body": "```\r\n% python -i $(which photos-to-sqlite) apple-photos photos.db \r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py\", line 611, in place\r\n return self._place # pylint: disable=access-member-before-definition\r\nAttributeError: 'PhotoInfo' object has no attribute '_place'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/bin/photos-to-sqlite\", line 11, in