{"id": 582517965, "node_id": "MDU6SXNzdWU1ODI1MTc5NjU=", "number": 698, "title": "Ability for a canned query to write to the database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 26, "created_at": "2020-03-16T18:31:59Z", "updated_at": "2020-06-06T19:43:49Z", "closed_at": "2020-06-06T19:43:48Z", "author_association": "OWNER", "pull_request": null, "body": "Canned queries are currently read-only: https://datasette.readthedocs.io/en/0.38/sql_queries.html#canned-queries\r\n\r\nAdd a `\"write\": true` option to their definition in `metadata.json` which turns them into queries that are submitted via POST and send their queries to the write queue.\r\n\r\nThen they can be used as a really quick way to define a writable interface and JSON API!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/698/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 582526961, "node_id": "MDU6SXNzdWU1ODI1MjY5NjE=", "number": 699, "title": "Authentication (and permissions) as a core concept", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 40, "created_at": "2020-03-16T18:48:00Z", "updated_at": "2020-06-06T19:42:11Z", "closed_at": "2020-06-06T19:42:11Z", "author_association": "OWNER", "pull_request": null, "body": "Right now Datasette authentication is provided exclusively by plugins:\r\n\r\n* https://github.com/simonw/datasette-auth-github\r\n* https://github.com/simonw/datasette-auth-existing-cookies\r\n\r\nThis is an all-or-nothing approach: either your Datasette instance requires authentication at the top level or it does not.\r\n\r\nBut... as I build new plugins like https://github.com/simonw/datasette-configure-fts and https://github.com/simonw/datasette-edit-tables I increasingly have individual features which should be reserved for logged-in users while still wanting other parts of Datasette to be open to all.\r\n\r\nThis is too much for plugins to own independently of Datasette core. Datasette needs to ship a single \"user is authenticated\" concept (independent of how users actually sign in) so that different plugins can integrate with it.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/699/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 628087971, "node_id": "MDU6SXNzdWU2MjgwODc5NzE=", "number": 786, "title": "Documentation page describing Datasette's authentication system", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 2, "created_at": "2020-06-01T01:10:06Z", "updated_at": "2020-06-06T19:40:20Z", "closed_at": "2020-06-06T19:40:20Z", "author_association": "OWNER", "pull_request": null, "body": "_Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636562999_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/786/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 629524205, "node_id": "MDU6SXNzdWU2Mjk1MjQyMDU=", "number": 793, "title": "CSRF protection for /-/messages tool and writable canned queries", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 3, "created_at": "2020-06-02T21:22:21Z", "updated_at": "2020-06-06T00:43:41Z", "closed_at": "2020-06-05T19:05:59Z", "author_association": "OWNER", "pull_request": null, "body": "> The `/-/messages` debug tool will need CSRF protection or people will be able to add messages using a hidden form on another website.\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/790#issuecomment-637790860_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/793/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 628025100, "node_id": "MDU6SXNzdWU2MjgwMjUxMDA=", "number": 785, "title": "Datasette secret mechanism - initially for signed cookies", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 11, "created_at": "2020-05-31T19:14:52Z", "updated_at": "2020-06-06T00:43:40Z", "closed_at": "2020-06-01T00:18:40Z", "author_association": "OWNER", "pull_request": null, "body": "See comment in https://github.com/simonw/datasette/issues/784#issuecomment-636514974\r\n\r\nDatasette needs to be able to set signed cookies - which means it needs a mechanism for safely handling a signing secret.\r\n\r\nSince Datasette is a long-running process the default behaviour here can be to create a random secret on startup. This means that if the server restarts any signed cookies will be invalidated.\r\n\r\nIf the user wants a persistent secret they'll have to generate it themselves - maybe by setting an environment variable?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/785/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 628121234, "node_id": "MDU6SXNzdWU2MjgxMjEyMzQ=", "number": 788, "title": " /-/permissions debugging tool", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5512395, "label": "Datasette 0.44"}, "comments": 2, "created_at": "2020-06-01T03:13:47Z", "updated_at": "2020-06-06T00:43:40Z", "closed_at": "2020-06-01T05:01:01Z", "author_association": "OWNER", "pull_request": null, "body": "> Debugging tool idea: `/-/permissions` page which shows you the actor and lets you type in the strings for `action`, `resource_type` and `resource_identifier` - then shows you EVERY plugin hook that would have executed and what it would have said, plus when the chain would have terminated.\r\n>\r\n> Bonus: if you're logged in as the `root` user (or a user that matches some kind of permission check, maybe a check for `permissions_debug`) you get to see a rolling log of the last 30 permission checks and what the results were across the whole of Datasette. This should make figuring out permissions policies a whole lot easier.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636576603_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/788/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 632056825, "node_id": "MDU6SXNzdWU2MzIwNTY4MjU=", "number": 802, "title": "\"datasette plugins\" command is broken", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-06-05T23:33:01Z", "updated_at": "2020-06-05T23:46:43Z", "closed_at": "2020-06-05T23:46:43Z", "author_association": "OWNER", "pull_request": null, "body": "I broke it in https://github.com/simonw/datasette/commit/a7137dfe069e5fceca56f78631baebd4a6a19967 - and it turns out there was no test coverage so I didn't realize it was broken.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/802/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 631789422, "node_id": "MDU6SXNzdWU2MzE3ODk0MjI=", "number": 799, "title": "TestResponse needs to handle multiple set-cookie headers", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-06-05T17:39:52Z", "updated_at": "2020-06-05T18:34:10Z", "closed_at": "2020-06-05T18:34:10Z", "author_association": "OWNER", "pull_request": null, "body": "Seeing this test failure on #798:\r\n```\r\n_______________________ test_auth_token _______________________\r\napp_client = \r\n def test_auth_token(app_client):\r\n \"The /-/auth-token endpoint sets the correct cookie\"\r\n assert app_client.ds._root_token is not None\r\n path = \"/-/auth-token?token={}\".format(app_client.ds._root_token)\r\n response = app_client.get(path, allow_redirects=False,)\r\n assert 302 == response.status\r\n assert \"/\" == response.headers[\"Location\"]\r\n> assert {\"id\": \"root\"} == app_client.ds.unsign(response.cookies[\"ds_actor\"], \"actor\")\r\nE KeyError: 'ds_actor'\r\ndatasette/tests/test_auth.py:12: KeyError\r\n```\r\nIt looks like that's happening because the ASGI middleware is adding another set-cookie header - but those two set-cookie headers are combined into one when the TestResponse is constructed:\r\n\r\nhttps://github.com/simonw/datasette/blob/0c064c5fe220b7b3d8dcf85b02b4e60452c47232/tests/fixtures.py#L113-L127", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/799/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 570301333, "node_id": "MDU6SXNzdWU1NzAzMDEzMzM=", "number": 684, "title": "Add documentation on Database introspection methods to internals.rst", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 4, "created_at": "2020-02-25T04:20:24Z", "updated_at": "2020-06-04T18:56:15Z", "closed_at": "2020-05-30T18:40:39Z", "author_association": "OWNER", "pull_request": null, "body": "`internals.rst` will be landing as part of #683", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/684/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 275082158, "node_id": "MDU6SXNzdWUyNzUwODIxNTg=", "number": 119, "title": "Build an \"export this data to google sheets\" plugin", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2017-11-18T14:14:51Z", "updated_at": "2020-06-04T18:46:40Z", "closed_at": "2020-06-04T18:46:39Z", "author_association": "OWNER", "pull_request": null, "body": "Inspired by https://github.com/kren1/tosheets\r\n\r\nIt should be a plug-in because I'd like to keep all interactions with proprietary / non-open-source software encapsulated in plugins rather than shipped as part of core.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/119/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 629535669, "node_id": "MDU6SXNzdWU2Mjk1MzU2Njk=", "number": 794, "title": "Show hooks implemented by each plugin on /-/plugins", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2020-06-02T21:44:38Z", "updated_at": "2020-06-02T22:30:17Z", "closed_at": "2020-06-02T21:50:10Z", "author_association": "OWNER", "pull_request": null, "body": "e.g.\r\n```json\r\n {\r\n \"name\": \"qs_actor.py\",\r\n \"static\": false,\r\n \"templates\": false,\r\n \"version\": null,\r\n \"hooks\": [\r\n \"actor_from_request\"\r\n ]\r\n }\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/794/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 629459637, "node_id": "MDU6SXNzdWU2Mjk0NTk2Mzc=", "number": 792, "title": "Replace response.body.decode(\"utf8\") with response.text in tests", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-06-02T19:32:24Z", "updated_at": "2020-06-02T21:29:58Z", "closed_at": "2020-06-02T21:29:58Z", "author_association": "OWNER", "pull_request": null, "body": "Make use of the `response.text` property to clean up the tests a tiny bit:\r\n\r\nhttps://github.com/simonw/datasette/blob/57cf5139c552cb7feab9947daa949ca434cc0a66/tests/fixtures.py#L26-L38", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/792/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 628156527, "node_id": "MDU6SXNzdWU2MjgxNTY1Mjc=", "number": 789, "title": "Mechanism for enabling pluggy tracing", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-06-01T05:10:14Z", "updated_at": "2020-06-01T05:11:03Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Could be useful for debugging plugins: https://pluggy.readthedocs.io/en/latest/#call-tracing\r\n\r\nI tried this out by adding these two lines in `plugins.py`:\r\n```python\r\npm = pluggy.PluginManager(\"datasette\")\r\npm.add_hookspecs(hookspecs)\r\n# Added these:\r\npm.trace.root.setwriter(print)\r\npm.enable_tracing()\r\n```\r\nOutput looked something like this:\r\n```\r\nINFO: 127.0.0.1:52724 - \"GET /-/-/static/app.css HTTP/1.1\" 404 Not Found\r\n actor_from_request [hook]\r\n datasette: \r\n request: \r\n\r\n finish actor_from_request --> [] [hook]\r\n\r\n extra_body_script [hook]\r\n template: show_json.html\r\n database: None\r\n table: None\r\n view_name: json_data\r\n datasette: \r\n\r\n finish extra_body_script --> [] [hook]\r\n\r\n extra_template_vars [hook]\r\n template: show_json.html\r\n database: None\r\n table: None\r\n view_name: json_data\r\n request: \r\n datasette: \r\n\r\n finish extra_template_vars --> [] [hook]\r\n\r\n extra_css_urls [hook]\r\n template: show_json.html\r\n database: None\r\n table: None\r\n datasette: \r\n\r\n finish extra_css_urls --> [] [hook]\r\n\r\n extra_js_urls [hook]\r\n template: show_json.html\r\n database: None\r\n table: None\r\n datasette: \r\n\r\n finish extra_js_urls --> [] [hook]\r\n\r\nINFO: 127.0.0.1:52724 - \"GET /-/actor HTTP/1.1\" 200 OK\r\n actor_from_request [hook]\r\n datasette: \r\n request: \r\n\r\n finish actor_from_request --> [] [hook]\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/789/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 268453968, "node_id": "MDU6SXNzdWUyNjg0NTM5Njg=", "number": 37, "title": "Ability to serialize massive JSON without blocking event loop", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2017-10-25T15:58:03Z", "updated_at": "2020-05-30T17:29:20Z", "closed_at": "2020-05-30T17:29:20Z", "author_association": "OWNER", "pull_request": null, "body": "We run the risk of someone attempting a select statement that returns thousands of rows and hence takes several seconds just to JSON encode the response, effectively blocking the event loop and pausing all other traffic.\r\n\r\nThe Twisted community have a solution for this, can we adapt that in some way? http://as.ynchrono.us/2010/06/asynchronous-json_18.html?m=1", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/37/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 451513541, "node_id": "MDU6SXNzdWU0NTE1MTM1NDE=", "number": 498, "title": "Full text search of all tables at once?", "user": {"value": 7936571, "label": "chrismp"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 12, "created_at": "2019-06-03T14:24:43Z", "updated_at": "2020-05-30T17:26:02Z", "closed_at": "2020-05-30T17:26:02Z", "author_association": "NONE", "pull_request": null, "body": "Does datasette have a built-in way, in a browser, to do a full-text search of all columns, in all databases and tables, that have full-text search enabled? Is there a plugin that does this?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/498/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 374953006, "node_id": "MDU6SXNzdWUzNzQ5NTMwMDY=", "number": 369, "title": "Interface should show same JSON shape options for custom SQL queries", "user": {"value": 416374, "label": "gfrmin"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 2, "created_at": "2018-10-29T10:39:15Z", "updated_at": "2020-05-30T17:24:06Z", "closed_at": null, "author_association": "CONTRIBUTOR", "pull_request": null, "body": "At the moment the page returning a custom SQL query shows the JSON and CSV APIs, but not the multiple JSON shapes. However, adding the `_shape` parameter to the JSON API URL manually still works, so perhaps there should be consistency in the interface by having the same \"Advanced Export\" box for custom SQL queries.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/369/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 570309546, "node_id": "MDU6SXNzdWU1NzAzMDk1NDY=", "number": 685, "title": "Document (and reconsider design of) Database.execute() and Database.execute_against_connection_in_thread()", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 15, "created_at": "2020-02-25T04:49:44Z", "updated_at": "2020-05-30T13:20:50Z", "closed_at": "2020-05-08T17:42:18Z", "author_association": "OWNER", "pull_request": null, "body": "In #683 I started a new section of internals documentation covering the `Database` class: https://datasette.readthedocs.io/en/latest/internals.html#database-class\r\n\r\nI decided not to document `.execute()` and `.execute_against_connection_in_thread()` yet because I'm not 100% happy with their API design yet.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/685/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 585633142, "node_id": "MDU6SXNzdWU1ODU2MzMxNDI=", "number": 706, "title": "Documentation for the \"request\" object", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 6, "created_at": "2020-03-22T02:55:50Z", "updated_at": "2020-05-30T13:20:00Z", "closed_at": "2020-05-27T22:31:22Z", "author_association": "OWNER", "pull_request": null, "body": "Since that object is passed to the `extra_template_vars` hooks AND the classes registered by `register_facet_classes` it should be part of the documented interface on https://datasette.readthedocs.io/en/stable/internals.html\r\n\r\nI could also start passing it to the `register_output_renderer` callback.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/706/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 626078521, "node_id": "MDU6SXNzdWU2MjYwNzg1MjE=", "number": 774, "title": "Consolidate request.raw_args and request.args", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 3268330, "label": "Datasette 1.0"}, "comments": 8, "created_at": "2020-05-27T22:30:59Z", "updated_at": "2020-05-29T23:27:35Z", "closed_at": "2020-05-29T23:22:38Z", "author_association": "OWNER", "pull_request": null, "body": "`request.raw_args` is not documented, and I'd like to remove it entirely.\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/706#issuecomment-634975252_\r\n\r\nI use it in a few places in other projects though, so I'll have to fix those first: https://github.com/search?q=user%3Asimonw+raw_args&type=Code", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/774/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 345469355, "node_id": "MDU6SXNzdWUzNDU0NjkzNTU=", "number": 351, "title": "Automatically create a GitHub release linking to release notes for every tagged release", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2018-07-28T18:31:12Z", "updated_at": "2020-05-28T18:56:16Z", "closed_at": "2020-05-28T18:56:15Z", "author_association": "OWNER", "pull_request": null, "body": "Can use this API called from Travis: https://developer.github.com/v3/repos/releases/#create-a-release\r\n\r\nThe release it generates should look like this one: https://github.com/simonw/datasette/releases/tag/0.24", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/351/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 455965174, "node_id": "MDU6SXNzdWU0NTU5NjUxNzQ=", "number": 508, "title": "Ability to set default sort order for a table or view in metadata.json", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": {"value": 9599, "label": "simonw"}, "milestone": null, "comments": 1, "created_at": "2019-06-13T21:40:51Z", "updated_at": "2020-05-28T18:53:03Z", "closed_at": "2020-05-28T18:53:02Z", "author_association": "OWNER", "pull_request": null, "body": "It can go here in the documentation: https://datasette.readthedocs.io/en/stable/metadata.html#setting-which-columns-can-be-used-for-sorting\r\n\r\nAlso need to fix this sentence which is no longer true:\r\n\r\n> By default, database views in Datasette do not support sorting", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/508/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 626663119, "node_id": "MDU6SXNzdWU2MjY2NjMxMTk=", "number": 781, "title": "request.url and request.scheme should obey force_https_urls config setting", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-05-28T16:54:47Z", "updated_at": "2020-05-28T17:39:54Z", "closed_at": "2020-05-28T17:10:13Z", "author_association": "OWNER", "pull_request": null, "body": "I'm trying to get the https://www.niche-museums.com/browse/feed.atom feed to validate and I git this from https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom\r\n\r\n> This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.\r\n> \r\n> [line 6](https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom#l6), column 73: Self reference doesn't match document location [[help](https://validator.w3.org/feed/docs/warning/SelfDoesntMatchLocation.html \"more information about this error\")]\r\n> \r\n> \r\n\r\nI tried to fix this using `force_https_urls` ([commit](https://github.com/simonw/museums/commit/5dc8e2c717c59f9e949b65e47a59878e01f929e4)) but it didn't work - because that setting isn't respected by the Request class:\r\n\r\nhttps://github.com/simonw/datasette/blob/40885ef24e32d91502b6b8bbad1c7376f50f2830/datasette/utils/asgi.py#L15-L32", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/781/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 626582657, "node_id": "MDU6SXNzdWU2MjY1ODI2NTc=", "number": 779, "title": "Make human_description_en explicitly available to output renderers", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-05-28T14:59:54Z", "updated_at": "2020-05-28T14:59:54Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "`datasette-atom` uses this:\r\n\r\nhttps://github.com/simonw/datasette-atom/blob/df98a6c43a443224b6cd232f84703ec297ef046b/datasette_atom/__init__.py#L36-L37\r\n```python\r\n if data.get(\"human_description_en\"):\r\n title += \": \" + data[\"human_description_en\"]\r\n```\r\nIt's a nice way to generate a useful title for a filtered table.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/779/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 608058890, "node_id": "MDU6SXNzdWU2MDgwNTg4OTA=", "number": 744, "title": "link_or_copy_directory() error - Invalid cross-device link", "user": {"value": 30607, "label": "aborruso"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 28, "created_at": "2020-04-28T06:26:45Z", "updated_at": "2020-05-28T14:32:53Z", "closed_at": "2020-05-27T06:01:28Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nwhen I run\r\n\r\n```\r\ndatasette publish heroku -n myapp --template-dir ./template mydb.db\r\n```\r\n\r\nI have this error\r\n\r\n```\r\nTraceback (most recent call last):\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py\", line 607, in link_or_copy_directory\r\n shutil.copytree(src, dst, copy_function=os.link)\r\n File \"/usr/lib/python3.7/shutil.py\", line 365, in copytree\r\n raise Error(errors)\r\nshutil.Error: [('/myfolder/youtubeComunePalermo/processing/./template/base.html', '/tmp/tmps9_4mzc4/templates/base.html', \"[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/base.html' -> '/tmp/tmps9_4mzc4/templates/base.html'\"), ('/myfolder/youtubeComunePalermo/processing/./template/index.html', '/tmp/tmps9_4mzc4/templates/index.html', \"[Errno 18] Invalid cross-device link: '/myfolder/youtubeComunePalermo/processing/./template/index.html' -> '/tmp/tmps9_4mzc4/templates/index.html'\")]\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File \"/home/aborruso/.local/bin/datasette\", line 8, in \r\n sys.exit(cli())\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/click/core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/click/core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py\", line 103, in heroku\r\n extra_metadata,\r\n File \"/usr/lib/python3.7/contextlib.py\", line 112, in __enter__\r\n return next(self.gen)\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py\", line 191, in temporary_heroku_directory\r\n os.path.join(tmp.name, \"templates\"),\r\n File \"/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py\", line 609, in link_or_copy_directory\r\n shutil.copytree(src, dst)\r\n File \"/usr/lib/python3.7/shutil.py\", line 321, in copytree\r\n os.makedirs(dst)\r\n File \"/usr/lib/python3.7/os.py\", line 221, in makedirs\r\n mkdir(name, mode)\r\nFileExistsError: [Errno 17] File exists: '/tmp/tmps9_4mzc4/templates'\r\n```\r\n\r\nI'm attaching my very basic template folder.\r\n\r\nThank you \r\n\r\n[template.zip](https://github.com/simonw/datasette/files/4543751/template.zip)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/744/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 612382643, "node_id": "MDU6SXNzdWU2MTIzODI2NDM=", "number": 758, "title": "Question: Access to immutable database-path", "user": {"value": 2181410, "label": "clausjuhl"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-05-05T07:01:18Z", "updated_at": "2020-05-28T08:23:27Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi Simon\r\n\r\nIs there anywhere in the app-context where one can access the hashed urlpath of the database? Currently it's included in the template-context (`databases[0][\"path\")` when rendering urls of the database (eg. `/db-44b06v9/cases`...), but where can I find the hashed url when rendering the index-page? I'm trying to avoid redirects. Thanks!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/758/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 625930207, "node_id": "MDU6SXNzdWU2MjU5MzAyMDc=", "number": 770, "title": "register_output_renderer can_render mechanism", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 4, "created_at": "2020-05-27T18:29:14Z", "updated_at": "2020-05-28T05:57:16Z", "closed_at": "2020-05-28T05:57:16Z", "author_association": "OWNER", "pull_request": null, "body": "I would like is the ability for renderers to opt-in / opt-out of being displayed as options on the page.\r\n\r\nhttps://www.niche-museums.com/browse/museums for example shows a atom link because the datasette-atom plugin is installed... but clicking it will give you a 400 error because the correct columns are not present.\r\n\r\n\"browse__museums__102_rows\"\r\n\r\nHere's the code that passes a list of renderers to the template:\r\n\r\nhttps://github.com/simonw/datasette/blob/2d099ad9c657d2cab59de91cdb8bfed2da236ef6/datasette/views/base.py#L411-L423\r\n\r\nA renderer is currently defined as a two-key dictionary:\r\n```python\r\n@hookimpl\r\ndef register_output_renderer(datasette):\r\n return {\r\n 'extension': 'test',\r\n 'callback': render_test\r\n }\r\n```\r\nI can add a third key, `\"should_suggest\"` which is a function that returns `True` or `False` for a given query. If that key is missing it is assumed to return `True`.\r\n\r\nOne catch: what arguments should be passed to the `should_suggest(...)` function?\r\n\r\nUPDATE: now calling it `can_render` instead.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/581#issuecomment-634856748_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/770/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611540797, "node_id": "MDU6SXNzdWU2MTE1NDA3OTc=", "number": 751, "title": "Ability to set custom default _size on a per-table basis", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 4, "created_at": "2020-05-04T00:13:03Z", "updated_at": "2020-05-28T05:00:22Z", "closed_at": "2020-05-28T05:00:20Z", "author_association": "OWNER", "pull_request": null, "body": "I have some tables where I'd like the default page size to be 10, without affecting the rest of my Datasette instance.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/751/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 626001501, "node_id": "MDU6SXNzdWU2MjYwMDE1MDE=", "number": 773, "title": "All plugin hooks should have unit tests", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 2, "created_at": "2020-05-27T20:17:41Z", "updated_at": "2020-05-28T04:12:11Z", "closed_at": "2020-05-28T04:09:25Z", "author_association": "OWNER", "pull_request": null, "body": "Four hooks currently missing tests:\r\n\r\n- [x] prepare_jinja2_environment\r\n- [x] publish_subcommand\r\n- [x] register_facet_classes\r\n- [x] register_output_renderer\r\n\r\n```\r\n$ pytest -k test_plugin_hooks_have_tests -vv\r\n====================================== test session starts ======================================\r\nplatform darwin -- Python 3.7.7, pytest-5.2.4, py-1.8.1, pluggy-0.13.1 -- /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/python\r\ncachedir: .pytest_cache\r\nrootdir: /Users/simon/Dropbox/Development/datasette, inifile: pytest.ini\r\nplugins: asyncio-0.10.0\r\ncollected 486 items / 475 deselected / 11 selected \r\n\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[asgi_wrapper] XPASS [ 9%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[extra_body_script] XPASS [ 18%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[extra_css_urls] XPASS [ 27%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[extra_js_urls] XPASS [ 36%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[extra_template_vars] XPASS [ 45%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[prepare_connection] XPASS [ 54%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[prepare_jinja2_environment] XFAIL [ 63%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[publish_subcommand] XFAIL [ 72%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[register_facet_classes] XFAIL [ 81%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[register_output_renderer] XFAIL [ 90%]\r\ntests/test_plugins.py::test_plugin_hooks_have_tests[render_cell] XPASS [100%]\r\n\r\n========================= 475 deselected, 4 xfailed, 7 xpassed in 1.70s =========================\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/771#issuecomment-634915104_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/773/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 626163974, "node_id": "MDU6SXNzdWU2MjYxNjM5NzQ=", "number": 776, "title": "register_output_renderer render callback should be optionally awaitable", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 1, "created_at": "2020-05-28T02:26:29Z", "updated_at": "2020-05-28T02:43:36Z", "closed_at": "2020-05-28T02:43:36Z", "author_association": "OWNER", "pull_request": null, "body": "In #581 I made a bunch of improvements to this, including making `datasette` available to it so it could execute queries.\r\n\r\nBut... it needs to be able to `await` in order to do that. Which means it should be optionally-awaitable.\r\n\r\nOriginal idea here: https://github.com/simonw/datasette/issues/645#issuecomment-560036740", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/776/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 502993509, "node_id": "MDU6SXNzdWU1MDI5OTM1MDk=", "number": 581, "title": "Redesign register_output_renderer callback", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 24, "created_at": "2019-10-05T17:43:23Z", "updated_at": "2020-05-28T02:24:14Z", "closed_at": "2020-05-28T02:21:50Z", "author_association": "OWNER", "pull_request": null, "body": "In building https://github.com/simonw/datasette-atom it became clear that the callback function (which currently accepts just args, data and view_name) would also benefit from access to a mechanism to render templates and a `datasette` instance so it can execute SQL.\r\n\r\nTo maintain backwards compatibility with existing plugins, we can introspect the callback function to see if it wants those new arguments or not.\r\n\r\nAt a minimum I want to make `datasette` and ASGI `scope` available.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/581/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 530653633, "node_id": "MDU6SXNzdWU1MzA2NTM2MzM=", "number": 645, "title": "Mechanism for register_output_renderer to suggest extension or not", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-12-01T01:26:27Z", "updated_at": "2020-05-28T02:22:18Z", "closed_at": "2020-05-28T02:22:12Z", "author_association": "OWNER", "pull_request": null, "body": "[datasette-atom](https://github.com/simonw/datasette-atom) only works if the user constructs a SQL query with specific output columns (`atom_id` ,`atom_updated` etc).\r\n\r\nIt would be good if the `.atom` link wasn't shown on the query/table page unless those columns were present. Right now you get a link which results in a 400 error:\r\n\r\n\"browse__museums__51_rows\"\r\n\r\nSee also #581.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/645/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 626131309, "node_id": "MDU6SXNzdWU2MjYxMzEzMDk=", "number": 775, "title": "Move test plugins into datasette/tests/plugins/ directory", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-28T00:46:58Z", "updated_at": "2020-05-28T00:57:31Z", "closed_at": "2020-05-28T00:57:31Z", "author_association": "OWNER", "pull_request": null, "body": "Right now the plugins used during test runs are baked into strings. It would be nicer if they were actual files on disk.\r\n\r\nWill make #581 easier to write tests for.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/775/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 620969465, "node_id": "MDU6SXNzdWU2MjA5Njk0NjU=", "number": 767, "title": "Allow to specify a URL fragment for canned queries", "user": {"value": 2657547, "label": "rixx"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 2, "created_at": "2020-05-19T13:17:42Z", "updated_at": "2020-05-27T21:52:25Z", "closed_at": "2020-05-27T21:52:25Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Canned queries are very useful to direct users to prepared data and views. I like to use them with charts using datasette-vega a lot, because people get a direct impression at first glance.\r\n\r\ndatasette-vega doesn't show up by default though, and users have to click through to it. Also, datasette-vega does not always guess the best way to render columns correctly though, so it would be nice if I could specify a URL fragment in my canned queries to make sure people see what I want them to see.\r\n\r\nMy current workaround is to include a fragement link in ``description_html`` and ask people to reload the page, like [here](https://data.rixx.de/songs/show_by_bpm#g.mark=bar&g.x_column=bpm_floor&g.x_type=ordinal&g.y_column=bpm_count&g.y_type=quantitative), which is a bit hacky.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/767/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 616012427, "node_id": "MDU6SXNzdWU2MTYwMTI0Mjc=", "number": 764, "title": "Add PyPI project urls to setup.py", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 3, "created_at": "2020-05-11T16:23:08Z", "updated_at": "2020-05-27T20:21:36Z", "closed_at": "2020-05-11T18:28:55Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted this example here:\r\n```python\r\n project_urls={\r\n \"Issues\": \"https://gitlab.com/Cyb3r-Jak3/ExifReader/issues\",\r\n \"Source Code\": \"https://gitlab.com/Cyb3r-Jak3/ExifReader/-/tree/publish\",\r\n \"CI\": \"https://gitlab.com/Cyb3r-Jak3/ExifReader/pipelines\",\r\n \"Releases\": \"https://github.com/Cyb3r-Jak3/ExifReader\"\r\n },\r\n```\r\nResults in this on https://pypi.org/project/ExifReader/\r\n\r\n\"ExifReader_\u00b7_PyPI\"\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/764/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 625980317, "node_id": "MDU6SXNzdWU2MjU5ODAzMTc=", "number": 771, "title": "Unit test that checks that all plugin hooks have corresponding unit tests", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5471110, "label": "Datasette 0.43"}, "comments": 5, "created_at": "2020-05-27T19:42:35Z", "updated_at": "2020-05-27T20:21:36Z", "closed_at": "2020-05-27T20:17:13Z", "author_association": "OWNER", "pull_request": null, "body": "Turns out some hooks are missing unit test coverage: https://github.com/simonw/datasette/issues/581#issuecomment-634893744_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/771/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 613006393, "node_id": "MDU6SXNzdWU2MTMwMDYzOTM=", "number": 20, "title": "Ability to serve thumbnailed Apple Photo from its place on disk", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 10, "created_at": "2020-05-06T02:17:50Z", "updated_at": "2020-05-25T20:14:22Z", "closed_at": "2020-05-25T20:09:41Z", "author_association": "MEMBER", "pull_request": null, "body": "A custom Datasette plugin that can be run locally on a Mac laptop which knows how to serve photos such that they can be seen in the browser.\r\n\r\n_Originally posted by @simonw in https://github.com/dogsheep/photos-to-sqlite/issues/19#issuecomment-624406285_", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 621332242, "node_id": "MDU6SXNzdWU2MjEzMzIyNDI=", "number": 25, "title": "Create a public demo", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-05-19T22:47:20Z", "updated_at": "2020-05-21T22:26:16Z", "closed_at": "2020-05-20T05:54:18Z", "author_association": "MEMBER", "pull_request": null, "body": "So I can show people what this does, using some of my photos.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 621486115, "node_id": "MDU6SXNzdWU2MjE0ODYxMTU=", "number": 27, "title": "photos_with_apple_metadata view should include labels", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-05-20T06:06:17Z", "updated_at": "2020-05-20T06:06:17Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "https://dogsheep-photos.dogsheep.net/public/photos_with_apple_metadata?place_city=New+Orleans&_facet=place_city&_facet_array=albums&_facet_array=persons\r\n\r\nHere's one way to add that:\r\n```sql\r\n select\r\n rowid,\r\n photo,\r\n (\r\n select\r\n json_group_array(\r\n json_object(\r\n 'label',\r\n normalized_string,\r\n 'href',\r\n '/photos/labelled?_hide_sql=1&label=' || normalized_string\r\n )\r\n )\r\n from\r\n labels\r\n where\r\n labels.uuid = photos_with_apple_metadata.uuid\r\n ) as labels,\r\n date,\r\n```", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/27/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 621323348, "node_id": "MDU6SXNzdWU2MjEzMjMzNDg=", "number": 24, "title": "Configurable URL for images", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-19T22:25:56Z", "updated_at": "2020-05-20T06:00:29Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "This is hard-coded at the moment, which is bad:\r\nhttps://github.com/dogsheep/photos-to-sqlite/blob/d5d69b9019703c47bc251444838578dd752801e2/photos_to_sqlite/cli.py#L269-L272", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/24/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 621444763, "node_id": "MDU6SXNzdWU2MjE0NDQ3NjM=", "number": 26, "title": "Rename project to dogsheep-photos", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2020-05-20T04:12:34Z", "updated_at": "2020-05-20T04:31:02Z", "closed_at": "2020-05-20T04:30:40Z", "author_association": "MEMBER", "pull_request": null, "body": "`photos-to-sqlite` doesn't really capture the full scope of this project anymore.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 621280529, "node_id": "MDU6SXNzdWU2MjEyODA1Mjk=", "number": 23, "title": "create-subset command for creating a publishable subset of a photos database", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-19T20:58:20Z", "updated_at": "2020-05-19T22:32:48Z", "closed_at": "2020-05-19T22:32:37Z", "author_association": "MEMBER", "pull_request": null, "body": "I want to share a subset of my photos, without sharing everything. Idea:\r\n\r\n $ photos-to-sqlite create-subset photos.db public.db \"select sha256 from ... where ...\"\r\n\r\nSo the command takes a SQL query that returns sha256 hashes, then creates a new file called `public.db` containing just the data corresponding to those photos.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/23/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 613002220, "node_id": "MDU6SXNzdWU2MTMwMDIyMjA=", "number": 19, "title": "apple-photos command should work even if upload has not run", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-06T02:02:25Z", "updated_at": "2020-05-19T20:59:59Z", "closed_at": "2020-05-19T20:59:59Z", "author_association": "MEMBER", "pull_request": null, "body": "I want people to be able to query their Apple Photos metadata without having to first run `upload` to upload all of their files to their own S3 bucket.\r\n\r\nTo do this I can have `apple-photos` calculate SHA256 hashes of each photo if the `uploads` table does not yet exist (or does not contain that photo).", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 603295970, "node_id": "MDU6SXNzdWU2MDMyOTU5NzA=", "number": 729, "title": "Visually distinguish integer and text columns", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2020-04-20T14:47:26Z", "updated_at": "2020-05-18T17:20:02Z", "closed_at": "2020-05-15T18:16:56Z", "author_association": "OWNER", "pull_request": null, "body": "It would be useful if I could tell from looking at the table page if a column was a integer or a text (or a float I guess?). This is particularly important for knowing if it safe to sort by that column.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/729/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 615626118, "node_id": "MDU6SXNzdWU2MTU2MjYxMTg=", "number": 22, "title": "Try out ExifReader", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-05-11T06:32:13Z", "updated_at": "2020-05-14T05:59:53Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "https://pypi.org/project/ExifReader/\r\n\r\nNew fork that should be able to handle EXIF in HEIC files.\r\n\r\nForked here: https://github.com/ianare/exif-py/issues/102#issuecomment-626376522\r\n\r\nRefs #3 ", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 610517472, "node_id": "MDU6SXNzdWU2MTA1MTc0NzI=", "number": 103, "title": "sqlite3.OperationalError: too many SQL variables in insert_all when using rows with varying numbers of columns", "user": {"value": 32605365, "label": "b0b5h4rp13"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2020-05-01T02:26:14Z", "updated_at": "2020-05-14T00:18:57Z", "closed_at": "2020-05-14T00:18:57Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "If using insert_all to put in 1000 rows of data with varying number of columns, it comes up with this message `sqlite3.OperationalError: too many SQL variables` if the number of columns is larger in later records (past the first row)\r\n\r\nI've reduced `SQLITE_MAX_VARS` by 100 to 899 at the top of `db.py` to add wiggle room, so that if the column count increases it wont go past SQLite's batch limit as calculated by this line of code based on the count of the first row's dict keys\r\n\r\n batch_size = max(1, min(batch_size, SQLITE_MAX_VARS // num_columns))", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/103/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 461215118, "node_id": "MDU6SXNzdWU0NjEyMTUxMTg=", "number": 30, "title": "Option to open database in read-only mode", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-06-26T22:50:38Z", "updated_at": "2020-05-11T19:17:17Z", "closed_at": "2020-05-11T19:17:17Z", "author_association": "OWNER", "pull_request": null, "body": "Would this make it 100% safe to run reads against a database file that is being written to by another process?", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/30/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 616087149, "node_id": "MDU6SXNzdWU2MTYwODcxNDk=", "number": 765, "title": "publish heroku should default to currently tagged version", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-11T18:24:06Z", "updated_at": "2020-05-11T18:25:43Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Had a report that deploying to Heroku was using the previously installed version of Datasette, not the latest.\r\n\r\nCould be because of this:\r\n\r\nhttps://github.com/simonw/datasette/blob/af6c6c5d6f929f951c0e63bfd1c82e37a071b50f/datasette/publish/heroku.py#L172-L179\r\n\r\nHeroku documentation recommends pinning to specific versions https://devcenter.heroku.com/articles/python-pip\r\n\r\nSo... we could ensure we default to an install value of `[\"datasette>=current_tag\"]`.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/765/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 615477131, "node_id": "MDU6SXNzdWU2MTU0NzcxMzE=", "number": 111, "title": "sqlite-utils drop-table and drop-view commands", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-10T21:10:42Z", "updated_at": "2020-05-11T01:58:36Z", "closed_at": "2020-05-11T00:44:26Z", "author_association": "OWNER", "pull_request": null, "body": "Would be useful to be able to drop views and tables from the CLI.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/111/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 613755043, "node_id": "MDU6SXNzdWU2MTM3NTUwNDM=", "number": 110, "title": "Support decimal.Decimal type", "user": {"value": 134771, "label": "dvhthomas"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-05-07T03:57:19Z", "updated_at": "2020-05-11T01:58:20Z", "closed_at": "2020-05-11T01:50:11Z", "author_association": "NONE", "pull_request": null, "body": "Decimal types in Postgres cause a failure in db.py data type selection\r\n---\r\nI have a Django app using a MoneyField, which uses a `numeric(14,0)` data type in Postgres (https://www.postgresql.org/docs/9.3/datatype-numeric.html). When attempting to export that table I get the following error:\r\n\r\n```bash\r\n$ db-to-sqlite --table isaweb_proposal \"postgres://connection\" test.db\r\n....\r\n column_type=COLUMN_TYPE_MAPPING[column_type],\r\nKeyError: \r\n```\r\n\r\nLooking at `sql_utils.db.py` at 292-ish it's clear that there is no matching type for what I assume SQLAlchemy interprets as Python decimal.Decimal.\r\n\r\nFrom the [SQLite docs](https://www.sqlite.org/datatype3.html#affinity_name_examples) it looks like DECIMAL in other DBs are considered numeric.\r\n\r\nI'm not quite sure if it's as simple as adding a data type to that list or if there are repercussions beyond it.\r\n\r\nThanks for a great tool!", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/110/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 322283067, "node_id": "MDU6SXNzdWUzMjIyODMwNjc=", "number": 254, "title": "Escaping named parameters in canned queries", "user": {"value": 247131, "label": "philroche"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2018-05-11T12:43:30Z", "updated_at": "2020-05-10T14:54:14Z", "closed_at": "2020-05-10T14:54:13Z", "author_association": "NONE", "pull_request": null, "body": "Thank you very much for this project.\r\n\r\nI have created some canned queries but some of the filters include a colon eg. \"com.ubuntu.cloud:server:18.04:amd64\". When saved these colons are parsed as named parameters. \r\n\r\nIs there a way to escape colons in a canned query?", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/254/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 534492501, "node_id": "MDU6SXNzdWU1MzQ0OTI1MDE=", "number": 648, "title": "Mechanism for adding arbitrary pages like /about", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 13, "created_at": "2019-12-08T04:55:19Z", "updated_at": "2020-05-07T15:21:19Z", "closed_at": "2020-04-26T18:46:45Z", "author_association": "OWNER", "pull_request": null, "body": "For www.niche-museums.com I solved this by creating an empty `about.db` database file - see https://simonwillison.net/2019/Nov/25/niche-museums/\r\n\r\nI want a neater mechanism for this.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/648/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 613467382, "node_id": "MDU6SXNzdWU2MTM0NjczODI=", "number": 761, "title": "Allow-list pragma_table_info(tablename) and similar", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2020-05-06T16:54:14Z", "updated_at": "2020-05-07T03:09:05Z", "closed_at": "2020-05-06T17:18:38Z", "author_association": "OWNER", "pull_request": null, "body": "It would be great if `pragma_table_info(tablename)` was allowed to be used in queries. See also https://github.com/simonw/til/blob/master/sqlite/list-all-columns-in-a-database.md\r\n> `select * from pragma_table_info(tablename);` is currently disallowed for user-provided queries via a regex restriction - but could help here too.\r\n> \r\n> https://github.com/simonw/datasette/blob/d349d57cdf3d577afb62bdf784af342a4d5be660/datasette/utils/__init__.py#L174\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette/issues/760#issuecomment-624729459_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/761/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 613491342, "node_id": "MDU6SXNzdWU2MTM0OTEzNDI=", "number": 762, "title": "Experiment with PRAGMA hard_heap_limit ", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-05-06T17:33:23Z", "updated_at": "2020-05-07T03:08:44Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "This was added in SQLite 2020-01-22 (3.31.0): https://www.sqlite.org/changes.html#version_3_31_0\r\n\r\n> Add the [sqlite3_hard_heap_limit64()](https://www.sqlite.org/c3ref/hard_heap_limit64.html) interface and the corresponding [PRAGMA hard_heap_limit](https://www.sqlite.org/pragma.html#pragma_hard_heap_limit) command. \r\n\r\nThis sounds like it could be a nice extra safety measure.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/762/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 613422636, "node_id": "MDU6SXNzdWU2MTM0MjI2MzY=", "number": 760, "title": "Way of seeing full schema for a database", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-05-06T15:46:08Z", "updated_at": "2020-05-06T23:49:06Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "I find myself wanting to quickly figure out all of the BLOB columns in a database.\r\n\r\nA `/-/schema` page showing the full schema (actually since it's per-database probably `/dbname/-/schema` or `/-/schema/dbname`) would be really handy.\r\n\r\nIt would need to be carefully constructed from various queries against `sqlite_master` - just doing `select * from sqlite_master where type='table'` isn't quite enough because I also want to show indexes, triggers etc.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/760/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 612378203, "node_id": "MDU6SXNzdWU2MTIzNzgyMDM=", "number": 757, "title": "Question: Any fixed date for the release with the uft8-encoding fix?", "user": {"value": 2181410, "label": "clausjuhl"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-05-05T06:51:20Z", "updated_at": "2020-05-06T18:41:29Z", "closed_at": "2020-05-06T18:41:29Z", "author_association": "NONE", "pull_request": null, "body": "Just a little impatient :)", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/757/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 612860758, "node_id": "MDU6SXNzdWU2MTI4NjA3NTg=", "number": 18, "title": "Switch CI solution to GitHub Actions with a macOS runner", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-05T20:03:50Z", "updated_at": "2020-05-05T23:49:18Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Refs #17.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/18/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 612860531, "node_id": "MDU6SXNzdWU2MTI4NjA1MzE=", "number": 17, "title": "Only install osxphotos if running on macOS", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-05-05T20:03:26Z", "updated_at": "2020-05-05T20:20:05Z", "closed_at": "2020-05-05T20:11:23Z", "author_association": "MEMBER", "pull_request": null, "body": "The build is broken right now because you can't `pip install osxphotos` on Ubuntu.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 612658444, "node_id": "MDU6SXNzdWU2MTI2NTg0NDQ=", "number": 109, "title": "table.create_index(..., ignore=True)", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-05T14:44:21Z", "updated_at": "2020-05-05T14:46:53Z", "closed_at": "2020-05-05T14:46:53Z", "author_association": "OWNER", "pull_request": null, "body": "Option to silently do nothing if the index already exists.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/109/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 612287234, "node_id": "MDU6SXNzdWU2MTIyODcyMzQ=", "number": 16, "title": "Import machine-learning detected labels (dog, llama etc) from Apple Photos", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 13, "created_at": "2020-05-05T02:45:43Z", "updated_at": "2020-05-05T05:38:16Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Follow-on from #1. Apple Photos runs some very sophisticated machine learning on-device to figure out if photos are of dogs, llamas and so on. I really want to extract those labels out into my own database.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16/reactions\", \"total_count\": 2, \"+1\": 0, \"-1\": 0, \"laugh\": 1, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 612089949, "node_id": "MDU6SXNzdWU2MTIwODk5NDk=", "number": 756, "title": "Add pipx to installation documentation", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-04T18:49:01Z", "updated_at": "2020-05-04T19:19:06Z", "closed_at": "2020-05-04T19:10:33Z", "author_association": "OWNER", "pull_request": null, "body": "Add to this page: https://datasette.readthedocs.io/en/stable/installation.html\r\n\r\nHere's how to install plugins: https://twitter.com/simonw/status/1257348687979778050\r\n```\r\n$ datasette plugins\r\n[]\r\n\r\n$ pipx inject datasette datasette-json-html \r\n injected package datasette-json-html into venv datasette\r\ndone! \u2728 \ud83c\udf1f \u2728\r\n\r\n$ datasette plugins\r\n[\r\n {\r\n \"name\": \"datasette-json-html\",\r\n \"static\": false,\r\n \"templates\": false,\r\n \"version\": \"0.6\"\r\n }\r\n]\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/756/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 612082842, "node_id": "MDU6SXNzdWU2MTIwODI4NDI=", "number": 755, "title": "Fix \"no such column: id\" output in tests", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-04T18:37:49Z", "updated_at": "2020-05-04T18:42:14Z", "closed_at": "2020-05-04T18:42:14Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\npytest\r\n...\r\ntests/test_custom_pages.py ........ [ 33%]\r\ntests/test_database.py ......no such column: id\r\n... [ 35%]\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/755/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611835285, "node_id": "MDU6SXNzdWU2MTE4MzUyODU=", "number": 752, "title": "Non-utf8 encoding in exceptionhandlers and custom-pages", "user": {"value": 2181410, "label": "clausjuhl"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-04T12:24:42Z", "updated_at": "2020-05-04T17:42:20Z", "closed_at": "2020-05-04T17:42:20Z", "author_association": "NONE", "pull_request": null, "body": "Hi Simon.\r\n\r\nWhenever a response is not piped through a router-view, the template is encoded in latin-1 (I think). This is especially a problem (for me) with the new custom_pages-functionality, but also problematic with the 404- and 500-handlers. Thanks!", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/752/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611997130, "node_id": "MDU6SXNzdWU2MTE5OTcxMzA=", "number": 754, "title": "Clean up aiofiles warnings on 3.8", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-04T16:14:59Z", "updated_at": "2020-05-04T16:22:30Z", "closed_at": "2020-05-04T16:22:30Z", "author_association": "OWNER", "pull_request": null, "body": "https://travis-ci.org/github/simonw/datasette/jobs/682624476\r\n\r\nLots of warnings like this:\r\n```\r\n/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33\r\n\r\n/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33\r\n\r\n /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/utils.py:33: DeprecationWarning: \"@coroutine\" decorator is deprecated since Python 3.8, use \"async def\" instead\r\n\r\n def method(self, *args, **kwargs):\r\n\r\n/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/__init__.py:27\r\n\r\n /home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/aiofiles/threadpool/__init__.py:27: DeprecationWarning: \"@coroutine\" decorator is deprecated since Python 3.8, use \"async def\" instead\r\n\r\n def _open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None,\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/754/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 602533300, "node_id": "MDU6SXNzdWU2MDI1MzMzMDA=", "number": 1, "title": "Import photo metadata from Apple Photos into SQLite", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": {"value": 5324096, "label": "Apple Photos online and securely browsable"}, "comments": 8, "created_at": "2020-04-18T19:23:26Z", "updated_at": "2020-05-04T02:41:40Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Faces, albums, locations, that kind of thing.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 610408908, "node_id": "MDU6SXNzdWU2MTA0MDg5MDg=", "number": 34, "title": "Command for retrieving dependents for a repo", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2020-04-30T21:47:51Z", "updated_at": "2020-05-03T15:53:01Z", "closed_at": "2020-05-03T15:53:01Z", "author_association": "MEMBER", "pull_request": null, "body": "I really, really want to start grabbing this data: https://github.com/simonw/datasette/network/dependents", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611222968, "node_id": "MDU6SXNzdWU2MTEyMjI5Njg=", "number": 107, "title": "sqlite-utils create-view CLI command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-02T16:15:13Z", "updated_at": "2020-05-03T15:36:58Z", "closed_at": "2020-05-03T15:36:37Z", "author_association": "OWNER", "pull_request": null, "body": "Can go with #27 - `sqlite-utils create-table`.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/107/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 455496504, "node_id": "MDU6SXNzdWU0NTU0OTY1MDQ=", "number": 27, "title": "sqlite-utils create-table command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2019-06-13T01:43:30Z", "updated_at": "2020-05-03T15:26:15Z", "closed_at": "2020-05-03T15:26:15Z", "author_association": "OWNER", "pull_request": null, "body": "Spun off from #24 - it would be useful if CLI users could create new tables (with explicit column types, not null rules and defaults) without having to insert an example record.\r\n\r\n- [x] Get it working\r\n- [x] Support `--pk`\r\n- [x] Support `--not-null`\r\n- [x] Support `--default`\r\n- [x] Support `--fk colname othertable othercol`\r\n- [x] Support `--replace` and `--ignore`\r\n- [x] Documentation", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/27/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611326701, "node_id": "MDU6SXNzdWU2MTEzMjY3MDE=", "number": 108, "title": "Documentation unit tests for CLI commands", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-03T03:58:42Z", "updated_at": "2020-05-03T04:13:57Z", "closed_at": "2020-05-03T04:13:57Z", "author_association": "OWNER", "pull_request": null, "body": "Have a test that ensures all CLI commands are documented.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/108/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611284481, "node_id": "MDU6SXNzdWU2MTEyODQ0ODE=", "number": 38, "title": "[Feature Request] Support Repo Name in Search \ud83e\udd7a", "user": {"value": 5779832, "label": "zzeleznick"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-05-02T22:08:51Z", "updated_at": "2020-05-03T02:34:32Z", "closed_at": "2020-05-02T23:15:11Z", "author_association": "NONE", "pull_request": null, "body": "## Description\r\nPer your [v2.2 release tweet](https://twitter.com/simonw/status/1256700238099693568) I played with the demo, but the output did not match my expectations.\r\n\r\n## Expected Behavior\r\nExpected a search query for \"twitter\" contained within the `repo` column to return non-zero results.\r\n\r\n## Actual Behavior\r\n\ud83d\ude2d [0 rows where repo contains \"twitter\" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=twitter&_sort_desc=starred_at) \r\n\r\n## Best Explanation\r\nPer the table schema (see appendix) `repo` is of type `INTEGER` which built from `repo_id` and does not expose the repo name in search.\r\n\r\n## Desired Behavior\r\nGiven that searching for \"206156866\" is less intuitive than \"twitter\", it would be great to support this via extending the search capabilities or by adding an additional column.\r\n\r\n\u2705 104 rows where repo contains \"twitter\"\r\n\u274c [104 rows where repo contains \"206156866\" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=206156866&_sort_desc=starred_at) \r\n\r\n## Appendix\r\n```\r\nCREATE TABLE [stars] (\r\n [user] INTEGER REFERENCES [users]([id]),\r\n [repo] INTEGER REFERENCES [repos]([id]),\r\n [starred_at] TEXT,\r\n PRIMARY KEY ([user], [repo])\r\n);\r\nCREATE INDEX [idx_stars_repo]\r\n ON [stars] ([repo]);\r\nCREATE INDEX [idx_stars_user]\r\n ON [stars] ([user]);\r\n```", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 493670730, "node_id": "MDU6SXNzdWU0OTM2NzA3MzA=", "number": 4, "title": "Command to fetch stargazers for one or more repos", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 8, "created_at": "2019-09-14T21:58:22Z", "updated_at": "2020-05-02T21:30:27Z", "closed_at": "2020-05-02T21:30:27Z", "author_association": "MEMBER", "pull_request": null, "body": "Maybe this:\r\n\r\n $ github-to-sqlite stargazers github.db simonw/datasette\r\n\r\nIt could accept more than one repos.\r\n\r\nMaybe have options similar to `--sql` in [twitter-to-sqlite](https://github.com/dogsheep/twitter-to-sqlite) so you can e.g. fetch all stargazers for all of the repos you have fetched into the database already (or all of the repos belonging to owner X)", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611252244, "node_id": "MDU6SXNzdWU2MTEyNTIyNDQ=", "number": 750, "title": "Add notlike table filter", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-05-02T18:54:36Z", "updated_at": "2020-05-02T19:10:44Z", "closed_at": "2020-05-02T19:10:44Z", "author_association": "OWNER", "pull_request": null, "body": "I found myself wanting that for applying the opposite of this: https://github-to-sqlite.dogsheep.net/github/dependent_repos?dependent__like=%25simonw%2F%25&_sort_desc=dependent_stars\r\n\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/750/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 520756546, "node_id": "MDU6SXNzdWU1MjA3NTY1NDY=", "number": 12, "title": "Add this view for seeing new releases", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2019-11-11T06:00:12Z", "updated_at": "2020-05-02T18:58:18Z", "closed_at": "2020-05-02T18:58:17Z", "author_association": "MEMBER", "pull_request": null, "body": "```sql\r\nCREATE VIEW recent_releases AS select\r\n json_object(\"label\", repos.full_name, \"href\", repos.html_url) as repo,\r\n json_object(\r\n \"href\",\r\n releases.html_url,\r\n \"label\",\r\n releases.name\r\n ) as release,\r\n substr(releases.published_at, 0, 11) as date,\r\n releases.body as body_markdown,\r\n releases.published_at\r\nfrom\r\n releases\r\n join repos on repos.id = releases.repo\r\norder by\r\n releases.published_at desc\r\n```", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 609950090, "node_id": "MDU6SXNzdWU2MDk5NTAwOTA=", "number": 33, "title": "Fall back to authentication via ENV", "user": {"value": 2029, "label": "garethr"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-04-30T12:58:14Z", "updated_at": "2020-05-02T18:46:10Z", "closed_at": "2020-05-02T18:45:37Z", "author_association": "NONE", "pull_request": null, "body": "Would you accept a PR that falls back to looking for an environment variable for the GitHub token? Specifically a change here:\r\nhttps://github.com/dogsheep/github-to-sqlite/blob/c34d5a18bfc41fa08755ba3d5cf9fe09ff204238/github_to_sqlite/cli.py#L271\r\n\r\nI'd like to use `github-to-sqlite` in a GitHub Action workflow and this would be simpler than trying to fill out the prompt or generate a file with sensitive content.\r\n\r\nWanted to check first, I'm happy to submit a PR with tests and updates to the docs. ", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/33/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610511450, "node_id": "MDU6SXNzdWU2MTA1MTE0NTA=", "number": 35, "title": "Create index on issue_comments(user) and other foreign keys", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-05-01T02:06:56Z", "updated_at": "2020-05-02T18:26:24Z", "closed_at": "2020-05-02T18:26:24Z", "author_association": "MEMBER", "pull_request": null, "body": "```\r\ncreate index issue_comments_user on issue_comments(user)\r\n```\r\nI'm sure there are other user columns that could benefit from an index.", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610842926, "node_id": "MDU6SXNzdWU2MTA4NDI5MjY=", "number": 36, "title": "Add view for better display of dependent repos", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-05-01T16:33:44Z", "updated_at": "2020-05-02T16:50:31Z", "closed_at": "2020-05-02T16:30:11Z", "author_association": "MEMBER", "pull_request": null, "body": "```sql\r\nselect\r\n repos.full_name as repo,\r\n 'https://github.com/' || repos2.full_name as dependent,\r\n repos2.created_at as dependent_repo_created,\r\n repos2.updated_at as dependent_repo_updated,\r\n repos2.stargazers_count as dependent_repo_stars,\r\n repos2.watchers_count as dependent_repo_watchers\r\nfrom\r\n dependents\r\n join repos as repos2 on dependents.dependent = repos2.id\r\n join repos on dependents.repo = repos.id\r\norder by\r\n repos2.created_at desc\r\n```\r\nhttps://dogsheep.simonwillison.net/github?sql=select%0D%0A++repos.full_name+as+repo%2C%0D%0A++%27https%3A%2F%2Fgithub.com%2F%27+%7C%7C+repos2.full_name+as+dependent%2C%0D%0A++repos2.created_at+as+dependent_repo_created%2C%0D%0A++repos2.updated_at+as+dependent_repo_updated%2C%0D%0A++repos2.stargazers_count+as+dependent_repo_stars%2C%0D%0A++repos2.watchers_count+as+dependent_repo_watchers%0D%0Afrom%0D%0A++dependents%0D%0A++join+repos+as+repos2+on+dependents.dependent+%3D+repos2.id%0D%0A++join+repos+on+dependents.repo+%3D+repos.id%0D%0Aorder+by%0D%0A++repos2.created_at+desc", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/36/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 516967682, "node_id": "MDU6SXNzdWU1MTY5Njc2ODI=", "number": 10, "title": "Add this repos_starred view", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-11-04T05:44:38Z", "updated_at": "2020-05-02T16:37:36Z", "closed_at": "2020-05-02T16:37:36Z", "author_association": "MEMBER", "pull_request": null, "body": "```sql\r\ncreate view repos_starred as select\r\n stars.starred_at,\r\n users.login,\r\n repos.*\r\nfrom\r\n repos\r\n join stars on repos.id = stars.repo\r\n join users on repos.owner = users.id\r\norder by\r\n starred_at desc;\r\n```", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610843136, "node_id": "MDU6SXNzdWU2MTA4NDMxMzY=", "number": 37, "title": "Mechanism for creating views if they don't yet exist", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-05-01T16:34:10Z", "updated_at": "2020-05-02T16:19:47Z", "closed_at": "2020-05-02T16:19:31Z", "author_association": "MEMBER", "pull_request": null, "body": "Needed for #36 #10 #12 ", "repo": {"value": 207052882, "label": "github-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 611216862, "node_id": "MDU6SXNzdWU2MTEyMTY4NjI=", "number": 106, "title": "create_view(..., ignore=True, replace=True) parameters", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-02T15:45:21Z", "updated_at": "2020-05-02T16:04:51Z", "closed_at": "2020-05-02T16:02:10Z", "author_association": "OWNER", "pull_request": null, "body": "Two new parameters which specify what should happen if the view already exists. I want this for https://github.com/dogsheep/github-to-sqlite/issues/37\r\n\r\nHere's the current `create_view()` implementation:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/b4d953d3ccef28bb81cea40ca165a647b59971fa/sqlite_utils/db.py#L325-L332\r\n\r\n`ignore=True` will not do anything if the view exists already.\r\n\r\n`replace=True` will drop and redefine the view - but only if its SQL definition differs, otherwise it will be left alone.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/106/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 602569315, "node_id": "MDU6SXNzdWU2MDI1NjkzMTU=", "number": 102, "title": "Can't store an array or dictionary containing a bytes value", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-18T22:49:21Z", "updated_at": "2020-05-01T20:45:45Z", "closed_at": "2020-05-01T20:45:45Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\nIn [1]: import sqlite_utils \r\n\r\nIn [2]: db = sqlite_utils.Database(memory=True) \r\n\r\nIn [3]: db[\"t\"].insert({\"id\": 1, \"data\": {\"foo\": b\"bytes\"}}) \r\n---------------------------------------------------------------------------\r\nTypeError Traceback (most recent call last)\r\n in \r\n----> 1 db[\"t\"].insert({\"id\": 1, \"data\": {\"foo\": b\"bytes\"}})\r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert(self, record, pk, foreign_keys, column_order, not_null, defaults, hash_id, alter, ignore, replace, extracts, conversions, columns)\r\n 950 extracts=extracts,\r\n 951 conversions=conversions,\r\n--> 952 columns=columns,\r\n 953 )\r\n 954 \r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, alter, ignore, replace, extracts, conversions, columns, upsert)\r\n 1052 for key in all_columns:\r\n 1053 value = jsonify_if_needed(\r\n-> 1054 record.get(key, None if key != hash_id else _hash(record))\r\n 1055 )\r\n 1056 if key in extracts:\r\n\r\n~/Dropbox/Development/sqlite-utils/sqlite_utils/db.py in jsonify_if_needed(value)\r\n 1318 def jsonify_if_needed(value):\r\n 1319 if isinstance(value, (dict, list, tuple)):\r\n-> 1320 return json.dumps(value)\r\n 1321 elif isinstance(value, (datetime.time, datetime.date, datetime.datetime)):\r\n 1322 return value.isoformat()\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)\r\n 229 cls is None and indent is None and separators is None and\r\n 230 default is None and not sort_keys and not kw):\r\n--> 231 return _default_encoder.encode(obj)\r\n 232 if cls is None:\r\n 233 cls = JSONEncoder\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in encode(self, o)\r\n 197 # exceptions aren't as detailed. The list call should be roughly\r\n 198 # equivalent to the PySequence_Fast that ''.join() would do.\r\n--> 199 chunks = self.iterencode(o, _one_shot=True)\r\n 200 if not isinstance(chunks, (list, tuple)):\r\n 201 chunks = list(chunks)\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in iterencode(self, o, _one_shot)\r\n 255 self.key_separator, self.item_separator, self.sort_keys,\r\n 256 self.skipkeys, _one_shot)\r\n--> 257 return _iterencode(o, 0)\r\n 258 \r\n 259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,\r\n\r\n/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/encoder.py in default(self, o)\r\n 177 \r\n 178 \"\"\"\r\n--> 179 raise TypeError(f'Object of type {o.__class__.__name__} '\r\n 180 f'is not JSON serializable')\r\n 181 \r\n\r\nTypeError: Object of type bytes is not JSON serializable\r\n```", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/102/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610853576, "node_id": "MDU6SXNzdWU2MTA4NTM1NzY=", "number": 105, "title": "\"sqlite-utils views\" command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-05-01T16:56:11Z", "updated_at": "2020-05-01T20:40:07Z", "closed_at": "2020-05-01T20:38:36Z", "author_association": "OWNER", "pull_request": null, "body": "Similar to `sqlite-utils tables`. See also #104.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/105/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610853393, "node_id": "MDU6SXNzdWU2MTA4NTMzOTM=", "number": 104, "title": "--schema option to \"sqlite-utils tables\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-05-01T16:55:49Z", "updated_at": "2020-05-01T17:12:37Z", "closed_at": "2020-05-01T17:12:37Z", "author_association": "OWNER", "pull_request": null, "body": "Adds output showing the table schema.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/104/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610342575, "node_id": "MDU6SXNzdWU2MTAzNDI1NzU=", "number": 748, "title": "?_searchmode=raw should be documented on full-text search page", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-30T19:50:06Z", "updated_at": "2020-04-30T21:06:12Z", "closed_at": "2020-04-30T21:06:12Z", "author_association": "OWNER", "pull_request": null, "body": "It's currently documented here: https://datasette.readthedocs.io/en/stable/json_api.html#special-table-arguments\r\n\r\nBut it should also be described here: https://datasette.readthedocs.io/en/stable/full_text_search.html#the-table-view-api", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/748/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610192152, "node_id": "MDU6SXNzdWU2MTAxOTIxNTI=", "number": 747, "title": "Directory configuration mode should support metadata.yaml", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-04-30T16:05:30Z", "updated_at": "2020-04-30T19:04:19Z", "closed_at": "2020-04-30T19:04:19Z", "author_association": "OWNER", "pull_request": null, "body": "Refs #739 - `metadata.yml` or `metadata.yaml` should be detected in the same way as `metadata.json` is.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/747/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 610284471, "node_id": "MDU6SXNzdWU2MTAyODQ0NzE=", "number": 46, "title": "Error running 'search' for the first time", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-30T18:11:20Z", "updated_at": "2020-04-30T18:11:58Z", "closed_at": "2020-04-30T18:11:58Z", "author_association": "MEMBER", "pull_request": null, "body": "```\r\n% twitter-to-sqlite search infodemic.db '#infodemic'\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/bin/twitter-to-sqlite\", line 11, in \r\n load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')()\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py\", line 867, in search\r\n for tweet in tweets:\r\n File \"/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py\", line 165, in fetch_timeline\r\n [since_type_id, since_key],\r\nsqlite3.OperationalError: no such table: since_ids\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/46/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 275814941, "node_id": "MDU6SXNzdWUyNzU4MTQ5NDE=", "number": 141, "title": "datasette publish can fail if /tmp is on a different device", "user": {"value": 21148, "label": "jacobian"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 2949431, "label": "Custom templates edition"}, "comments": 5, "created_at": "2017-11-21T18:28:05Z", "updated_at": "2020-04-29T03:27:54Z", "closed_at": "2017-12-08T16:06:36Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "`datasette publish` uses hard links to avoid copying the db into a tmp directory. This can fail if `/tmp` is on another device, because hardlinks can't cross devices. You'll see something like this:\r\n\r\n```\r\n$ datasette publish heroku whatever.db\r\n...\r\nOSError: [Errno 18] Invalid cross-device link: '/mnt/c/Users/jacob/c/datasette/whatever.db' -> '/tmp/tmpvxq2yof6/whatever.db'\r\n```\r\n[In my case this is failing because I'm on a Windows machine, using WSL, so my code's on a different virtual filesystem from the Linux subsystem, Because Reasons.]\r\n\r\nI'm not sure if it's possible to detect this (can you figure out which device `/tmp` is on?), or what the fallback should be (soft link? copy?).", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/141/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 522334771, "node_id": "MDU6SXNzdWU1MjIzMzQ3NzE=", "number": 633, "title": "Publish to Heroku is broken: \"WARNING: You must pass the application as an import string to enable 'reload' or 'workers\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-11-13T16:32:11Z", "updated_at": "2020-04-28T20:37:50Z", "closed_at": "2019-11-13T16:43:23Z", "author_association": "OWNER", "pull_request": null, "body": "```\r\n2019-11-13T16:27:59.821483+00:00 heroku[web.1]: Starting process with command `datasette serve --host 0.0.0.0 -i fixtures.db --cors --port 36817 --inspect-file inspect-data.json`\r\n2019-11-13T16:28:01.856471+00:00 heroku[web.1]: State changed from starting to crashed\r\n2019-11-13T16:28:01.750253+00:00 app[web.1]: Serve! files=() (immutables=('fixtures.db',)) on port 36817\r\n2019-11-13T16:28:01.771524+00:00 app[web.1]: WARNING: You must pass the application as an import string to enable 'reload' or 'workers'.\r\n2019-11-13T16:28:01.837839+00:00 heroku[web.1]: Process exited with status 1\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/633/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 608512747, "node_id": "MDU6SXNzdWU2MDg1MTI3NDc=", "number": 14, "title": "Annotate photos using the Google Cloud Vision API", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-04-28T18:09:03Z", "updated_at": "2020-04-28T18:19:06Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "It can detect faces, run OCR, do image labeling (it knows what a lemur is!) and do object localization where it identifies objects and returns bounding polygons for them.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14/reactions\", \"total_count\": 3, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 602533352, "node_id": "MDU6SXNzdWU2MDI1MzMzNTI=", "number": 2, "title": "Ability to convert HEIC images to JPEG", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": {"value": 5324096, "label": "Apple Photos online and securely browsable"}, "comments": 1, "created_at": "2020-04-18T19:23:43Z", "updated_at": "2020-04-28T16:47:21Z", "closed_at": "2020-04-28T16:47:21Z", "author_association": "MEMBER", "pull_request": null, "body": "", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 605806386, "node_id": "MDU6SXNzdWU2MDU4MDYzODY=", "number": 735, "title": "Error when I click on \"View and edit SQL\"", "user": {"value": 30607, "label": "aborruso"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-04-23T19:31:32Z", "updated_at": "2020-04-28T06:10:20Z", "closed_at": "2020-04-27T19:00:30Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nwhen I do it [here](https://my-database.now.sh/commissioniComunePalermo/youtube), I have \"unrecognized token: \"[\"\" error.\r\n\r\nIs it normal?\r\n\r\nThank you", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/735/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 606720674, "node_id": "MDU6SXNzdWU2MDY3MjA2NzQ=", "number": 736, "title": "strange behavior using accented characters", "user": {"value": 30607, "label": "aborruso"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-04-25T08:34:51Z", "updated_at": "2020-04-28T06:09:28Z", "closed_at": "2020-04-27T18:59:16Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nwhen I search `incompatibilit\u00e0` [here](https://my-database.now.sh/commissioniComunePalermo/youtube), using full text search, it becomes `incompatibilit\u00c3\u0083\u00c2\u00a0` and I have no result.\r\n\r\nIf I encode the `\u00e0` char in the URL (`incompatibilit%C3%A0`) I have the right result.\r\n\r\n![image](https://user-images.githubusercontent.com/30607/80275201-00a79380-86e0-11ea-865e-f7e1474e8098.png)\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/736/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 607888367, "node_id": "MDU6SXNzdWU2MDc4ODgzNjc=", "number": 13, "title": "Also upload movie files", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-04-27T22:11:25Z", "updated_at": "2020-04-28T00:39:45Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "The `upload` command currently only handles static images:\r\n\r\nhttps://github.com/dogsheep/photos-to-sqlite/blob/d939455af00e07866686457ee2fcb9b2d1b7194e/photos_to_sqlite/utils.py#L26-L33\r\n\r\nNeed to cover movies taken by my phone and DSLR too.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/13/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 600120439, "node_id": "MDU6SXNzdWU2MDAxMjA0Mzk=", "number": 726, "title": "Foreign key : case of a link to the associated row not displayed", "user": {"value": 6371750, "label": "JBPressac"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-04-15T08:31:27Z", "updated_at": "2020-04-27T22:05:47Z", "closed_at": "2020-04-27T22:05:46Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Hello,\r\nI use Datasette to publish tsv files linked together by foreign keys declared thanks to sqlite-utils. In one table, [prelib_personne](http://crbc-dataset.huma-num.fr/prelib/prelib_personne), the foreign keys are properly noticed by a link to the associated row (for instance ville_naissance_id is properly linked to prelib_ville). But every link to the foreign key prelib_oeuvre.id fails. For instance, [prelib_ecritoeuvre](http://crbc-dataset.huma-num.fr/prelib/prelib_ecritoeuvre) has links to prelib_personne but none to prelib_oeuvre. In despite of the schema:\r\n\r\nCREATE TABLE \"prelib_ecritoeuvre\" (\r\n\"id\" INTEGER,\r\n \"fonction_id\" INTEGER,\r\n \"oeuvre_id\" INTEGER,\r\n \"personne_id\" INTEGER\r\n ,PRIMARY KEY ([id]),\r\n FOREIGN KEY(fonction_id) REFERENCES prelib_fonctionecritoeuvre(id),\r\n FOREIGN KEY(personne_id) REFERENCES prelib_personne(id),\r\n FOREIGN KEY(oeuvre_id) REFERENCES prelib_oeuvre(id)\r\n); \r\n\r\nWould you have any clue to investigate the reason of this problem?\r\nThanks,", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/726/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 607770595, "node_id": "MDU6SXNzdWU2MDc3NzA1OTU=", "number": 743, "title": "escape_fts() does not correctly escape * wildcards", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2020-04-27T18:48:53Z", "updated_at": "2020-04-27T19:11:30Z", "closed_at": "2020-04-27T19:11:01Z", "author_association": "OWNER", "pull_request": null, "body": "Spotted in #732. This should not return any results... but it does:\r\n\r\nhttps://latest.datasette.io/fixtures/searchable?_search=bar%2A&_trace=1\r\n\r\n\"fixtures__searchable__1_row_where_where_search_matches__bar__\"\r\n\r\nThe query from trace is:\r\n```\r\n \"sql\": \"select count(*) from searchable where rowid in (select rowid from searchable_fts where searchable_fts match escape_fts(:search))\",\r\n \"params\": {\r\n \"search\": \"bar*\"\r\n }\r\n```", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/743/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 607211058, "node_id": "MDU6SXNzdWU2MDcyMTEwNTg=", "number": 740, "title": "Don't throw 500 error on attempted directory browse", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-04-27T03:50:11Z", "updated_at": "2020-04-27T18:29:15Z", "closed_at": "2020-04-27T18:29:15Z", "author_association": "OWNER", "pull_request": null, "body": "\"Error_500\"\r\n\r\nThis should be a 403 error instead, because the `--static` mechanism doesn't allow directory browsing.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/740/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 607243940, "node_id": "MDU6SXNzdWU2MDcyNDM5NDA=", "number": 742, "title": "Speed up tests with scope=\"session\"?", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-04-27T05:23:54Z", "updated_at": "2020-04-27T18:24:53Z", "closed_at": "2020-04-27T18:24:53Z", "author_association": "OWNER", "pull_request": null, "body": "Tests are pretty slow - could I speed them up with pytest `scope=\"session\"` on some of the fixtures?\r\n\r\nEg https://travis-ci.org/github/simonw/datasette/jobs/679940036 ran 452 tests in 3m53s - the `test_html` ones seem particularly slow.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/742/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 605110015, "node_id": "MDU6SXNzdWU2MDUxMTAwMTU=", "number": 731, "title": "Option to automatically configure based on directory layout", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 9, "created_at": "2020-04-22T22:17:47Z", "updated_at": "2020-04-27T16:32:44Z", "closed_at": "2020-04-27T16:30:26Z", "author_association": "OWNER", "pull_request": null, "body": "My Datasette projects increasingly take on the following structure:\r\n\r\n- `metadata.json` with the metadata\r\n- One or more `something.db` database files\r\n- A `templates/` folder with some custom templates\r\n- A `plugins/` folder with some custom plugins\r\n\r\nThen I have to run Datasette like this:\r\n\r\n datasette *.db -m metadata.json --template-dir=templates --plugins-dir=plugins\r\n\r\nIt would be really interesting if Datasette had a special mode where you could point it at a directory with the above layout and it would automatically configure itself based on the contents.\r\n\r\nMaybe even allow `datasette serve` to detect if it was passed a single argument that's a directory, not a file, and kick in to \"directory layout configuration mode\" in that case:\r\n\r\n datasette .\r\n", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/731/reactions\", \"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 607086780, "node_id": "MDU6SXNzdWU2MDcwODY3ODA=", "number": 738, "title": "Pass a request object to custom page templates", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-04-26T18:57:48Z", "updated_at": "2020-04-26T19:01:54Z", "closed_at": "2020-04-26T19:01:54Z", "author_association": "OWNER", "pull_request": null, "body": "Follow-up to #648. I'm not passing a request object to `.render_template()` at the moment, which breaks any other custom plugins using e.g. `extra_template_vars()` that were expecting to be able to access the request.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/738/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 606032950, "node_id": "MDU6SXNzdWU2MDYwMzI5NTA=", "number": 11, "title": "Try running S3 uploads in a thread pool", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-24T04:34:31Z", "updated_at": "2020-04-24T16:45:41Z", "closed_at": "2020-04-24T16:45:41Z", "author_association": "MEMBER", "pull_request": null, "body": "Since #10 provided such a speedup, can the same thing be done for the actual uploads?\r\n\r\nhttp://ls.pwd.io/2013/06/parallel-s3-uploads-using-boto-and-threads-in-python/ suggests it can really help performance.", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/11/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 606033104, "node_id": "MDU6SXNzdWU2MDYwMzMxMDQ=", "number": 12, "title": "If less than 500MB, show size in MB not GB", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-04-24T04:35:01Z", "updated_at": "2020-04-24T04:35:25Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Just saw this:\r\n```\r\nUploading 0.05 GB\r\n```", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 606028272, "node_id": "MDU6SXNzdWU2MDYwMjgyNzI=", "number": 10, "title": "Speed up hashing step using threads", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-24T04:20:08Z", "updated_at": "2020-04-24T04:32:35Z", "closed_at": "2020-04-24T04:32:35Z", "author_association": "MEMBER", "pull_request": null, "body": "This TODO from the code:\r\n\r\nhttps://github.com/dogsheep/photos-to-sqlite/blob/2e7f2c67cc18b02c75bb64992a05b0196e507252/photos_to_sqlite/cli.py#L82-L90", "repo": {"value": 256834907, "label": "dogsheep-photos"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/dogsheep-photos/issues/10/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}