{"html_url": "https://github.com/simonw/datasette/pull/518#issuecomment-504798977", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/518", "id": 504798977, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc5ODk3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T23:52:38Z", "updated_at": "2019-06-23T23:52:38Z", "author_association": "OWNER", "body": "Last thing is to replace `sanic.response`:\r\n\r\n* `response.text(\"\")`\r\n* `response.html()`\r\n* `response.redirect(path)`\r\n* `response.HTTPResponse`\r\n\r\nImplementations here: https://github.com/huge-success/sanic/blob/0.7.0/sanic/response.py#L175-L285", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459587155, "label": "Port Datasette from Sanic to ASGI + Uvicorn"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/518#issuecomment-504795648", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/518", "id": 504795648, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc5NTY0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T23:07:06Z", "updated_at": "2019-06-23T23:07:06Z", "author_association": "OWNER", "body": "For the request object.... what are the fields of it I actually use?\r\n\r\n* `request.url`\r\n* `request.query_string`\r\n* `request.path`\r\n* `request.method`\r\n* `request.args`\r\n* `request.raw_args`\r\n\r\nALL of those are things that can be derived from the `scope` - so I think my new `Request` class (in `utils/asgi.py`) is just going to be a wrapper around a `scope`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459587155, "label": "Port Datasette from Sanic to ASGI + Uvicorn"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/498#issuecomment-504793988", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/498", "id": 504793988, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc5Mzk4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T22:40:25Z", "updated_at": "2019-06-23T22:41:25Z", "author_association": "OWNER", "body": "You can use a database query against `sqlite_master` to find all the FTS tables:\r\n\r\nhttp://sf-trees.datasettes.com/trees-b64f0cb?sql=select+name+from+sqlite_master+where+name+like+%22%25_fts%22+and+type+%3D+%22table%22\r\n\r\n`select name from sqlite_master where name like \"%_fts\" and type = \"table\"`\r\n\r\nYou could then construct a crafty UNION query to get results back from all of those tables at once:\r\n\r\nhttp://sf-trees.datasettes.com/trees-b64f0cb?sql=select+%27PlantType_value_fts%27%2C+rowid%2C+value+from+PlantType_value_fts+where+PlantType_value_fts+match+%3Asearch%0D%0A++union%0D%0Aselect+%27qCareAssistant_value_fts%27%2C+rowid%2C+value+from+qCareAssistant_value_fts+where+qCareAssistant_value_fts+match+%3Asearch%0D%0A++union%0D%0Aselect+%27qCaretaker_value_fts%27%2C+rowid%2C+value+from+qCaretaker_value_fts+where+qCaretaker_value_fts+match+%3Asearch%0D%0A++union%0D%0Aselect+%27qLegalStatus_value_fts%27%2C+rowid%2C+value+from+qLegalStatus_value_fts+where+qLegalStatus_value_fts+match+%3Asearch%0D%0A++union%0D%0Aselect+%27qSiteInfo_value_fts%27%2C+rowid%2C+value+from+qSiteInfo_value_fts+where+qSiteInfo_value_fts+match+%3Asearch%0D%0A++union%0D%0Aselect+%27qSpecies_value_fts%27%2C+rowid%2C+value+from+qSpecies_value_fts+where+qSpecies_value_fts+match+%3Asearch&search=fi%2A\r\n\r\n(I'm searching for `fi*` here to demonstrate wildcards)\r\n\r\nThe problem, as discussed earlier, is relevance: there's no way to compare the scores you're getting across different tables, so you won't be able to order by anything.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 451513541, "label": "Full text search of all tables at once?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/514#issuecomment-504793379", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/514", "id": 504793379, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc5MzM3OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T22:31:29Z", "updated_at": "2019-06-23T22:31:48Z", "author_association": "OWNER", "body": "I suggest trying a full path in `ExecStart` like this:\r\n\r\n`ExecStart=/home/chris/Env/datasette/bin/datasette serve -h 0.0.0.0 /home/chris/digital-library/databases/*.db --cors --metadata /home/chris/digital-library/metadata.json`\r\n\r\nThat should eliminate the chance of some kind of path confusion.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459397625, "label": "Documentation with recommendations on running Datasette in production without using Docker"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/518#issuecomment-504791053", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/518", "id": 504791053, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc5MTA1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T22:00:56Z", "updated_at": "2019-06-23T22:00:56Z", "author_association": "OWNER", "body": "The `InvalidUsage` exception is thrown by Sanic when it gets an unhandled request - usually a HEAD. It was added in efbb4e83374a2c795e436c72fa79f70da72309b8\r\n\r\nI'm going to replace it with specific handling for HEAD requests plus a unit test.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459587155, "label": "Port Datasette from Sanic to ASGI + Uvicorn"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/518#issuecomment-504790825", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/518", "id": 504790825, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc5MDgyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T21:57:58Z", "updated_at": "2019-06-23T21:57:58Z", "author_association": "OWNER", "body": "The big one: **Replace Sanic request and response objects with my own classes, so I can remove Sanic dependency**", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459587155, "label": "Port Datasette from Sanic to ASGI + Uvicorn"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/514#issuecomment-504789231", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/514", "id": 504789231, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc4OTIzMQ==", "user": {"value": 7936571, "label": "chrismp"}, "created_at": "2019-06-23T21:35:33Z", "updated_at": "2019-06-23T21:35:33Z", "author_association": "NONE", "body": "@russss \r\n\r\nThanks, just one more thing.\r\n\r\nI edited `datasette.service`:\r\n\r\n```\r\n[Unit]\r\nDescription=Datasette\r\nAfter=network.target\r\n\r\n[Service]\r\nType=simple\r\nUser=chris\r\nWorkingDirectory=/home/chris/digital-library\r\nExecStart=/home/chris/Env/datasette/bin/datasette serve -h 0.0.0.0 databases/*.db --cors --metadata metadata.json\r\nRestart=on-failure\r\n\r\n[Install]\r\nWantedBy=multi-user.target\r\n```\r\n\r\nThen ran:\r\n\r\n```\r\n$ sudo systemctl daemon-reload\r\n$ sudo systemctl enable datasette\r\n$ sudo systemctl start datasette\r\n```\r\n\r\nBut the logs from `journalctl` show this datasette error:\r\n\r\n```\r\nJun 23 23:31:41 ns331247 datasette[1771]: Error: Invalid value for \"[FILES]...\": Path \"databases/*.db\" does not exist.\r\nJun 23 23:31:44 ns331247 datasette[1778]: Usage: datasette serve [OPTIONS] [FILES]...\r\nJun 23 23:31:44 ns331247 datasette[1778]: Try \"datasette serve --help\" for help.\r\n```\r\n\r\nBut the `databases` directory does exist in the directory specified by `WorkingDirectory`. Is this a datasette problem or did I write something incorrectly in the `.service` file?\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459397625, "label": "Documentation with recommendations on running Datasette in production without using Docker"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/514#issuecomment-504686266", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/514", "id": 504686266, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDY4NjI2Ng==", "user": {"value": 7936571, "label": "chrismp"}, "created_at": "2019-06-22T17:58:50Z", "updated_at": "2019-06-23T21:21:57Z", "author_association": "NONE", "body": "@russss \r\n\r\nActually, here's what I've got in `/etc/systemd/system/datasette.service`\r\n\r\n```\r\n[Unit]\r\nDescription=Datasette\r\nAfter=network.target\r\n\r\n[Service]\r\nType=simple\r\nUser=chris\r\nWorkingDirectory=/home/chris/digital-library\r\nExecStart=/home/chris/Env/datasette/lib/python3.7/site-packages/datasette serve -h 0.0.0.0 databases/*.db --cors --metadata metadata.json\r\nRestart=on-failure\r\n\r\n[Install]\r\nWantedBy=multi-user.target\r\n```\r\n\r\nI ran: \r\n```\r\n$ sudo systemctl daemon-reload\r\n$ sudo systemctl enable datasette\r\n$ sudo systemctl start datasette\r\n```\r\nThen I ran:\r\n`$ journalctl -u datasette -f`\r\n\r\nGot this message.\r\n\r\n```\r\nHint: You are currently not seeing messages from other users and the system.\r\n Users in groups 'adm', 'systemd-journal', 'wheel' can see all messages.\r\n Pass -q to turn off this notice.\r\n-- Logs begin at Thu 2019-06-20 00:05:23 CEST. --\r\nJun 22 19:55:57 ns331247 systemd[16176]: datasette.service: Failed to execute command: Permission denied\r\nJun 22 19:55:57 ns331247 systemd[16176]: datasette.service: Failed at step EXEC spawning /home/chris/Env/datasette/lib/python3.7/site-packages/datasette: Permission denied\r\nJun 22 19:55:57 ns331247 systemd[16184]: datasette.service: Failed to execute command: Permission denied\r\nJun 22 19:55:57 ns331247 systemd[16184]: datasette.service: Failed at step EXEC spawning /home/chris/Env/datasette/lib/python3.7/site-packages/datasette: Permission denied\r\nJun 22 19:55:58 ns331247 systemd[16186]: datasette.service: Failed to execute command: Permission denied\r\nJun 22 19:55:58 ns331247 systemd[16186]: datasette.service: Failed at step EXEC spawning /home/chris/Env/datasette/lib/python3.7/site-packages/datasette: Permission denied\r\nJun 22 19:55:58 ns331247 systemd[16190]: datasette.service: Failed to execute command: Permission denied\r\nJun 22 19:55:58 ns331247 systemd[16190]: datasette.service: Failed at step EXEC spawning /home/chris/Env/datasette/lib/python3.7/site-packages/datasette: Permission denied\r\nJun 22 19:55:58 ns331247 systemd[16191]: datasette.service: Failed to execute command: Permission denied\r\nJun 22 19:55:58 ns331247 systemd[16191]: datasette.service: Failed at step EXEC spawning /home/chris/Env/datasette/lib/python3.7/site-packages/datasette: Permission denied\r\n```\r\nWhen I go to the address for my server, I am met with the standard \"Welcome to nginx\" message:\r\n\r\n```\r\nWelcome to nginx!\r\nIf you see this page, the nginx web server is successfully installed and working. Further configuration is required.\r\n\r\nFor online documentation and support please refer to nginx.org.\r\nCommercial support is available at nginx.com.\r\n\r\nThank you for using nginx.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459397625, "label": "Documentation with recommendations on running Datasette in production without using Docker"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/498#issuecomment-504785662", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/498", "id": 504785662, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc4NTY2Mg==", "user": {"value": 7936571, "label": "chrismp"}, "created_at": "2019-06-23T20:47:37Z", "updated_at": "2019-06-23T20:47:37Z", "author_association": "NONE", "body": "Very cool, thank you.\r\n\r\nUsing http://search-24ways.herokuapp.com as an example, let's say I want to search all FTS columns in all tables in all databases for the word \"web.\" \r\n\r\n[Here's a link](http://search-24ways.herokuapp.com/24ways-f8f455f?sql=select+count%28*%29from+articles+where+rowid+in+%28select+rowid+from+articles_fts+where+articles_fts+match+%3Asearch%29&search=web) to the query I'd need to run to search \"web\" on FTS columns in `articles` table of the `24ways` database. \r\n\r\nAnd [here's a link](http://search-24ways.herokuapp.com/24ways-f8f455f.json?sql=select+count%28*%29from+articles+where+rowid+in+%28select+rowid+from+articles_fts+where+articles_fts+match+%3Asearch%29&search=web) to the JSON version of the above result. I'd like to get the JSON result of that query for each FTS table of each database in my datasette project. \r\n\r\nIs it possible in Javascript to automate the construction of query URLs like the one I linked, but for every FTS table in my datasette project?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 451513541, "label": "Full text search of all tables at once?"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/518#issuecomment-504782618", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/518", "id": 504782618, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc4MjYxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T20:05:44Z", "updated_at": "2019-06-23T20:05:59Z", "author_association": "OWNER", "body": "**Replacement for @app.listener(\"before_server_start\")** - this is what the [ASGI lifespan protocol](https://asgi.readthedocs.io/en/latest/specs/lifespan.html) is for.\r\n\r\nI know Uvicorn supports this because it keeps saying `ASGI 'lifespan' protocol appears unsupported` on the console.\r\n\r\nI think the solution here will be to introduce another ASGI wrapper class similar to `AsgiTracer`. I'll model this on the example in the ASGI lifespan spec.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459587155, "label": "Port Datasette from Sanic to ASGI + Uvicorn"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/520#issuecomment-504772599", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/520", "id": 504772599, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc3MjU5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T17:44:36Z", "updated_at": "2019-06-23T17:44:36Z", "author_association": "OWNER", "body": "These plugins will need access to configuration and the ability to execute SQL - which means we need to make the `datasette` instance available to them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459598080, "label": "asgi_wrapper plugin hook"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/516#issuecomment-504768147", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/516", "id": 504768147, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc2ODE0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T16:43:23Z", "updated_at": "2019-06-23T16:43:23Z", "author_association": "OWNER", "body": "The Starlette lint and test scripts do this, and also apply autoflake to remove any unnecessary imports: https://github.com/encode/starlette/tree/8c8cc2ec0a5cb834a9a15b871ae8b480503abb67/scripts", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 459509126, "label": "Enforce import sort order with isort"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/518#issuecomment-504765738", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/518", "id": 504765738, "node_id": "MDEyOklzc3VlQ29tbWVudDUwNDc2NTczOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-06-23T16:11:49Z", "updated_at": "2019-06-23T16:20:44Z", "author_association": "OWNER", "body": "OK, for **Get ?_trace=1 working again**. The old code lives in two places:\r\n\r\nhttps://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/app.py#L546-L560\r\n\r\nAnd then:\r\n\r\nhttps://github.com/simonw/datasette/blob/35429f90894321eda7f2db31b9ea7976f31f73ac/datasette/app.py#L653-L672\r\n\r\nSo it's stashing something on the request to tell the rest of the code it should be tracing, then using that collected data from the request to add information to the final body.\r\n\r\nOne possible shape for the replacement is a new ASGI middleware that wraps everything else. We don't have a mutable request object here though, so we will need to untangle this entirely from the request object.\r\n\r\nAlso tricky is that in ASGI land we handle streams - we don't usually wait around for the entire response body to be compiled for us. This means the code that modifies the response (adding to the JSON or appending inside the `