{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1112711115", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1112711115, "node_id": "IC_kwDOBm6k_c5CUp_L", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-28T22:26:56Z", "updated_at": "2022-04-28T22:26:56Z", "author_association": "OWNER", "body": "I'm not going to use `asyncinject` in this refactor - at least not until I really need it. My research in these issues has put me off the idea ( in favour of `asyncio.gather()` or even not trying for parallel execution at all):\r\n\r\n- #1727", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1110265087", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1110265087, "node_id": "IC_kwDOBm6k_c5CLUz_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-26T21:26:17Z", "updated_at": "2022-04-26T21:26:17Z", "author_association": "OWNER", "body": "Running facets and facet suggestions in parallel using `asyncio.gather()` turns out to be a lot less hassle than I had thought - maybe I don't need `asyncinject` for this at all?\r\n\r\n```diff\r\n if not nofacet:\r\n- for facet in facet_instances:\r\n- (\r\n- instance_facet_results,\r\n- instance_facets_timed_out,\r\n- ) = await facet.facet_results()\r\n+ # Run them in parallel\r\n+ facet_awaitables = [facet.facet_results() for facet in facet_instances]\r\n+ facet_awaitable_results = await asyncio.gather(*facet_awaitables)\r\n+ for (\r\n+ instance_facet_results,\r\n+ instance_facets_timed_out,\r\n+ ) in facet_awaitable_results:\r\n for facet_info in instance_facet_results:\r\n base_key = facet_info[\"name\"]\r\n key = base_key\r\n@@ -522,8 +540,10 @@ class TableView(DataView):\r\n and not nofacet\r\n and not nosuggest\r\n ):\r\n- for facet in facet_instances:\r\n- suggested_facets.extend(await facet.suggest())\r\n+ # Run them in parallel\r\n+ facet_suggest_awaitables = [facet.suggest() for facet in facet_instances]\r\n+ for suggest_result in await asyncio.gather(*facet_suggest_awaitables):\r\n+ suggested_facets.extend(suggest_result)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1110246593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1110246593, "node_id": "IC_kwDOBm6k_c5CLQTB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-26T21:03:56Z", "updated_at": "2022-04-26T21:03:56Z", "author_association": "OWNER", "body": "Well this is fun... I applied this change:\r\n\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex d66adb8..85f9e44 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -1,3 +1,4 @@\r\n+import asyncio\r\n import itertools\r\n import json\r\n \r\n@@ -5,6 +6,7 @@ import markupsafe\r\n \r\n from datasette.plugins import pm\r\n from datasette.database import QueryInterrupted\r\n+from datasette import tracer\r\n from datasette.utils import (\r\n await_me_maybe,\r\n CustomRow,\r\n@@ -174,8 +176,11 @@ class TableView(DataView):\r\n write=bool(canned_query.get(\"write\")),\r\n )\r\n \r\n- is_view = bool(await db.get_view_definition(table_name))\r\n- table_exists = bool(await db.table_exists(table_name))\r\n+ with tracer.trace_child_tasks():\r\n+ is_view, table_exists = map(bool, await asyncio.gather(\r\n+ db.get_view_definition(table_name),\r\n+ db.table_exists(table_name)\r\n+ ))\r\n \r\n # If table or view not found, return 404\r\n if not is_view and not table_exists:\r\n```\r\nAnd now using https://datasette.io/plugins/datasette-pretty-traces I get this:\r\n\r\n\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1110219185", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1110219185, "node_id": "IC_kwDOBm6k_c5CLJmx", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-26T20:28:40Z", "updated_at": "2022-04-26T20:56:48Z", "author_association": "OWNER", "body": "The refactor I did in #1719 pretty much clashes with all of the changes in https://github.com/simonw/datasette/commit/5053f1ea83194ecb0a5693ad5dada5b25bf0f7e6 so I'll probably need to start my `api-extras` branch again from scratch.\r\n\r\nUsing a new `tableview-asyncinject` branch.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1110239536", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1110239536, "node_id": "IC_kwDOBm6k_c5CLOkw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-26T20:54:53Z", "updated_at": "2022-04-26T20:54:53Z", "author_association": "OWNER", "body": "`pytest tests/test_table_*` runs the tests quickly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1110238896", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1110238896, "node_id": "IC_kwDOBm6k_c5CLOaw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-26T20:53:59Z", "updated_at": "2022-04-26T20:53:59Z", "author_association": "OWNER", "body": "I'm going to rename `database` to `database_name` and `table` to `table_name` to avoid confusion with the `Database` object as opposed to the string name for the database.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1110229319", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1110229319, "node_id": "IC_kwDOBm6k_c5CLMFH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-26T20:41:32Z", "updated_at": "2022-04-26T20:44:38Z", "author_association": "OWNER", "body": "This time I'm not going to bother with the `filter_args` thing - I'm going to just try to use `asyncinject` to execute some big high level things in parallel - facets, suggested facets, counts, the query - and then combine it with the `extras` mechanism I'm trying to introduce too.\r\n\r\nMost importantly: I want that `extra_template()` function that adds more template context for the HTML to be executed as part of an `asyncinject` flow!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1108875068", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1108875068, "node_id": "IC_kwDOBm6k_c5CGBc8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-25T18:03:13Z", "updated_at": "2022-04-25T18:06:33Z", "author_association": "OWNER", "body": "The `RowTableShared` class is making this a whole lot more complicated.\r\n\r\nI'm going to split the `RowView` view out into an entirely separate `views/row.py` module.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1108877454", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1108877454, "node_id": "IC_kwDOBm6k_c5CGCCO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-25T18:04:27Z", "updated_at": "2022-04-25T18:04:27Z", "author_association": "OWNER", "body": "Pushed my WIP on this to the `api-extras` branch: 5053f1ea83194ecb0a5693ad5dada5b25bf0f7e6", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1106989581", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1106989581, "node_id": "IC_kwDOBm6k_c5B-1IN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-22T23:03:29Z", "updated_at": "2022-04-22T23:03:29Z", "author_association": "OWNER", "body": "I'm having second thoughts about injecting `request` - might be better to have the view function pull the relevant pieces out of the request before triggering the rest of the resolution.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1106947168", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1106947168, "node_id": "IC_kwDOBm6k_c5B-qxg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-22T22:25:57Z", "updated_at": "2022-04-22T22:26:06Z", "author_association": "OWNER", "body": "```python\r\nasync def database(request: Request, datasette: Datasette) -> Database:\r\n database_route = tilde_decode(request.url_vars[\"database\"])\r\n try:\r\n return datasette.get_database(route=database_route)\r\n except KeyError:\r\n raise NotFound(\"Database not found: {}\".format(database_route))\r\n\r\nasync def table_name(request: Request) -> str:\r\n return tilde_decode(request.url_vars[\"table\"])\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1106945876", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1106945876, "node_id": "IC_kwDOBm6k_c5B-qdU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-22T22:24:29Z", "updated_at": "2022-04-22T22:24:29Z", "author_association": "OWNER", "body": "Looking at the start of `TableView.data()`:\r\n\r\nhttps://github.com/simonw/datasette/blob/d57c347f35bcd8cff15f913da851b4b8eb030867/datasette/views/table.py#L333-L346\r\n\r\nI'm going to resolve `table_name` and `database` from the URL - `table_name` will be a string, `database` will be the DB object returned by `datasette.get_database()`. Then those can be passed in separately too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1715#issuecomment-1106908642", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1715", "id": 1106908642, "node_id": "IC_kwDOBm6k_c5B-hXi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-04-22T21:47:55Z", "updated_at": "2022-04-22T21:47:55Z", "author_association": "OWNER", "body": "I need a `asyncio.Registry` with functions registered to perform the role of the table view.\r\n\r\nSomething like this perhaps:\r\n```python\r\ndef table_html_context(facet_results, query, datasette, rows):\r\n return {...}\r\n```\r\nThat then gets called like this:\r\n```python\r\nasync def view(request):\r\n registry = Registry(facet_results, query, datasette, rows)\r\n context = await registry.resolve(table_html, request=request, datasette=datasette)\r\n return Reponse.html(await datasette.render(\"table.html\", context)\r\n```\r\nIt's also interesting to start thinking about this from a Python client library point of view. If I'm writing code outside of the HTTP request cycle, what would it look like?\r\n\r\nOne thing I could do: break out is the code that turns a request into a list of pairs extracted from the request - this code here: https://github.com/simonw/datasette/blob/8338c66a57502ef27c3d7afb2527fbc0663b2570/datasette/views/table.py#L442-L449\r\n\r\nI could turn that into a typed dependency injection function like this:\r\n\r\n```python\r\ndef filter_args(request: Request) -> List[Tuple[str, str]]:\r\n # Arguments that start with _ and don't contain a __ are\r\n # special - things like ?_search= - and should not be\r\n # treated as filters.\r\n filter_args = []\r\n for key in request.args:\r\n if not (key.startswith(\"_\") and \"__\" not in key):\r\n for v in request.args.getlist(key):\r\n filter_args.append((key, v))\r\n return filter_args\r\n```\r\nThen I can either pass a `request` into a `.resolve()` call, or I can instead skip that function by passing:\r\n\r\n```python\r\noutput = registry.resolve(table_context, filter_args=[(\"foo\", \"bar\")])\r\n```\r\nI do need to think about where plugins get executed in all of this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1212823665, "label": "Refactor TableView to use asyncinject"}, "performed_via_github_app": null}