html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/simonw/datasette/issues/1715#issuecomment-1112711115,https://api.github.com/repos/simonw/datasette/issues/1715,1112711115,IC_kwDOBm6k_c5CUp_L,9599,simonw,2022-04-28T22:26:56Z,2022-04-28T22:26:56Z,OWNER,"I'm not going to use `asyncinject` in this refactor - at least not until I really need it. My research in these issues has put me off the idea ( in favour of `asyncio.gather()` or even not trying for parallel execution at all):
- #1727","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1110265087,https://api.github.com/repos/simonw/datasette/issues/1715,1110265087,IC_kwDOBm6k_c5CLUz_,9599,simonw,2022-04-26T21:26:17Z,2022-04-26T21:26:17Z,OWNER,"Running facets and facet suggestions in parallel using `asyncio.gather()` turns out to be a lot less hassle than I had thought - maybe I don't need `asyncinject` for this at all?
```diff
if not nofacet:
- for facet in facet_instances:
- (
- instance_facet_results,
- instance_facets_timed_out,
- ) = await facet.facet_results()
+ # Run them in parallel
+ facet_awaitables = [facet.facet_results() for facet in facet_instances]
+ facet_awaitable_results = await asyncio.gather(*facet_awaitables)
+ for (
+ instance_facet_results,
+ instance_facets_timed_out,
+ ) in facet_awaitable_results:
for facet_info in instance_facet_results:
base_key = facet_info[""name""]
key = base_key
@@ -522,8 +540,10 @@ class TableView(DataView):
and not nofacet
and not nosuggest
):
- for facet in facet_instances:
- suggested_facets.extend(await facet.suggest())
+ # Run them in parallel
+ facet_suggest_awaitables = [facet.suggest() for facet in facet_instances]
+ for suggest_result in await asyncio.gather(*facet_suggest_awaitables):
+ suggested_facets.extend(suggest_result)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1110246593,https://api.github.com/repos/simonw/datasette/issues/1715,1110246593,IC_kwDOBm6k_c5CLQTB,9599,simonw,2022-04-26T21:03:56Z,2022-04-26T21:03:56Z,OWNER,"Well this is fun... I applied this change:
```diff
diff --git a/datasette/views/table.py b/datasette/views/table.py
index d66adb8..85f9e44 100644
--- a/datasette/views/table.py
+++ b/datasette/views/table.py
@@ -1,3 +1,4 @@
+import asyncio
import itertools
import json
@@ -5,6 +6,7 @@ import markupsafe
from datasette.plugins import pm
from datasette.database import QueryInterrupted
+from datasette import tracer
from datasette.utils import (
await_me_maybe,
CustomRow,
@@ -174,8 +176,11 @@ class TableView(DataView):
write=bool(canned_query.get(""write"")),
)
- is_view = bool(await db.get_view_definition(table_name))
- table_exists = bool(await db.table_exists(table_name))
+ with tracer.trace_child_tasks():
+ is_view, table_exists = map(bool, await asyncio.gather(
+ db.get_view_definition(table_name),
+ db.table_exists(table_name)
+ ))
# If table or view not found, return 404
if not is_view and not table_exists:
```
And now using https://datasette.io/plugins/datasette-pretty-traces I get this:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1110219185,https://api.github.com/repos/simonw/datasette/issues/1715,1110219185,IC_kwDOBm6k_c5CLJmx,9599,simonw,2022-04-26T20:28:40Z,2022-04-26T20:56:48Z,OWNER,"The refactor I did in #1719 pretty much clashes with all of the changes in https://github.com/simonw/datasette/commit/5053f1ea83194ecb0a5693ad5dada5b25bf0f7e6 so I'll probably need to start my `api-extras` branch again from scratch.
Using a new `tableview-asyncinject` branch.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1110239536,https://api.github.com/repos/simonw/datasette/issues/1715,1110239536,IC_kwDOBm6k_c5CLOkw,9599,simonw,2022-04-26T20:54:53Z,2022-04-26T20:54:53Z,OWNER,`pytest tests/test_table_*` runs the tests quickly.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1110238896,https://api.github.com/repos/simonw/datasette/issues/1715,1110238896,IC_kwDOBm6k_c5CLOaw,9599,simonw,2022-04-26T20:53:59Z,2022-04-26T20:53:59Z,OWNER,I'm going to rename `database` to `database_name` and `table` to `table_name` to avoid confusion with the `Database` object as opposed to the string name for the database.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1110229319,https://api.github.com/repos/simonw/datasette/issues/1715,1110229319,IC_kwDOBm6k_c5CLMFH,9599,simonw,2022-04-26T20:41:32Z,2022-04-26T20:44:38Z,OWNER,"This time I'm not going to bother with the `filter_args` thing - I'm going to just try to use `asyncinject` to execute some big high level things in parallel - facets, suggested facets, counts, the query - and then combine it with the `extras` mechanism I'm trying to introduce too.
Most importantly: I want that `extra_template()` function that adds more template context for the HTML to be executed as part of an `asyncinject` flow!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1108875068,https://api.github.com/repos/simonw/datasette/issues/1715,1108875068,IC_kwDOBm6k_c5CGBc8,9599,simonw,2022-04-25T18:03:13Z,2022-04-25T18:06:33Z,OWNER,"The `RowTableShared` class is making this a whole lot more complicated.
I'm going to split the `RowView` view out into an entirely separate `views/row.py` module.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1108877454,https://api.github.com/repos/simonw/datasette/issues/1715,1108877454,IC_kwDOBm6k_c5CGCCO,9599,simonw,2022-04-25T18:04:27Z,2022-04-25T18:04:27Z,OWNER,Pushed my WIP on this to the `api-extras` branch: 5053f1ea83194ecb0a5693ad5dada5b25bf0f7e6,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1106989581,https://api.github.com/repos/simonw/datasette/issues/1715,1106989581,IC_kwDOBm6k_c5B-1IN,9599,simonw,2022-04-22T23:03:29Z,2022-04-22T23:03:29Z,OWNER,I'm having second thoughts about injecting `request` - might be better to have the view function pull the relevant pieces out of the request before triggering the rest of the resolution.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1106947168,https://api.github.com/repos/simonw/datasette/issues/1715,1106947168,IC_kwDOBm6k_c5B-qxg,9599,simonw,2022-04-22T22:25:57Z,2022-04-22T22:26:06Z,OWNER,"```python
async def database(request: Request, datasette: Datasette) -> Database:
database_route = tilde_decode(request.url_vars[""database""])
try:
return datasette.get_database(route=database_route)
except KeyError:
raise NotFound(""Database not found: {}"".format(database_route))
async def table_name(request: Request) -> str:
return tilde_decode(request.url_vars[""table""])
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1106945876,https://api.github.com/repos/simonw/datasette/issues/1715,1106945876,IC_kwDOBm6k_c5B-qdU,9599,simonw,2022-04-22T22:24:29Z,2022-04-22T22:24:29Z,OWNER,"Looking at the start of `TableView.data()`:
https://github.com/simonw/datasette/blob/d57c347f35bcd8cff15f913da851b4b8eb030867/datasette/views/table.py#L333-L346
I'm going to resolve `table_name` and `database` from the URL - `table_name` will be a string, `database` will be the DB object returned by `datasette.get_database()`. Then those can be passed in separately too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,
https://github.com/simonw/datasette/issues/1715#issuecomment-1106908642,https://api.github.com/repos/simonw/datasette/issues/1715,1106908642,IC_kwDOBm6k_c5B-hXi,9599,simonw,2022-04-22T21:47:55Z,2022-04-22T21:47:55Z,OWNER,"I need a `asyncio.Registry` with functions registered to perform the role of the table view.
Something like this perhaps:
```python
def table_html_context(facet_results, query, datasette, rows):
return {...}
```
That then gets called like this:
```python
async def view(request):
registry = Registry(facet_results, query, datasette, rows)
context = await registry.resolve(table_html, request=request, datasette=datasette)
return Reponse.html(await datasette.render(""table.html"", context)
```
It's also interesting to start thinking about this from a Python client library point of view. If I'm writing code outside of the HTTP request cycle, what would it look like?
One thing I could do: break out is the code that turns a request into a list of pairs extracted from the request - this code here: https://github.com/simonw/datasette/blob/8338c66a57502ef27c3d7afb2527fbc0663b2570/datasette/views/table.py#L442-L449
I could turn that into a typed dependency injection function like this:
```python
def filter_args(request: Request) -> List[Tuple[str, str]]:
# Arguments that start with _ and don't contain a __ are
# special - things like ?_search= - and should not be
# treated as filters.
filter_args = []
for key in request.args:
if not (key.startswith(""_"") and ""__"" not in key):
for v in request.args.getlist(key):
filter_args.append((key, v))
return filter_args
```
Then I can either pass a `request` into a `.resolve()` call, or I can instead skip that function by passing:
```python
output = registry.resolve(table_context, filter_args=[(""foo"", ""bar"")])
```
I do need to think about where plugins get executed in all of this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1212823665,Refactor TableView to use asyncinject,