{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-1073037939", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 1073037939, "node_id": "IC_kwDOBm6k_c4_9UJz", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-19T16:19:30Z", "updated_at": "2022-03-19T16:19:30Z", "author_association": "OWNER", "body": "On revisiting https://gist.github.com/simonw/281eac9c73b062c3469607ad86470eb2 a few months later I'm having second thoughts about using `@inject` on the `main()` method.\r\n\r\nBut I still like the pattern as a way to resolve more complex cases like \"to generate GeoJSON of the expanded view with labels, the label expansion code needs to run once at some before the GeoJSON formatting code does\".\r\n\r\nSo I'm going to stick with it a tiny bit longer, but maybe try to make it a lot more explicit when it's going to happen rather than having the main view methods themselves also use async DI.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-1001699559", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 1001699559, "node_id": "IC_kwDOBm6k_c47tLjn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-27T18:53:04Z", "updated_at": "2021-12-27T18:53:04Z", "author_association": "OWNER", "body": "I'm going to see if I can come up with the simplest possible version of this pattern for the `/-/metadata` and `/-/metadata.json` page, then try it for the database query page, before tackling the much more complex table page.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-973678931", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 973678931, "node_id": "IC_kwDOBm6k_c46CSlT", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-19T02:51:17Z", "updated_at": "2021-11-19T02:51:17Z", "author_association": "OWNER", "body": "OK, I managed to get a table to render! Here's the code I used - I had to copy a LOT of stuff. https://gist.github.com/simonw/281eac9c73b062c3469607ad86470eb2\r\n\r\nI'm going to move this work into a new, separate issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-973635157", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 973635157, "node_id": "IC_kwDOBm6k_c46CH5V", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-19T01:07:08Z", "updated_at": "2021-11-19T01:07:08Z", "author_association": "OWNER", "body": "This exercise is proving so useful in getting my head around how the enormous and complex `TableView` class works again.\r\n\r\nHere's where I've got to now - I'm systematically working through the variables that are returned for HTML and for JSON copying across code to get it to work:\r\n\r\n```python\r\nfrom datasette.database import QueryInterrupted\r\nfrom datasette.utils import escape_sqlite\r\nfrom datasette.utils.asgi import Response, NotFound, Forbidden\r\nfrom datasette.views.base import DatasetteError\r\nfrom datasette import hookimpl\r\nfrom asyncinject import AsyncInject, inject\r\nfrom pprint import pformat\r\n\r\n\r\nclass Table(AsyncInject):\r\n    @inject\r\n    async def database(self, request, datasette):\r\n        # TODO: all that nasty hash resolving stuff can go here\r\n        db_name = request.url_vars[\"db_name\"]\r\n        try:\r\n            db = datasette.databases[db_name]\r\n        except KeyError:\r\n            raise NotFound(f\"Database '{db_name}' does not exist\")\r\n        return db\r\n\r\n    @inject\r\n    async def table_and_format(self, request, database, datasette):\r\n        table_and_format = request.url_vars[\"table_and_format\"]\r\n        # TODO: be a lot smarter here\r\n        if \".\" in table_and_format:\r\n            return table_and_format.split(\".\", 2)\r\n        else:\r\n            return table_and_format, \"html\"\r\n\r\n    @inject\r\n    async def main(self, request, database, table_and_format, datasette):\r\n        # TODO: if this is actually a canned query, dispatch to it\r\n\r\n        table, format = table_and_format\r\n\r\n        is_view = bool(await database.get_view_definition(table))\r\n        table_exists = bool(await database.table_exists(table))\r\n        if not is_view and not table_exists:\r\n            raise NotFound(f\"Table not found: {table}\")\r\n\r\n        await check_permissions(\r\n            datasette,\r\n            request,\r\n            [\r\n                (\"view-table\", (database.name, table)),\r\n                (\"view-database\", database.name),\r\n                \"view-instance\",\r\n            ],\r\n        )\r\n\r\n        private = not await datasette.permission_allowed(\r\n            None, \"view-table\", (database.name, table), default=True\r\n        )\r\n\r\n        pks = await database.primary_keys(table)\r\n        table_columns = await database.table_columns(table)\r\n\r\n        specified_columns = await columns_to_select(datasette, database, table, request)\r\n        select_specified_columns = \", \".join(\r\n            escape_sqlite(t) for t in specified_columns\r\n        )\r\n        select_all_columns = \", \".join(escape_sqlite(t) for t in table_columns)\r\n\r\n        use_rowid = not pks and not is_view\r\n        if use_rowid:\r\n            select_specified_columns = f\"rowid, {select_specified_columns}\"\r\n            select_all_columns = f\"rowid, {select_all_columns}\"\r\n            order_by = \"rowid\"\r\n            order_by_pks = \"rowid\"\r\n        else:\r\n            order_by_pks = \", \".join([escape_sqlite(pk) for pk in pks])\r\n            order_by = order_by_pks\r\n\r\n        if is_view:\r\n            order_by = \"\"\r\n\r\n        nocount = request.args.get(\"_nocount\")\r\n        nofacet = request.args.get(\"_nofacet\")\r\n\r\n        if request.args.get(\"_shape\") in (\"array\", \"object\"):\r\n            nocount = True\r\n            nofacet = True\r\n\r\n        # Next, a TON of SQL to build where_params and filters and suchlike\r\n        # skipping that and jumping straight to...\r\n        where_clauses = []\r\n        where_clause = \"\"\r\n        if where_clauses:\r\n            where_clause = f\"where {' and '.join(where_clauses)} \"\r\n\r\n        from_sql = \"from {table_name} {where}\".format(\r\n            table_name=escape_sqlite(table),\r\n            where=(\"where {} \".format(\" and \".join(where_clauses)))\r\n            if where_clauses\r\n            else \"\",\r\n        )\r\n        from_sql_params ={}\r\n        params = {}\r\n        count_sql = f\"select count(*) {from_sql}\"\r\n        sql_no_order_no_limit = (\r\n            \"select {select_all_columns} from {table_name} {where}\".format(\r\n                select_all_columns=select_all_columns,\r\n                table_name=escape_sqlite(table),\r\n                where=where_clause,\r\n            )\r\n        )\r\n\r\n        page_size = 100\r\n        offset = \" offset 0\"\r\n\r\n        sql = \"select {select_specified_columns} from {table_name} {where}{order_by} limit {page_size}{offset}\".format(\r\n            select_specified_columns=select_specified_columns,\r\n            table_name=escape_sqlite(table),\r\n            where=where_clause,\r\n            order_by=order_by,\r\n            page_size=page_size + 1,\r\n            offset=offset,\r\n        )\r\n\r\n        # Fetch rows\r\n        results = await database.execute(sql, params, truncate=True)\r\n        columns = [r[0] for r in results.description]\r\n        rows = list(results.rows)\r\n\r\n        # Fetch count\r\n        filtered_table_rows_count = None\r\n        if count_sql:\r\n            try:\r\n                count_rows = list(await database.execute(count_sql, from_sql_params))\r\n                filtered_table_rows_count = count_rows[0][0]\r\n            except QueryInterrupted:\r\n                pass\r\n\r\n\r\n        vars = {\r\n            \"json\": {\r\n                # THIS STUFF is from the regular JSON\r\n                \"database\": database.name,\r\n                \"table\": table,\r\n                \"is_view\": is_view,\r\n                # \"human_description_en\": human_description_en,\r\n                \"rows\": rows[:page_size],\r\n                \"truncated\": results.truncated,\r\n                \"filtered_table_rows_count\": filtered_table_rows_count,\r\n                # \"expanded_columns\": expanded_columns,\r\n                # \"expandable_columns\": expandable_columns,\r\n                \"columns\": columns,\r\n                \"primary_keys\": pks,\r\n                # \"units\": units,\r\n                \"query\": {\"sql\": sql, \"params\": params},\r\n                # \"facet_results\": facet_results,\r\n                # \"suggested_facets\": suggested_facets,\r\n                # \"next\": next_value and str(next_value) or None,\r\n                # \"next_url\": next_url,\r\n                \"private\": private,\r\n                \"allow_execute_sql\": await datasette.permission_allowed(\r\n                    request.actor, \"execute-sql\", database, default=True\r\n                ),\r\n            },\r\n            \"html\": {\r\n                # ... this is the HTML special stuff\r\n                # \"table_actions\": table_actions,\r\n                # \"supports_search\": bool(fts_table),\r\n                # \"search\": search or \"\",\r\n                \"use_rowid\": use_rowid,\r\n                # \"filters\": filters,\r\n                # \"display_columns\": display_columns,\r\n                # \"filter_columns\": filter_columns,\r\n                # \"display_rows\": display_rows,\r\n                # \"facets_timed_out\": facets_timed_out,\r\n                # \"sorted_facet_results\": sorted(\r\n                #     facet_results.values(),\r\n                #     key=lambda f: (len(f[\"results\"]), f[\"name\"]),\r\n                #     reverse=True,\r\n                # ),\r\n                # \"show_facet_counts\": special_args.get(\"_facet_size\") == \"max\",\r\n                # \"extra_wheres_for_ui\": extra_wheres_for_ui,\r\n                # \"form_hidden_args\": form_hidden_args,\r\n                # \"is_sortable\": any(c[\"sortable\"] for c in display_columns),\r\n                # \"path_with_replaced_args\": path_with_replaced_args,\r\n                # \"path_with_removed_args\": path_with_removed_args,\r\n                # \"append_querystring\": append_querystring,\r\n                \"request\": request,\r\n                # \"sort\": sort,\r\n                # \"sort_desc\": sort_desc,\r\n                \"disable_sort\": is_view,\r\n                # \"custom_table_templates\": [\r\n                #     f\"_table-{to_css_class(database)}-{to_css_class(table)}.html\",\r\n                #     f\"_table-table-{to_css_class(database)}-{to_css_class(table)}.html\",\r\n                #     \"_table.html\",\r\n                # ],\r\n                # \"metadata\": metadata,\r\n                # \"view_definition\": await db.get_view_definition(table),\r\n                # \"table_definition\": await db.get_table_definition(table),\r\n            },\r\n        }\r\n\r\n        # I'm just trying to get HTML to work for the moment\r\n        if format == \"json\":\r\n            return Response.json(dict(vars, locals=locals()), default=repr)\r\n        else:\r\n            return Response.html(repr(vars[\"html\"]))\r\n\r\n    async def view(self, request, datasette):\r\n        return await self.main(request=request, datasette=datasette)\r\n\r\n\r\n@hookimpl\r\ndef register_routes():\r\n    return [\r\n        (r\"/t/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?$)\", Table().view),\r\n    ]\r\n\r\n\r\nasync def check_permissions(datasette, request, permissions):\r\n    \"\"\"permissions is a list of (action, resource) tuples or 'action' strings\"\"\"\r\n    for permission in permissions:\r\n        if isinstance(permission, str):\r\n            action = permission\r\n            resource = None\r\n        elif isinstance(permission, (tuple, list)) and len(permission) == 2:\r\n            action, resource = permission\r\n        else:\r\n            assert (\r\n                False\r\n            ), \"permission should be string or tuple of two items: {}\".format(\r\n                repr(permission)\r\n            )\r\n        ok = await datasette.permission_allowed(\r\n            request.actor,\r\n            action,\r\n            resource=resource,\r\n            default=None,\r\n        )\r\n        if ok is not None:\r\n            if ok:\r\n                return\r\n            else:\r\n                raise Forbidden(action)\r\n\r\n\r\nasync def columns_to_select(datasette, database, table, request):\r\n    table_columns = await database.table_columns(table)\r\n    pks = await database.primary_keys(table)\r\n    columns = list(table_columns)\r\n    if \"_col\" in request.args:\r\n        columns = list(pks)\r\n        _cols = request.args.getlist(\"_col\")\r\n        bad_columns = [column for column in _cols if column not in table_columns]\r\n        if bad_columns:\r\n            raise DatasetteError(\r\n                \"_col={} - invalid columns\".format(\", \".join(bad_columns)),\r\n                status=400,\r\n            )\r\n        # De-duplicate maintaining order:\r\n        columns.extend(dict.fromkeys(_cols))\r\n    if \"_nocol\" in request.args:\r\n        # Return all columns EXCEPT these\r\n        bad_columns = [\r\n            column\r\n            for column in request.args.getlist(\"_nocol\")\r\n            if (column not in table_columns) or (column in pks)\r\n        ]\r\n        if bad_columns:\r\n            raise DatasetteError(\r\n                \"_nocol={} - invalid columns\".format(\", \".join(bad_columns)),\r\n                status=400,\r\n            )\r\n        tmp_columns = [\r\n            column for column in columns if column not in request.args.getlist(\"_nocol\")\r\n        ]\r\n        columns = tmp_columns\r\n    return columns\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-973568285", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 973568285, "node_id": "IC_kwDOBm6k_c46B3kd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-19T00:29:20Z", "updated_at": "2021-11-19T00:29:20Z", "author_association": "OWNER", "body": "This is working!\r\n```python\r\nfrom datasette.utils.asgi import Response\r\nfrom datasette import hookimpl\r\nimport html\r\nfrom asyncinject import AsyncInject, inject\r\n\r\n\r\nclass Table(AsyncInject):\r\n    @inject\r\n    async def database(self, request):\r\n        return request.url_vars[\"db_name\"]\r\n\r\n    @inject\r\n    async def main(self, request, database):\r\n        return Response.html(\"Database: {}\".format(\r\n            html.escape(database)\r\n        ))\r\n\r\n    async def view(self, request):\r\n        return await self.main(request=request)\r\n\r\n\r\n@hookimpl\r\ndef register_routes():\r\n    return [\r\n        (r\"/t/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?$)\", Table().view),\r\n    ]\r\n```\r\nThis project will definitely show me if I actually like the `asyncinject` patterns or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-973564260", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 973564260, "node_id": "IC_kwDOBm6k_c46B2lk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-19T00:27:06Z", "updated_at": "2021-11-19T00:27:06Z", "author_association": "OWNER", "body": "Problem: the fancy `asyncinject` stuff inteferes with the fancy Datasette thing that introspects view functions to look for what parameters they take:\r\n```python\r\nclass Table(asyncinject.AsyncInjectAll):\r\n    async def view(self, request):\r\n        return Response.html(\"Hello from {}\".format(\r\n            html.escape(repr(request.url_vars))\r\n        ))\r\n\r\n\r\n@hookimpl\r\ndef register_routes():\r\n    return [\r\n        (r\"/t/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?$)\", Table().view),\r\n    ]\r\n```\r\nThis failed with error: \"Table.view() takes 1 positional argument but 2 were given\"\r\n\r\nSo I'm going to use `AsyncInject` and have the `view` function NOT use the `@inject` decorator.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-973554024", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 973554024, "node_id": "IC_kwDOBm6k_c46B0Fo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-19T00:21:20Z", "updated_at": "2021-11-19T00:21:20Z", "author_association": "OWNER", "body": "That's annoying: it looks like plugins can't use `register_routes()` to over-ride default routes within Datasette itself. This didn't work:\r\n```python\r\nfrom datasette.utils.asgi import Response\r\nfrom datasette import hookimpl\r\nimport html\r\n\r\n\r\nasync def table(request):\r\n    return Response.html(\"Hello from {}\".format(\r\n        html.escape(repr(request.url_vars))\r\n    ))\r\n\r\n\r\n@hookimpl\r\ndef register_routes():\r\n    return [\r\n        (r\"/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?$)\", table),\r\n    ]\r\n```\r\nI'll use a `/t/` prefix for the moment, but this is probably something I'll fix in Datasette itself later.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-973542284", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 973542284, "node_id": "IC_kwDOBm6k_c46BxOM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-19T00:16:44Z", "updated_at": "2021-11-19T00:16:44Z", "author_association": "OWNER", "body": "```\r\nDevelopment % cookiecutter gh:simonw/datasette-plugin\r\nYou've downloaded /Users/simon/.cookiecutters/datasette-plugin before. Is it okay to delete and re-download it? [yes]: yes\r\nplugin_name []: table-new\r\ndescription []: New implementation of TableView, see https://github.com/simonw/datasette/issues/878\r\nhyphenated [table-new]: \r\nunderscored [table_new]: \r\ngithub_username []: simonw\r\nauthor_name []: Simon Willison\r\ninclude_static_directory []:  \r\ninclude_templates_directory []: \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-973527870", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 973527870, "node_id": "IC_kwDOBm6k_c46Bts-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-19T00:13:43Z", "updated_at": "2021-11-19T00:13:43Z", "author_association": "OWNER", "body": "New plan: I'm going to build a brand new implementation of `TableView` starting out as a plugin, using the `register_routes()` plugin hook.\r\n\r\nIt will reuse the existing HTML template but will be a completely new Python implementation, based on `asyncinject`.\r\n\r\nI'm going to start by just getting the table to show up on the page - then I'll add faceting, suggested facets, filters and so-on.\r\n\r\nBonus: I'm going to see if I can get it to work for arbitrary SQL queries too (stretch goal).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-971209475", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 971209475, "node_id": "IC_kwDOBm6k_c4543sD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-17T05:41:42Z", "updated_at": "2021-11-17T05:41:42Z", "author_association": "OWNER", "body": "I'm going to build a brand new implementation of the `TableView` class that doesn't subclass `BaseView` at all, instead using `asyncinject`. If I'm lucky that will clean up the grungiest part of the codebase.\r\n\r\nI can maybe even run the tests against old `TableView` and `TableView2` to check that they behave the same.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-971057553", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 971057553, "node_id": "IC_kwDOBm6k_c454SmR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-17T01:40:45Z", "updated_at": "2021-11-17T01:40:45Z", "author_association": "OWNER", "body": "I shipped that code as a new library, `asyncinject`: https://pypi.org/project/asyncinject/ - I'll open a new PR to attempt to refactor `TableView` to use it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970712713", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970712713, "node_id": "IC_kwDOBm6k_c452-aJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T21:54:33Z", "updated_at": "2021-11-16T21:54:33Z", "author_association": "OWNER", "body": "I'm going to continue working on this in a PR.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970705738", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970705738, "node_id": "IC_kwDOBm6k_c4528tK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T21:44:31Z", "updated_at": "2021-11-16T21:44:31Z", "author_association": "OWNER", "body": "Wrote a TIL about what I learned using `TopologicalSorter`: https://til.simonwillison.net/python/graphlib-topologicalsorter", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970673085", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970673085, "node_id": "IC_kwDOBm6k_c4520u9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T20:58:24Z", "updated_at": "2021-11-16T20:58:24Z", "author_association": "OWNER", "body": "New test:\r\n```python\r\n\r\nclass Complex(AsyncBase):\r\n    def __init__(self):\r\n        self.log = []\r\n\r\n    async def d(self):\r\n        await asyncio.sleep(random() * 0.1)\r\n        print(\"LOG: d\")\r\n        self.log.append(\"d\")\r\n\r\n    async def c(self):\r\n        await asyncio.sleep(random() * 0.1)\r\n        print(\"LOG: c\")\r\n        self.log.append(\"c\")\r\n\r\n    async def b(self, c, d):\r\n        print(\"LOG: b\")\r\n        self.log.append(\"b\")\r\n\r\n    async def a(self, b, c):\r\n        print(\"LOG: a\")\r\n        self.log.append(\"a\")\r\n\r\n    async def go(self, a):\r\n        print(\"LOG: go\")\r\n        self.log.append(\"go\")\r\n        return self.log\r\n\r\n\r\n@pytest.mark.asyncio\r\nasync def test_complex():\r\n    result = await Complex().go()\r\n    # 'c' should only be called once\r\n    assert tuple(result) in (\r\n        # c and d could happen in either order\r\n        (\"c\", \"d\", \"b\", \"a\", \"go\"),\r\n        (\"d\", \"c\", \"b\", \"a\", \"go\"),\r\n    )\r\n```\r\nAnd this code passes it:\r\n```python\r\nimport asyncio\r\nfrom functools import wraps\r\nimport inspect\r\n\r\ntry:\r\n    import graphlib\r\nexcept ImportError:\r\n    from . import vendored_graphlib as graphlib\r\n\r\n\r\nclass AsyncMeta(type):\r\n    def __new__(cls, name, bases, attrs):\r\n        # Decorate any items that are 'async def' methods\r\n        _registry = {}\r\n        new_attrs = {\"_registry\": _registry}\r\n        for key, value in attrs.items():\r\n            if inspect.iscoroutinefunction(value) and not value.__name__ == \"resolve\":\r\n                new_attrs[key] = make_method(value)\r\n                _registry[key] = new_attrs[key]\r\n            else:\r\n                new_attrs[key] = value\r\n        # Gather graph for later dependency resolution\r\n        graph = {\r\n            key: {\r\n                p\r\n                for p in inspect.signature(method).parameters.keys()\r\n                if p != \"self\" and not p.startswith(\"_\")\r\n            }\r\n            for key, method in _registry.items()\r\n        }\r\n        new_attrs[\"_graph\"] = graph\r\n        return super().__new__(cls, name, bases, new_attrs)\r\n\r\n\r\ndef make_method(method):\r\n    parameters = inspect.signature(method).parameters.keys()\r\n\r\n    @wraps(method)\r\n    async def inner(self, _results=None, **kwargs):\r\n        print(\"\\n{}.{}({}) _results={}\".format(self, method.__name__, kwargs, _results))\r\n\r\n        # Any parameters not provided by kwargs are resolved from registry\r\n        to_resolve = [p for p in parameters if p not in kwargs and p != \"self\"]\r\n        missing = [p for p in to_resolve if p not in self._registry]\r\n        assert (\r\n            not missing\r\n        ), \"The following DI parameters could not be found in the registry: {}\".format(\r\n            missing\r\n        )\r\n\r\n        results = {}\r\n        results.update(kwargs)\r\n        if to_resolve:\r\n            resolved_parameters = await self.resolve(to_resolve, _results)\r\n            results.update(resolved_parameters)\r\n        return_value = await method(self, **results)\r\n        if _results is not None:\r\n            _results[method.__name__] = return_value\r\n        return return_value\r\n\r\n    return inner\r\n\r\n\r\nclass AsyncBase(metaclass=AsyncMeta):\r\n    async def resolve(self, names, results=None):\r\n        print(\"\\n  resolve: \", names)\r\n        if results is None:\r\n            results = {}\r\n\r\n        # Come up with an execution plan, just for these nodes\r\n        ts = graphlib.TopologicalSorter()\r\n        to_do = set(names)\r\n        done = set()\r\n        while to_do:\r\n            item = to_do.pop()\r\n            dependencies = self._graph[item]\r\n            ts.add(item, *dependencies)\r\n            done.add(item)\r\n            # Add any not-done dependencies to the queue\r\n            to_do.update({k for k in dependencies if k not in done})\r\n\r\n        ts.prepare()\r\n        plan = []\r\n        while ts.is_active():\r\n            node_group = ts.get_ready()\r\n            plan.append(node_group)\r\n            ts.done(*node_group)\r\n\r\n        print(\"plan:\", plan)\r\n\r\n        results = {}\r\n        for node_group in plan:\r\n            awaitables = [\r\n                self._registry[name](\r\n                    self,\r\n                    _results=results,\r\n                    **{k: v for k, v in results.items() if k in self._graph[name]},\r\n                )\r\n                for name in node_group\r\n            ]\r\n            print(\"    results = \", results)\r\n            print(\"    awaitables: \", awaitables)\r\n            awaitable_results = await asyncio.gather(*awaitables)\r\n            results.update(\r\n                {p[0].__name__: p[1] for p in zip(awaitables, awaitable_results)}\r\n            )\r\n\r\n        print(\"  End of resolve(), returning\", results)\r\n        return {key: value for key, value in results.items() if key in names}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970660299", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970660299, "node_id": "IC_kwDOBm6k_c452xnL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T20:39:43Z", "updated_at": "2021-11-16T20:42:27Z", "author_association": "OWNER", "body": "But that does seem to be the plan that `TopographicalSorter` provides:\r\n```python\r\ngraph = {\"go\": {\"a\"}, \"a\": {\"b\", \"c\"}, \"b\": {\"c\", \"d\"}}\r\n\r\nts = TopologicalSorter(graph)\r\nts.prepare()\r\nwhile ts.is_active():\r\n    nodes = ts.get_ready()\r\n    print(nodes)\r\n    ts.done(*nodes)\r\n```\r\nOutputs:\r\n```\r\n('c', 'd')\r\n('b',)\r\n('a',)\r\n('go',)\r\n```\r\nAlso:\r\n```python\r\ngraph = {\"go\": {\"d\", \"e\", \"f\"}, \"d\": {\"b\", \"c\"}, \"b\": {\"c\"}}\r\n\r\nts = TopologicalSorter(graph)\r\nts.prepare()\r\nwhile ts.is_active():\r\n    nodes = ts.get_ready()\r\n    print(nodes)\r\n    ts.done(*nodes)\r\n```\r\nGives:\r\n```\r\n('e', 'f', 'c')\r\n('b',)\r\n('d',)\r\n('go',)\r\n```\r\nI'm confident that `TopologicalSorter` is the way to do this. I think I need to rewrite my code to call it once to get that plan, then `await asyncio.gather(*nodes)` in turn to execute it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970657874", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970657874, "node_id": "IC_kwDOBm6k_c452xBS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T20:36:01Z", "updated_at": "2021-11-16T20:36:01Z", "author_association": "OWNER", "body": "My goal here is to calculate the most efficient way to resolve the different nodes, running them in parallel where possible.\r\n\r\nSo for this class:\r\n\r\n```python\r\nclass Complex(AsyncBase):\r\n    async def d(self):\r\n        pass\r\n\r\n    async def c(self):\r\n        pass\r\n\r\n    async def b(self, c, d):\r\n        pass\r\n\r\n    async def a(self, b, c):\r\n        pass\r\n\r\n    async def go(self, a):\r\n        pass\r\n```\r\nA call to `go()` should do this:\r\n\r\n- `c` and `d` in parallel\r\n- `b`\r\n- `a`\r\n- `go`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970655927", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970655927, "node_id": "IC_kwDOBm6k_c452wi3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T20:33:11Z", "updated_at": "2021-11-16T20:33:11Z", "author_association": "OWNER", "body": "What should be happening here instead is it should resolve the full graph and notice that `c` is depended on by both `b` and `a` - so it should run `c` first, then run the next ones in parallel.\r\n\r\nSo maybe the algorithm I'm inheriting from https://docs.python.org/3/library/graphlib.html isn't the correct algorithm?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970655304", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970655304, "node_id": "IC_kwDOBm6k_c452wZI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T20:32:16Z", "updated_at": "2021-11-16T20:32:16Z", "author_association": "OWNER", "body": "This code is really fiddly. I just got to this version:\r\n```python\r\nimport asyncio\r\nfrom functools import wraps\r\nimport inspect\r\n\r\ntry:\r\n    import graphlib\r\nexcept ImportError:\r\n    from . import vendored_graphlib as graphlib\r\n\r\n\r\nclass AsyncMeta(type):\r\n    def __new__(cls, name, bases, attrs):\r\n        # Decorate any items that are 'async def' methods\r\n        _registry = {}\r\n        new_attrs = {\"_registry\": _registry}\r\n        for key, value in attrs.items():\r\n            if inspect.iscoroutinefunction(value) and not value.__name__ == \"resolve\":\r\n                new_attrs[key] = make_method(value)\r\n                _registry[key] = new_attrs[key]\r\n            else:\r\n                new_attrs[key] = value\r\n        # Gather graph for later dependency resolution\r\n        graph = {\r\n            key: {\r\n                p\r\n                for p in inspect.signature(method).parameters.keys()\r\n                if p != \"self\" and not p.startswith(\"_\")\r\n            }\r\n            for key, method in _registry.items()\r\n        }\r\n        new_attrs[\"_graph\"] = graph\r\n        return super().__new__(cls, name, bases, new_attrs)\r\n\r\n\r\ndef make_method(method):\r\n    @wraps(method)\r\n    async def inner(self, _results=None, **kwargs):\r\n        print(\"inner - _results=\", _results)\r\n        parameters = inspect.signature(method).parameters.keys()\r\n        # Any parameters not provided by kwargs are resolved from registry\r\n        to_resolve = [p for p in parameters if p not in kwargs and p != \"self\"]\r\n        missing = [p for p in to_resolve if p not in self._registry]\r\n        assert (\r\n            not missing\r\n        ), \"The following DI parameters could not be found in the registry: {}\".format(\r\n            missing\r\n        )\r\n        results = {}\r\n        results.update(kwargs)\r\n        if to_resolve:\r\n            resolved_parameters = await self.resolve(to_resolve, _results)\r\n            results.update(resolved_parameters)\r\n        return_value = await method(self, **results)\r\n        if _results is not None:\r\n            _results[method.__name__] = return_value\r\n        return return_value\r\n\r\n    return inner\r\n\r\n\r\nclass AsyncBase(metaclass=AsyncMeta):\r\n    async def resolve(self, names, results=None):\r\n        print(\"\\n  resolve: \", names)\r\n        if results is None:\r\n            results = {}\r\n\r\n        # Resolve them in the correct order\r\n        ts = graphlib.TopologicalSorter()\r\n        for name in names:\r\n            ts.add(name, *self._graph[name])\r\n        ts.prepare()\r\n\r\n        async def resolve_nodes(nodes):\r\n            print(\"    resolve_nodes\", nodes)\r\n            print(\"    (current results = {})\".format(repr(results)))\r\n            awaitables = [\r\n                self._registry[name](\r\n                    self,\r\n                    _results=results,\r\n                    **{k: v for k, v in results.items() if k in self._graph[name]},\r\n                )\r\n                for name in nodes\r\n                if name not in results\r\n            ]\r\n            print(\"    awaitables: \", awaitables)\r\n            awaitable_results = await asyncio.gather(*awaitables)\r\n            results.update(\r\n                {p[0].__name__: p[1] for p in zip(awaitables, awaitable_results)}\r\n            )\r\n\r\n        if not ts.is_active():\r\n            # Nothing has dependencies - just resolve directly\r\n            print(\"    no dependencies, resolve directly\")\r\n            await resolve_nodes(names)\r\n        else:\r\n            # Resolve in topological order\r\n            while ts.is_active():\r\n                nodes = ts.get_ready()\r\n                print(\"    ts.get_ready() returned nodes:\", nodes)\r\n                await resolve_nodes(nodes)\r\n                for node in nodes:\r\n                    ts.done(node)\r\n\r\n        print(\"  End of resolve(), returning\", results)\r\n        return {key: value for key, value in results.items() if key in names}\r\n```\r\nWith this test:\r\n```python\r\nclass Complex(AsyncBase):\r\n    def __init__(self):\r\n        self.log = []\r\n\r\n    async def c(self):\r\n        print(\"LOG: c\")\r\n        self.log.append(\"c\")\r\n\r\n    async def b(self, c):\r\n        print(\"LOG: b\")\r\n        self.log.append(\"b\")\r\n\r\n    async def a(self, b, c):\r\n        print(\"LOG: a\")\r\n        self.log.append(\"a\")\r\n\r\n    async def go(self, a):\r\n        print(\"LOG: go\")\r\n        self.log.append(\"go\")\r\n        return self.log\r\n\r\n\r\n@pytest.mark.asyncio\r\nasync def test_complex():\r\n    result = await Complex().go()\r\n    # 'c' should only be called once\r\n    assert result == [\"c\", \"b\", \"a\", \"go\"]\r\n```\r\nThis test sometimes passes, and sometimes fails!\r\n\r\nOutput for a pass:\r\n```\r\ntests/test_asyncdi.py inner - _results= None\r\n\r\n  resolve:  ['a']\r\n    ts.get_ready() returned nodes: ('c', 'b')\r\n    resolve_nodes ('c', 'b')\r\n    (current results = {})\r\n    awaitables:  [<coroutine object Complex.c at 0x1074ac890>, <coroutine object Complex.b at 0x1074ac820>]\r\ninner - _results= {}\r\nLOG: c\r\ninner - _results= {'c': None}\r\n\r\n  resolve:  ['c']\r\n    ts.get_ready() returned nodes: ('c',)\r\n    resolve_nodes ('c',)\r\n    (current results = {'c': None})\r\n    awaitables:  []\r\n  End of resolve(), returning {'c': None}\r\nLOG: b\r\n    ts.get_ready() returned nodes: ('a',)\r\n    resolve_nodes ('a',)\r\n    (current results = {'c': None, 'b': None})\r\n    awaitables:  [<coroutine object Complex.a at 0x1074ac7b0>]\r\ninner - _results= {'c': None, 'b': None}\r\nLOG: a\r\n  End of resolve(), returning {'c': None, 'b': None, 'a': None}\r\nLOG: go\r\n```\r\nOutput for a fail:\r\n```\r\ntests/test_asyncdi.py inner - _results= None\r\n\r\n  resolve:  ['a']\r\n    ts.get_ready() returned nodes: ('b', 'c')\r\n    resolve_nodes ('b', 'c')\r\n    (current results = {})\r\n    awaitables:  [<coroutine object Complex.b at 0x10923c890>, <coroutine object Complex.c at 0x10923c820>]\r\ninner - _results= {}\r\n\r\n  resolve:  ['c']\r\n    ts.get_ready() returned nodes: ('c',)\r\n    resolve_nodes ('c',)\r\n    (current results = {})\r\n    awaitables:  [<coroutine object Complex.c at 0x10923c6d0>]\r\ninner - _results= {}\r\nLOG: c\r\ninner - _results= {'c': None}\r\nLOG: c\r\n  End of resolve(), returning {'c': None}\r\nLOG: b\r\n    ts.get_ready() returned nodes: ('a',)\r\n    resolve_nodes ('a',)\r\n    (current results = {'c': None, 'b': None})\r\n    awaitables:  [<coroutine object Complex.a at 0x10923c6d0>]\r\ninner - _results= {'c': None, 'b': None}\r\nLOG: a\r\n  End of resolve(), returning {'c': None, 'b': None, 'a': None}\r\nLOG: go\r\nF\r\n\r\n=================================================================================================== FAILURES ===================================================================================================\r\n_________________________________________________________________________________________________ test_complex _________________________________________________________________________________________________\r\n\r\n    @pytest.mark.asyncio\r\n    async def test_complex():\r\n        result = await Complex().go()\r\n        # 'c' should only be called once\r\n>       assert result == [\"c\", \"b\", \"a\", \"go\"]\r\nE       AssertionError: assert ['c', 'c', 'b', 'a', 'go'] == ['c', 'b', 'a', 'go']\r\nE         At index 1 diff: 'c' != 'b'\r\nE         Left contains one more item: 'go'\r\nE         Use -v to get the full diff\r\n\r\ntests/test_asyncdi.py:48: AssertionError\r\n================== short test summary info ================================\r\nFAILED tests/test_asyncdi.py::test_complex - AssertionError: assert ['c', 'c', 'b', 'a', 'go'] == ['c', 'b', 'a', 'go']\r\n```\r\nI figured out why this is happening.\r\n\r\n`a` requires `b` and `c`\r\n\r\n`b` also requires `c`\r\n\r\nThe code decides to run `b` and `c` in parallel.\r\n\r\nIf `c` completes first, then when `b` runs it gets to use the already-calculated result for `c` - so it doesn't need to call `c` again.\r\n\r\nIf `b` gets to that point before `c` does it also needs to call `c`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-970624197", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 970624197, "node_id": "IC_kwDOBm6k_c452ozF", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-16T19:49:05Z", "updated_at": "2021-11-16T19:49:05Z", "author_association": "OWNER", "body": "Here's the latest version of my weird dependency injection async class:\r\n```python\r\nimport inspect\r\n\r\nclass AsyncMeta(type):\r\n    def __new__(cls, name, bases, attrs):\r\n        # Decorate any items that are 'async def' methods\r\n        _registry = {}\r\n        new_attrs = {\"_registry\": _registry}\r\n        for key, value in attrs.items():\r\n            if inspect.iscoroutinefunction(value) and not value.__name__ == \"resolve\":\r\n                new_attrs[key] = make_method(value)\r\n                _registry[key] = new_attrs[key]\r\n            else:\r\n                new_attrs[key] = value\r\n\r\n        # Topological sort of _registry by parameter dependencies\r\n        graph = {\r\n            key: {\r\n                p for p in inspect.signature(method).parameters.keys()\r\n                if p != \"self\" and not p.startswith(\"_\")\r\n            }\r\n            for key, method in _registry.items()\r\n        }\r\n        new_attrs[\"_graph\"] = graph\r\n        return super().__new__(cls, name, bases, new_attrs)\r\n\r\n\r\ndef make_method(method):\r\n    @wraps(method)\r\n    async def inner(self, **kwargs):\r\n        parameters = inspect.signature(method).parameters.keys()\r\n        # Any parameters not provided by kwargs are resolved from registry\r\n        to_resolve = [p for p in parameters if p not in kwargs and p != \"self\"]\r\n        missing = [p for p in to_resolve if p not in self._registry]\r\n        assert (\r\n            not missing\r\n        ), \"The following DI parameters could not be found in the registry: {}\".format(\r\n            missing\r\n        )\r\n        results = {}\r\n        results.update(kwargs)\r\n        results.update(await self.resolve(to_resolve))\r\n        return await method(self, **results)\r\n\r\n    return inner\r\n\r\n\r\nbad = [0]\r\n\r\nclass AsyncBase(metaclass=AsyncMeta):\r\n    async def resolve(self, names):\r\n        print(\" resolve({})\".format(names))\r\n        results = {}\r\n        # Resolve them in the correct order\r\n        ts = TopologicalSorter()\r\n        ts2 = TopologicalSorter()\r\n        print(\" names = \", names)\r\n        print(\" self._graph = \", self._graph)\r\n        for name in names:\r\n            if self._graph[name]:\r\n                ts.add(name, *self._graph[name])\r\n                ts2.add(name, *self._graph[name])\r\n        print(\" static_order =\", tuple(ts2.static_order()))\r\n        ts.prepare()\r\n        while ts.is_active():\r\n            print(\"  is_active, i = \", bad[0])\r\n            bad[0] += 1\r\n            if bad[0] > 20:\r\n                print(\"    Infinite loop?\")\r\n                break\r\n            nodes = ts.get_ready()\r\n            print(\"  Do nodes:\", nodes)\r\n            awaitables = [self._registry[name](self, **{\r\n                k: v for k, v in results.items() if k in self._graph[name]\r\n            }) for name in nodes]\r\n            print(\"  awaitables: \", awaitables)\r\n            awaitable_results = await asyncio.gather(*awaitables)\r\n            results.update({\r\n                p[0].__name__: p[1] for p in zip(awaitables, awaitable_results)\r\n            })\r\n            print(results)\r\n            for node in nodes:\r\n                ts.done(node)\r\n\r\n        return results\r\n```\r\nExample usage:\r\n```python\r\nclass Foo(AsyncBase):\r\n    async def graa(self, boff):\r\n        print(\"graa\")\r\n        return 5\r\n    async def boff(self):\r\n        print(\"boff\")\r\n        return 8\r\n    async def other(self, boff, graa):\r\n        print(\"other\")\r\n        return 5 + boff + graa\r\n\r\nfoo = Foo()\r\nawait foo.other()\r\n```\r\nOutput:\r\n```\r\n resolve(['boff', 'graa'])\r\n names =  ['boff', 'graa']\r\n self._graph =  {'graa': {'boff'}, 'boff': set(), 'other': {'graa', 'boff'}}\r\n static_order = ('boff', 'graa')\r\n  is_active, i =  0\r\n  Do nodes: ('boff',)\r\n  awaitables:  [<coroutine object Foo.boff at 0x10bd81a40>]\r\n resolve([])\r\n names =  []\r\n self._graph =  {'graa': {'boff'}, 'boff': set(), 'other': {'graa', 'boff'}}\r\n static_order = ()\r\nboff\r\n{'boff': 8}\r\n  is_active, i =  1\r\n  Do nodes: ('graa',)\r\n  awaitables:  [<coroutine object Foo.graa at 0x10d66b340>]\r\n resolve([])\r\n names =  []\r\n self._graph =  {'graa': {'boff'}, 'boff': set(), 'other': {'graa', 'boff'}}\r\n static_order = ()\r\ngraa\r\n{'boff': 8, 'graa': 5}\r\nother\r\n18\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-951740637", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 951740637, "node_id": "IC_kwDOBm6k_c44umjd", "user": {"value": 30934, "label": "20after4"}, "created_at": "2021-10-26T09:12:15Z", "updated_at": "2021-10-26T09:12:15Z", "author_association": "NONE", "body": "This sounds really ambitious but also really awesome.  I like the idea that basically any piece of a page could be selectively replaced.\r\n\r\nIt sort of sounds like a python asyncio version of https://github.com/observablehq/runtime", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-803473015", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 803473015, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzQ3MzAxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-20T22:33:05Z", "updated_at": "2021-03-20T22:33:05Z", "author_association": "OWNER", "body": "Things this mechanism needs to be able to support:\r\n\r\n- Returning a default JSON representation\r\n- Defining \"extra\" JSON representations blocks, which can be requested using `?_extra=`\r\n- Returning rendered HTML, based on the default JSON + one or more extras + a template\r\n- Using Datasette output renderers to return e.g. CSV data\r\n- Potentially also supporting streaming output renderers for streaming CSV/TSV/JSON-nl etc", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-803472595", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 803472595, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzQ3MjU5NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-20T22:28:12Z", "updated_at": "2021-03-20T22:28:12Z", "author_association": "OWNER", "body": "Another idea I had: a view is a class that takes the `datasette` instance in its constructor, and defines a `__call__` method that accepts a request and returns a response. Except `await __call__` looks like it might be a bit messy, discussion in https://github.com/encode/starlette/issues/886", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-803472278", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 803472278, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzQ3MjI3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-20T22:25:04Z", "updated_at": "2021-03-20T22:25:04Z", "author_association": "OWNER", "body": "I came up with a slightly wild idea for this that would involve pytest-style dependency injection.\r\n\r\nPrototype here: https://gist.github.com/simonw/496b24fdad44f6f8b7237fe394a0ced7\r\n\r\nCopying from my private notes:\r\n\r\n> Use the lazy evaluated DI mechanism to break up table view into different pieces eg for faceting\r\n> \r\n> Use that to solve JSON vs HTML views\r\n> \r\n> Oh here's an idea: what if the various components of the table view were each defined  as async functions.... and then executed using asyncio.gather in order to run the SQL queries in parallel? Then try benchmarking with different numbers of threads?\r\n> \r\n> The async_call_with_arguments function could do this automatically for any awaitable dependencies\r\n> \r\n> This would give me massively parallel dependency injection\r\n> \r\n> (I could build an entire framework around this and call it c64)\r\n> \r\n> Idea: arguments called eg \"count\" are executed and the result passed to the function. If called count_fn then a reference to the not-yet-called function is passed instead \r\n> \r\n> I'm not going to completely combine the views mechanism and the render hooks. Instead, the core view will define a bunch of functions used to compose the page and the render hook will have conditional access to those functions - which will otherwise be asyncio.gather executed directly by the HTML page version\r\n> \r\n> Using asyncio.gather to execute facets and suggest facets in parallel would be VERY interesting \r\n> \r\n> suggest facets should be VERY cachable - doesn't matter if it's wrong unlike actual facets themselves\r\n> \r\n> What if all Datasette views were defined in terms of dependency injection - and those dependency functions could themselves depend on others just like pytest fixtures. Everything would become composable and async stuff could execute in parallel\r\n> \r\n> FURTHER IDEA: use this for the ?_extra= mechanism as well.\r\n> \r\n> Any view in Datasette can be defined as a collection of named keys. Each of those keys maps to a function or an async function that accepts as input other named keys, using DI to handle them.\r\n> \r\n> The HTML view is a defined function. So are the other outputs.\r\n> \r\n> Default original inputs include \u201crequest\u201d and \u201cdatasette\u201d.\r\n> \r\n> So\u2026 maybe a view function is a class methods that use DI. One of those methods as an .html() method used for the default page.\r\n> \r\n> Output formats are a bit more complicated because they are supposed to be defined separately in plugins. They are unified across query, row and table though.\r\n> \r\n> I\u2019m going to try breaking up the TableView to see what happens.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 1}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-803471917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 803471917, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzQ3MTkxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-20T22:21:33Z", "updated_at": "2021-03-20T22:21:33Z", "author_association": "OWNER", "body": "This has been blocking things for too long.\r\n\r\nIf this becomes a documented pattern, things like adding a JSON output to https://github.com/dogsheep/dogsheep-beta becomes easier too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-709503359", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 709503359, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTUwMzM1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T18:15:28Z", "updated_at": "2020-10-15T18:15:28Z", "author_association": "OWNER", "body": "I think this is blocking #619", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/878#issuecomment-709502889", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/878", "id": 709502889, "node_id": "MDEyOklzc3VlQ29tbWVudDcwOTUwMjg4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-15T18:14:34Z", "updated_at": "2020-10-15T18:14:34Z", "author_association": "OWNER", "body": "The `BaseView` class does this for Datasette internals at the moment, but I'm not convinced it works as well as it could.\r\n\r\nI'd like to turn this into a class that is documented and available to plugins as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648435885, "label": "New pattern for views that return either JSON or HTML, available for plugins"}, "performed_via_github_app": null}