{"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399341151", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399341151, "node_id": "IC_kwDOBm6k_c5TaEBf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:03:20Z", "updated_at": "2023-01-21T22:03:20Z", "author_association": "OWNER", "body": "I think I'm going to have to write a new view function from scratch which completely ignores the existing BaseView/DataView/TableView hierarchy.\r\n\r\nHere's what I get on the incoming request:\r\n```\r\n(Pdb) request.url, request.full_path, request.host, request.url_vars\r\n('http://127.0.0.1:8001/content/repos.json', '/content/repos.json', '127.0.0.1:8001',\r\n {'database': 'content', 'table': 'repos', 'format': 'json'})\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399341658", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399341658, "node_id": "IC_kwDOBm6k_c5TaEJa", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:06:29Z", "updated_at": "2023-01-21T22:07:30Z", "author_association": "OWNER", "body": "Relevant:\r\n- #1101\r\n- #1672\r\n- #1062", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1399343659", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1399343659, "node_id": "IC_kwDOBm6k_c5TaEor", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-01-21T22:19:20Z", "updated_at": "2023-02-06T23:02:12Z", "author_association": "OWNER", "body": "HTML mode needs a list of renderers so it can show links to `.geojson` etc - can do that as a hidden extra (maybe called `renderers`), repeating this code:\r\n\r\nhttps://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/views/base.py#L477-L497", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1421988953", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1421988953, "node_id": "IC_kwDOBm6k_c5UwdRZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-02-08T04:35:44Z", "updated_at": "2023-02-08T05:27:48Z", "author_association": "OWNER", "body": "Next step: get `?_next=...` working (it is ignored at the moment, even though the returned JSON includes the `\"next\"` key).\r\n\r\nThen... figure out how to render HTML and other requested formats.\r\n\r\nThen get the tests to pass!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460759358", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460759358, "node_id": "IC_kwDOBm6k_c5XEWs-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T19:48:13Z", "updated_at": "2023-03-20T18:47:12Z", "author_association": "OWNER", "body": "Breaking this down into smaller steps:\r\n\r\n- [x] Get `?_next=` working\r\n- [x] Implement extensions - so `.json` is needed again for the JSON version, and anything without an extension is passed through a new code path for HTML\r\n- [ ] That HTML view should only access JSON data, which can be seen by using `.context` - this will require a lot of updates to templates (it may be necessary to still provide access to some helper functions though). This will form the basis of the ambition to fully document the template context.\r\n- [ ] Get a bunch of the existing table HTML and JSON tests to pass\r\n- [ ] Use those tests to refactor the nasty `_next` code, see https://github.com/simonw/datasette/pull/1999#issuecomment-1460905469\r\n- [ ] Figure out how the [register_output_renderer(datasette)](https://docs.datasette.io/en/stable/plugin_hooks.html#register-output-renderer-datasette) plugin hook should work", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460760116", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460760116, "node_id": "IC_kwDOBm6k_c5XEW40", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T19:48:52Z", "updated_at": "2023-03-08T19:48:52Z", "author_association": "OWNER", "body": "I'm trying to get `http://127.0.0.1:8001/fixtures/compound_three_primary_keys?_next=a,d,v` to return the correct results.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460905469", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460905469, "node_id": "IC_kwDOBm6k_c5XE6X9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:32:56Z", "updated_at": "2023-03-08T21:32:56Z", "author_association": "OWNER", "body": "I'd really like to extract this ugly logic out into a helper function:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1617-L1684\r\n\r\nI copied it in and asked ChatGPT to \"List all of the variable that are either modified or assigned to by the above code\":\r\n\r\n> Modified variables:\r\n> \r\n> offset\r\n> where_clauses\r\n> params\r\n> order_by\r\n> \r\n> Assigned variables:\r\n> \r\n> sort_value\r\n> next_by_pk_clauses\r\n> order_by (when sort or sort_desc is not None)\r\n\r\nThen I asked which variables were used as inputs, and argued with it a bit about whether it should be counting functions. Eventually got to this:\r\n\r\n> My apologies for the oversight. Here are the variables needed as input by the above code, excluding any functions:\r\n> \r\n> _next\r\n> is_view\r\n> sort\r\n> sort_desc\r\n> use_rowid\r\n> pks\r\n> order_by\r\n\r\nNote that `use_rowid` is actually defined earlier in terms of two of those other variables: https://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1540", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460906741", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460906741, "node_id": "IC_kwDOBm6k_c5XE6r1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:08Z", "updated_at": "2023-03-08T21:34:08Z", "author_association": "OWNER", "body": "So maybe I can refactor it to look a bit more like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L1602-L1604\r\n\r\nOne thing that's useful here is that `is_view` is handled early, like this:\r\n\r\nhttps://github.com/simonw/datasette/blob/db1a88f4e17a1f50bdaa681e8beddb2276503e7c/datasette/views/table.py#L466-L472\r\n\r\nSo if I omit the `is_view` bit from the extracted function I can simplify more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460907148", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460907148, "node_id": "IC_kwDOBm6k_c5XE6yM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:34:30Z", "updated_at": "2023-03-08T21:34:30Z", "author_association": "OWNER", "body": "I'm going to hold off on that refactor until later, when I have tests to show me if the refactor works or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460916405", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460916405, "node_id": "IC_kwDOBm6k_c5XE9C1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T21:43:27Z", "updated_at": "2023-03-08T21:43:27Z", "author_association": "OWNER", "body": "Just noticed that `_json=colname` is not working, and that's because it's handled by the renderer here:\r\n\r\nhttps://github.com/simonw/datasette/blob/56b0758a5fbf85d01ff80a40c9b028469d7bb65f/datasette/renderer.py#L29-L40\r\n\r\nBut that's not currently being called by my new code.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460943097", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460943097, "node_id": "IC_kwDOBm6k_c5XFDj5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:09:24Z", "updated_at": "2023-03-08T22:09:47Z", "author_association": "OWNER", "body": "The ease with which I added that `?_extra=query` feature in https://github.com/simonw/datasette/pull/1999/commits/96e94f9b7b2db53865e61390bcce6761727f26d8 made me feel really confident that this architecture is going in the right direction.\r\n\r\n```diff\r\ndiff --git a/datasette/views/table.py b/datasette/views/table.py\r\nindex 8d3bb2c930..3e1db9c85f 100644\r\n--- a/datasette/views/table.py\r\n+++ b/datasette/views/table.py\r\n@@ -1913,6 +1913,13 @@ async def extra_request():\r\n \"args\": request.args._data,\r\n }\r\n \r\n+ async def extra_query():\r\n+ \"Details of the underlying SQL query\"\r\n+ return {\r\n+ \"sql\": sql,\r\n+ \"params\": params,\r\n+ }\r\n+\r\n async def extra_extras():\r\n \"Available ?_extra= blocks\"\r\n return {\r\n@@ -1938,6 +1945,7 @@ async def extra_extras():\r\n extra_primary_keys,\r\n extra_debug,\r\n extra_request,\r\n+ extra_query,\r\n extra_extras,\r\n )\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460970807", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460970807, "node_id": "IC_kwDOBm6k_c5XFKU3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:31:49Z", "updated_at": "2023-03-08T22:33:03Z", "author_association": "OWNER", "body": "For the HTML version, I need to decide where all of the stuff that happens in `async def extra_template()` is going to live.\r\n\r\nI think it's another one of those extra functions, triggered for `?_extra=context`.\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L813-L912", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460986533", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460986533, "node_id": "IC_kwDOBm6k_c5XFOKl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:40:28Z", "updated_at": "2023-03-08T22:40:28Z", "author_association": "OWNER", "body": "Figuring out what to do with `display_columns_and_rows()` is hard. That returns rows as this special kind of object, which is designed to be accessed from the HTML templates:\r\n\r\nhttps://github.com/simonw/datasette/blob/96e94f9b7b2db53865e61390bcce6761727f26d8/datasette/views/table.py#L45-L71", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1460988975", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1460988975, "node_id": "IC_kwDOBm6k_c5XFOwv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:42:57Z", "updated_at": "2023-03-08T22:42:57Z", "author_association": "OWNER", "body": "Aside idea: it might be interesting if there were \"lazy\" template variables available in the context: things that are not actually executed unless a template author requests them.\r\n\r\nImagine if `metadata` was a lazy template reference, such that custom templates that don't display any metadata don't trigger it to be resolved (which might involve additional database queries some day).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1551694938, "label": "?_extra= support (draft)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1999#issuecomment-1461002039", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1999", "id": 1461002039, "node_id": "IC_kwDOBm6k_c5XFR83", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-03-08T22:58:16Z", "updated_at": "2023-03-08T23:02:09Z", "author_association": "OWNER", "body": "The reason for that `Row` thing is that it allows custom templates that do things like this:\r\n\r\nhttps://docs.datasette.io/en/stable/changelog.html#easier-custom-templates-for-table-rows\r\n\r\n```html+jinja\r\n{% for row in display_rows %}\r\n
{{ row[\"description\"] }} Category: {{ row.display(\"category_id\") }}