{"html_url": "https://github.com/simonw/datasette/issues/1394#issuecomment-881129149", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1394", "id": 881129149, "node_id": "IC_kwDOBm6k_c40hPa9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-07-16T02:23:32Z", "updated_at": "2021-07-16T02:23:32Z", "author_association": "OWNER", "body": "Wrote about this in the annotated release notes for 0.58: https://simonwillison.net/2021/Jul/16/datasette-058/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 944870799, "label": "Big performance boost on faceting: skip the inner order by"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1394#issuecomment-880900534", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1394", "id": 880900534, "node_id": "MDEyOklzc3VlQ29tbWVudDg4MDkwMDUzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-07-15T17:58:03Z", "updated_at": "2021-07-15T17:58:03Z", "author_association": "OWNER", "body": "Started a conversation about this on the SQLite forum: https://sqlite.org/forum/forumpost/2d76f2bcf65d256a?t=h", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 944870799, "label": "Big performance boost on faceting: skip the inner order by"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1394#issuecomment-880287483", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1394", "id": 880287483, "node_id": "MDEyOklzc3VlQ29tbWVudDg4MDI4NzQ4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-07-15T00:01:47Z", "updated_at": "2021-07-15T00:01:47Z", "author_association": "OWNER", "body": "I wrote this code:\r\n```python\r\n_order_by_re = re.compile(r\"(^.*) order by [a-zA-Z_][a-zA-Z0-9_]+( desc)?$\", re.DOTALL)\r\n_order_by_braces_re = re.compile(r\"(^.*) order by \\[[^\\]]+\\]( desc)?$\", re.DOTALL)\r\n\r\n\r\ndef strip_order_by(sql):\r\n    for regex in (_order_by_re, _order_by_braces_re):\r\n        match = regex.match(sql)\r\n        if match is not None:\r\n            return match.group(1)\r\n    return sql\r\n\r\n@pytest.mark.parametrize(\r\n    \"sql,expected\",\r\n    [\r\n        (\"blah\", \"blah\"),\r\n        (\"select * from foo\", \"select * from foo\"),\r\n        (\"select * from foo order by bah\", \"select * from foo\"),\r\n        (\"select * from foo order by bah desc\", \"select * from foo\"),\r\n        (\"select * from foo order by [select]\", \"select * from foo\"),\r\n        (\"select * from foo order by [select] desc\", \"select * from foo\"),\r\n    ],\r\n)\r\ndef test_strip_order_by(sql, expected):\r\n    assert strip_order_by(sql) == expected\r\n```\r\nBut it turns out I don't need it! The SQL that is passed to the facet class is created by this code: https://github.com/simonw/datasette/blob/ba11ef27edd6981eeb26d7ecf5aa236707f5f8ce/datasette/views/table.py#L677-L684\r\n\r\nAnd the only place that uses that `sql_no_limit` variable is here: https://github.com/simonw/datasette/blob/ba11ef27edd6981eeb26d7ecf5aa236707f5f8ce/datasette/views/table.py#L733-L745\r\n\r\nSo I can change that to `sql_no_limit_no_order` and fix the bug that way instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 944870799, "label": "Big performance boost on faceting: skip the inner order by"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1394#issuecomment-880278256", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1394", "id": 880278256, "node_id": "MDEyOklzc3VlQ29tbWVudDg4MDI3ODI1Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-07-14T23:35:18Z", "updated_at": "2021-07-14T23:35:18Z", "author_association": "OWNER", "body": "The challenge here is that faceting doesn't currently modify the inner SQL at all - it wraps it so that it can work against any SQL statement (though Datasette itself does not yet take advantage of that ability, only offering faceting on table pages).\r\n\r\nSo just removing the order by wouldn't be appropriate if the inner query looked something like this:\r\n\r\n```sql\r\nselect * from items order by created desc limit 100\r\n```\r\nSince the intent there would be to return facet counts against only the most recent 100 items.\r\n\r\nIn SQLite the `limit` has to come after the `order by` though, so the fix here could be as easy as using a regular expression to identify queries that end with `order by COLUMN (desc)?` and stripping off that clause.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 944870799, "label": "Big performance boost on faceting: skip the inner order by"}, "performed_via_github_app": null}