{"html_url": "https://github.com/simonw/datasette/issues/422#issuecomment-488558896", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/422", "id": 488558896, "node_id": "MDEyOklzc3VlQ29tbWVudDQ4ODU1ODg5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-05-02T05:43:11Z", "updated_at": "2019-05-02T05:43:11Z", "author_association": "OWNER", "body": "I'm doing this with time limits right now, which I think is good enough for the moment.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 423316403, "label": "Figure out what to do about table counts in a mutable world"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/422#issuecomment-475277571", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/422", "id": 475277571, "node_id": "MDEyOklzc3VlQ29tbWVudDQ3NTI3NzU3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-03-21T15:30:03Z", "updated_at": "2019-03-21T15:30:03Z", "author_association": "OWNER", "body": "It would be useful to be able to detect if a table is a rowid table or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 423316403, "label": "Figure out what to do about table counts in a mutable world"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/422#issuecomment-474890920", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/422", "id": 474890920, "node_id": "MDEyOklzc3VlQ29tbWVudDQ3NDg5MDkyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-03-20T15:39:58Z", "updated_at": "2019-03-20T15:39:58Z", "author_association": "OWNER", "body": "The page with the most table counts on it is the index page: https://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d\r\n\r\n<img width=\"1032\" alt=\"Screen Shot 2019-03-20 at 8 36 00 AM\" src=\"https://user-images.githubusercontent.com/9599/54697753-5710ce80-4aeb-11e9-89aa-13f843a78d2c.png\">\r\n\r\nIf I paginate this (which needs to happen anyway for Datasette Library #417) the impact here won't be as bad. I could even load in the table row counts asynchronously via JavaScript?\r\n\r\nBigger problem is this total summed count representation on the homepage:\r\n\r\n<img width=\"1022\" alt=\"Screen Shot 2019-03-20 at 8 38 53 AM\" src=\"https://user-images.githubusercontent.com/9599/54697913-a3f4a500-4aeb-11e9-97a4-be501d737642.png\">\r\n\r\nI think that feature just won't be feasibly against large databases in a mutable world. Maybe we consider to show that total but only for immutable databases? May be easier just to drop it entirely (we will still show the table count).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 423316403, "label": "Figure out what to do about table counts in a mutable world"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/422#issuecomment-474888132", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/422", "id": 474888132, "node_id": "MDEyOklzc3VlQ29tbWVudDQ3NDg4ODEzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-03-20T15:34:37Z", "updated_at": "2019-03-20T15:34:37Z", "author_association": "OWNER", "body": "Here's a trick for lower bound counts which looks like it might actually work. Consider the following queries:\r\n\r\n```\r\nselect count(*) from (\r\n  select rowid from [most-common-name/surnames] limit 1000\r\n)\r\n```\r\nhttps://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d?sql=select+count%28*%29+from+%28%0D%0A++select+rowid+from+%5Bmost-common-name%2Fsurnames%5D+limit+1000%0D%0A%29\r\n\r\nTakes 0.827ms (it took longer with `select * from` in the subquery).\r\n\r\nSame query but with limit 10,000:\r\n\r\nhttps://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d?sql=select+count%28*%29+from+%28%0D%0A++select+rowid++from+%5Bmost-common-name%2Fsurnames%5D+limit+10000%0D%0A%29\r\n\r\nTook 2.335ms\r\n\r\nWith 100,000 limit:\r\n\r\nhttps://fivethirtyeight.datasettes.com/fivethirtyeight-b76415d?sql=select+count%28*%29+from+%28%0D%0A++select+rowid++from+%5Bmost-common-name%2Fsurnames%5D+limit+100000%0D%0A%29\r\n\r\nTook 27.558ms\r\n\r\nSo one solution here would be to pick an upper bound (maybe 100,001) and use this query, which should give an accurate count below that upper bound but allow us to show \"100,000+\" as a count if the table exceeds that boundary.\r\n\r\nMaybe the boundary is a config setting? Also, if a tighter timeout (maybe 20ms) is exceeded for that boundary we could halve it and try again.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 423316403, "label": "Figure out what to do about table counts in a mutable world"}, "performed_via_github_app": null}