{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753668099", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753668099, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzY2ODA5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-03T19:55:53Z", "updated_at": "2021-01-03T19:55:53Z", "author_association": "OWNER", "body": "So if you instantiate the `Database()` constructor with `use_counts_table=True` any access to the `.count` properties will go through this table - otherwise regular `count(*)` queries will be executed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753665521", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753665521, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzY2NTUyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-03T19:31:33Z", "updated_at": "2021-01-03T19:31:33Z", "author_association": "OWNER", "body": "I'm having second thoughts about this being the default behaviour. It's pretty weird. I feel like HUGE databases that need this are rare, so having it on by default doesn't make sense.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753662490", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753662490, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzY2MjQ5MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-03T19:05:53Z", "updated_at": "2021-01-03T19:05:53Z", "author_association": "OWNER", "body": "Idea: a `.execute_count()` method that never uses the cache.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753661292", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753661292, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzY2MTI5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-03T18:56:06Z", "updated_at": "2021-01-03T18:56:23Z", "author_association": "OWNER", "body": "Another option: on creation of the `Database()` object, check to see if the `_counts` table exists and use that as the default for a `use_counts_table` property. Also flip that property to `True` if the user calls `.enable_counts()` at any time.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753661158", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753661158, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzY2MTE1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-03T18:55:16Z", "updated_at": "2021-01-03T18:55:16Z", "author_association": "OWNER", "body": "Alternative implementation: provided `db.should_trust_counts` is `True`, try running the query:\r\n```sql\r\nselect count from _counts where [table] = ?\r\n```\r\nIf the query fails to return a result OR throws an error because the table doesn't exist, run the `count(*)` query.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753660814", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753660814, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzY2MDgxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-03T18:53:05Z", "updated_at": "2021-01-03T18:53:05Z", "author_association": "OWNER", "body": "Here's the current `.count` property: https://github.com/simonw/sqlite-utils/blob/036ec6d32313487527c66dea613a3e7118b97459/sqlite_utils/db.py#L597-L609\r\n\r\nIt's implemented on `Queryable` which means it's available on both `Table` and `View` - the optimization doesn't make sense for views.\r\n\r\nI'm a bit cautious about making that property so much more complex. In order to decide if it should try the `_counts` table first it needs to know:\r\n\r\n- Should it be trusting the counts? I'm thinking a `.should_trust_counts` property on `Database` which defaults to `True` would be good - then advanced users can turn that off if they know the counts should not be trusted.\r\n- Does the `_counts` table exist?\r\n- Are the triggers defined?\r\n\r\nThen it can do the query, and if the query fails it can fall back on the `count(*)`. That's quite a lot of extra activity though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753660379", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753660379, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzY2MDM3OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-03T18:50:15Z", "updated_at": "2021-01-03T18:50:15Z", "author_association": "OWNER", "body": "```python\r\n    def cached_counts(self, tables=None):\r\n        sql = \"select [table], count from {}\".format(self._counts_table_name)\r\n        if tables:\r\n            sql += \" where [table] in ({})\".format(\", \".join(\"?\" for table in tables))\r\n        return {r[0]: r[1] for r in self.execute(sql, tables).fetchall()}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753545757, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzU0NTc1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-02T23:58:07Z", "updated_at": "2021-01-02T23:58:07Z", "author_association": "OWNER", "body": "Thought: maybe there should be a `.reset_counts()` method too, for if the table gets out of date with the triggers.\r\n\r\nOne way that could happen is if a table is dropped and recreated - the counts in the `_counts` table would likely no longer match the number of rows in that table.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545381", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/215", "id": 753545381, "node_id": "MDEyOklzc3VlQ29tbWVudDc1MzU0NTM4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-02T23:52:52Z", "updated_at": "2021-01-02T23:52:52Z", "author_association": "OWNER", "body": "Idea: a `db.cached_counts()` method that returns a dictionary of data from the `_counts` table. Call it with a list of tables to get back the counts for just those tables.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777535402, "label": "Use _counts to speed up counts"}, "performed_via_github_app": null}