home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

9 rows where issue = 777535402 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • simonw 9

issue 1

  • Use _counts to speed up counts · 9 ✖

author_association 1

  • OWNER 9
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
753668099 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753668099 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzY2ODA5OQ== simonw 9599 2021-01-03T19:55:53Z 2021-01-03T19:55:53Z OWNER

So if you instantiate the Database() constructor with use_counts_table=True any access to the .count properties will go through this table - otherwise regular count(*) queries will be executed.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753665521 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753665521 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzY2NTUyMQ== simonw 9599 2021-01-03T19:31:33Z 2021-01-03T19:31:33Z OWNER

I'm having second thoughts about this being the default behaviour. It's pretty weird. I feel like HUGE databases that need this are rare, so having it on by default doesn't make sense.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753662490 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753662490 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzY2MjQ5MA== simonw 9599 2021-01-03T19:05:53Z 2021-01-03T19:05:53Z OWNER

Idea: a .execute_count() method that never uses the cache.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753661292 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753661292 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzY2MTI5Mg== simonw 9599 2021-01-03T18:56:06Z 2021-01-03T18:56:23Z OWNER

Another option: on creation of the Database() object, check to see if the _counts table exists and use that as the default for a use_counts_table property. Also flip that property to True if the user calls .enable_counts() at any time.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753661158 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753661158 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzY2MTE1OA== simonw 9599 2021-01-03T18:55:16Z 2021-01-03T18:55:16Z OWNER

Alternative implementation: provided db.should_trust_counts is True, try running the query: sql select count from _counts where [table] = ? If the query fails to return a result OR throws an error because the table doesn't exist, run the count(*) query.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753660814 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753660814 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzY2MDgxNA== simonw 9599 2021-01-03T18:53:05Z 2021-01-03T18:53:05Z OWNER

Here's the current .count property: https://github.com/simonw/sqlite-utils/blob/036ec6d32313487527c66dea613a3e7118b97459/sqlite_utils/db.py#L597-L609

It's implemented on Queryable which means it's available on both Table and View - the optimization doesn't make sense for views.

I'm a bit cautious about making that property so much more complex. In order to decide if it should try the _counts table first it needs to know:

  • Should it be trusting the counts? I'm thinking a .should_trust_counts property on Database which defaults to True would be good - then advanced users can turn that off if they know the counts should not be trusted.
  • Does the _counts table exist?
  • Are the triggers defined?

Then it can do the query, and if the query fails it can fall back on the count(*). That's quite a lot of extra activity though.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753660379 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753660379 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzY2MDM3OQ== simonw 9599 2021-01-03T18:50:15Z 2021-01-03T18:50:15Z OWNER

python def cached_counts(self, tables=None): sql = "select [table], count from {}".format(self._counts_table_name) if tables: sql += " where [table] in ({})".format(", ".join("?" for table in tables)) return {r[0]: r[1] for r in self.execute(sql, tables).fetchall()}

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753545757 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545757 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzU0NTc1Nw== simonw 9599 2021-01-02T23:58:07Z 2021-01-02T23:58:07Z OWNER

Thought: maybe there should be a .reset_counts() method too, for if the table gets out of date with the triggers.

One way that could happen is if a table is dropped and recreated - the counts in the _counts table would likely no longer match the number of rows in that table.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  
753545381 https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753545381 https://api.github.com/repos/simonw/sqlite-utils/issues/215 MDEyOklzc3VlQ29tbWVudDc1MzU0NTM4MQ== simonw 9599 2021-01-02T23:52:52Z 2021-01-02T23:52:52Z OWNER

Idea: a db.cached_counts() method that returns a dictionary of data from the _counts table. Call it with a list of tables to get back the counts for just those tables.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Use _counts to speed up counts 777535402  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 25.032ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows