issue_comments
8 rows where author_association = "OWNER", "created_at" is on date 2020-12-17, issue = 770436876 and reactions = "{"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}" sorted by updated_at descending
This data as json, CSV (advanced)
Suggested facets: created_at (date), updated_at (date)
issue 1
- Maintain an in-memory SQLite table of connected databases and their tables · 8 ✖
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
747768112 | https://github.com/simonw/datasette/issues/1150#issuecomment-747768112 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc2ODExMg== | simonw 9599 | 2020-12-17T23:25:21Z | 2020-12-17T23:25:21Z | OWNER | Next challenge: figure out how to use the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 | |
747767598 | https://github.com/simonw/datasette/issues/1150#issuecomment-747767598 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc2NzU5OA== | simonw 9599 | 2020-12-17T23:24:03Z | 2020-12-17T23:24:03Z | OWNER | I'm going to assume that even the heaviest user will have trouble going beyond a few hundred database files, so this is fine. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 | |
747767499 | https://github.com/simonw/datasette/issues/1150#issuecomment-747767499 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc2NzQ5OQ== | simonw 9599 | 2020-12-17T23:23:44Z | 2020-12-17T23:23:44Z | OWNER | Grabbing the schema version of 380 files in the root directory takes 70ms. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 | |
747767055 | https://github.com/simonw/datasette/issues/1150#issuecomment-747767055 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc2NzA1NQ== | simonw 9599 | 2020-12-17T23:22:41Z | 2020-12-17T23:22:41Z | OWNER | It's just recursion that's expensive. I created 380 empty SQLite databases in a folder and timed So maybe I tell users that all SQLite databases have to be in the root folder. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 | |
747766310 | https://github.com/simonw/datasette/issues/1150#issuecomment-747766310 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc2NjMxMA== | simonw 9599 | 2020-12-17T23:20:49Z | 2020-12-17T23:20:49Z | OWNER | I tried against my entire So it looks like connecting to a SQLite database file and getting the schema version is extremely fast. Scanning directories is slower. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 | |
747764712 | https://github.com/simonw/datasette/issues/1150#issuecomment-747764712 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc2NDcxMg== | simonw 9599 | 2020-12-17T23:16:31Z | 2020-12-17T23:16:31Z | OWNER | Quick micro-benchmark, run against a folder with 46 database files adding up to 1.4GB total: ```python import pathlib, sqlite3, time paths = list(pathlib.Path(".").glob('*.db')) def schema_version(path): db = sqlite3.connect(path) version = db.execute("PRAGMA schema_version").fetchall()[0] db.close() return version def all(): versions = {} for path in paths: versions[path.name] = schema_version(path) return versions start = time.time(); all(); print(time.time() - start) 0.012346982955932617``` So that's 12ms. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 | |
747754229 | https://github.com/simonw/datasette/issues/1150#issuecomment-747754229 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc1NDIyOQ== | simonw 9599 | 2020-12-17T23:04:38Z | 2020-12-17T23:04:38Z | OWNER | Open question: will this work for hundreds of database files, or is the overhead of connecting to each of 100 databases in turn to run |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 | |
747754082 | https://github.com/simonw/datasette/issues/1150#issuecomment-747754082 | https://api.github.com/repos/simonw/datasette/issues/1150 | MDEyOklzc3VlQ29tbWVudDc0Nzc1NDA4Mg== | simonw 9599 | 2020-12-17T23:04:13Z | 2020-12-17T23:04:13Z | OWNER | Pages that need a list of all databases - the index page and /-/databases for example - could trigger a "check for new directories in the configured directories" scan. That scan would run at most once every 5 (n) seconds - the check is triggered if it’s run more recently than that it doesn’t run. Hopefully this means it could be done as a blocking operation, rather than trying to run it in a thread. When it runs it scans for .db or .sqlite files (maybe one or two other extensions) that it hasn’t seen before. It also checks that the existing list of known database files still exists. If it finds any new ones it connects to them once to run |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maintain an in-memory SQLite table of connected databases and their tables 770436876 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
user 1