issue_comments

Table actions

22 rows where user = 536941 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, reactions, created_at (date), updated_at (date)

user 1

  • fgregg · 22 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1009548580 https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1009548580 https://api.github.com/repos/simonw/sqlite-utils/issues/365 IC_kwDOCGYnMM48LH0k fgregg 536941 2022-01-11T02:43:34Z 2022-01-11T02:43:34Z NONE

thanks so much! always a pleasure to see how you work through these things

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 create-index should run analyze after creating index 1096558279  
1008275546 https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008275546 https://api.github.com/repos/simonw/sqlite-utils/issues/365 IC_kwDOCGYnMM48GRBa fgregg 536941 2022-01-09T11:01:15Z 2022-01-09T13:37:51Z NONE

i don’t want to be such a partisan for analyze, but the query planner deciding not to use an index based on information collected by analyze is not necessarily a bug, but could be the correct choice.

<s>the original poster in that stack overflow doesn’t say there’s a performance regression </s>

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 create-index should run analyze after creating index 1096558279  
1008166084 https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008166084 https://api.github.com/repos/simonw/sqlite-utils/issues/365 IC_kwDOCGYnMM48F2TE fgregg 536941 2022-01-08T22:32:47Z 2022-01-08T22:32:47Z NONE

or using “ pragma optimize”

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 create-index should run analyze after creating index 1096558279  
1008164786 https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008164786 https://api.github.com/repos/simonw/sqlite-utils/issues/365 IC_kwDOCGYnMM48F1-y fgregg 536941 2022-01-08T22:24:19Z 2022-01-08T22:24:19Z NONE

the out-of-date scenario you describe could be addressed by automatically adding an analyze to the insert or convert commands if they implicate an index

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 create-index should run analyze after creating index 1096558279  
1008164116 https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008164116 https://api.github.com/repos/simonw/sqlite-utils/issues/365 IC_kwDOCGYnMM48F10U fgregg 536941 2022-01-08T22:18:57Z 2022-01-08T22:18:57Z NONE

the table with the query ran so bad was about 50k.

i think the scenario should not be worse than no stats.

i also did not know that sqlite was so different from postgres and needed an explicit analyze call.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 create-index should run analyze after creating index 1096558279  
1008161965 https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008161965 https://api.github.com/repos/simonw/sqlite-utils/issues/365 IC_kwDOCGYnMM48F1St fgregg 536941 2022-01-08T22:02:56Z 2022-01-08T22:02:56Z NONE

for options 2 and 3, i would worry about discoverablity.

in other db’s it is not necessary to explicitly call analyze for most indices. ie for postgres

The system regularly collects statistics on all of a table's columns. Newly-created non-expression indexes can immediately use these statistics to determine an index's usefulness.

i suppose i would propose raising a warning if the stats table is created that explains what is going on and informs users about a —no-analyze argument.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 create-index should run analyze after creating index 1096558279  
1007844190 https://github.com/simonw/datasette/pull/1574#issuecomment-1007844190 https://api.github.com/repos/simonw/datasette/issues/1574 IC_kwDOBm6k_c48Ente fgregg 536941 2022-01-08T00:42:12Z 2022-01-08T00:42:12Z NONE

is there a reason to not always use the slim option?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 introduce new option for datasette package to use a slim base image 1084193403  
1007636709 https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1007636709 https://api.github.com/repos/simonw/sqlite-utils/issues/365 IC_kwDOCGYnMM48D1Dl fgregg 536941 2022-01-07T18:28:33Z 2022-01-07T18:29:43Z NONE

i added an index to one table with sqlite-utils, and then a query that used to take about 1 second started taking hundreds of seconds.

running analyze got me back to sub second speed.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 create-index should run analyze after creating index 1096558279  
1002825217 https://github.com/simonw/datasette/issues/1583#issuecomment-1002825217 https://api.github.com/repos/simonw/datasette/issues/1583 IC_kwDOBm6k_c47xeYB fgregg 536941 2021-12-30T00:34:16Z 2021-12-30T00:34:16Z NONE

if that is not desirable, it might be good to document that users might want to set up a lifecycle rule to automatically delete these build artifacts. something like https://stackoverflow.com/questions/59937542/can-i-delete-container-images-from-google-cloud-storage-artifacts-bucket

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 consider adding deletion step of cloudbuild artifacts to gcloud publish 1090810196  
997128712 https://github.com/simonw/datasette/issues/1561#issuecomment-997128712 https://api.github.com/repos/simonw/datasette/issues/1561 IC_kwDOBm6k_c47bvoI fgregg 536941 2021-12-18T02:35:48Z 2021-12-18T02:35:48Z NONE

interesting! i love this feature. this + full caching with cloudflare is really super!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 add hash id to "_memory" url if hashed url mode is turned on and crossdb is also turned on 1082765654  
993078038 https://github.com/simonw/datasette/issues/526#issuecomment-993078038 https://api.github.com/repos/simonw/datasette/issues/526 IC_kwDOBm6k_c47MSsW fgregg 536941 2021-12-14T01:46:52Z 2021-12-14T01:46:52Z NONE

the nested query idea is very nice, and i stole if for my client side paginator. However, it won't do the right thing if the original query orders by random().

If you go the nested query route, maybe raise a 4XX status code if the query has such a clause?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Stream all results for arbitrary SQL and canned queries 459882902  
993014772 https://github.com/simonw/datasette/issues/1553#issuecomment-993014772 https://api.github.com/repos/simonw/datasette/issues/1553 IC_kwDOBm6k_c47MDP0 fgregg 536941 2021-12-13T23:46:18Z 2021-12-13T23:46:18Z NONE

these headers would also be relevant for json exports of custom queries

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 if csv export is truncated in non streaming mode set informative response header 1079111498  
992986587 https://github.com/simonw/datasette/issues/1553#issuecomment-992986587 https://api.github.com/repos/simonw/datasette/issues/1553 IC_kwDOBm6k_c47L8Xb fgregg 536941 2021-12-13T22:57:04Z 2021-12-13T22:57:04Z NONE

would also be good if the header said the what the max row limit was

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 if csv export is truncated in non streaming mode set informative response header 1079111498  
992971072 https://github.com/simonw/datasette/issues/526#issuecomment-992971072 https://api.github.com/repos/simonw/datasette/issues/526 IC_kwDOBm6k_c47L4lA fgregg 536941 2021-12-13T22:29:34Z 2021-12-13T22:29:34Z NONE

just came by to open this issue. would make my data analysis in observable a lot better!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Stream all results for arbitrary SQL and canned queries 459882902  
991754237 https://github.com/simonw/datasette/issues/1549#issuecomment-991754237 https://api.github.com/repos/simonw/datasette/issues/1549 IC_kwDOBm6k_c47HPf9 fgregg 536941 2021-12-11T19:14:39Z 2021-12-11T19:14:39Z NONE

that option is not available on custom queries.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Set content type for CSV so that browsers will attempt to download instead opening in the browser 1077620955  
991405755 https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991405755 https://api.github.com/repos/simonw/sqlite-utils/issues/353 IC_kwDOCGYnMM47F6a7 fgregg 536941 2021-12-11T01:38:29Z 2021-12-11T01:38:29Z NONE

wow! that's awesome! thanks so much, @simonw!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Allow passing a file of code to "sqlite-utils convert" 1077102934  
964205475 https://github.com/simonw/sqlite-utils/issues/26#issuecomment-964205475 https://api.github.com/repos/simonw/sqlite-utils/issues/26 IC_kwDOCGYnMM45eJuj fgregg 536941 2021-11-09T14:31:29Z 2021-11-09T14:31:29Z NONE

i was just reaching for a tool to do this this morning

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Mechanism for turning nested JSON into foreign keys / many-to-many 455486286  
954384496 https://github.com/simonw/datasette/pull/1495#issuecomment-954384496 https://api.github.com/repos/simonw/datasette/issues/1495 IC_kwDOBm6k_c444sBw fgregg 536941 2021-10-29T03:07:13Z 2021-10-29T03:07:13Z NONE

okay @simonw, made the requested changes. tests are running locally. i think this is ready for you to look at again.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Allow routes to have extra options 1033678984  
949604763 https://github.com/simonw/datasette/issues/1284#issuecomment-949604763 https://api.github.com/repos/simonw/datasette/issues/1284 IC_kwDOBm6k_c44mdGb fgregg 536941 2021-10-22T12:54:34Z 2021-10-22T12:54:34Z NONE

i'm going to take a swing at this today. we'll see.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Feature or Documentation Request: Individual table as home page template 845794436  
893114612 https://github.com/simonw/datasette/issues/1419#issuecomment-893114612 https://api.github.com/repos/simonw/datasette/issues/1419 IC_kwDOBm6k_c41O9j0 fgregg 536941 2021-08-05T02:29:06Z 2021-08-05T02:29:06Z NONE

there's a lot of complexity here, that's probably not worth addressing. i got what i needed by patching the dockerfile that cloudrun uses to install a newer version of sqlite.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `publish cloudrun` should deploy a more recent SQLite version 959710008  
892276385 https://github.com/simonw/datasette/issues/1419#issuecomment-892276385 https://api.github.com/repos/simonw/datasette/issues/1419 IC_kwDOBm6k_c41Lw6h fgregg 536941 2021-08-04T00:58:49Z 2021-08-04T00:58:49Z NONE 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `publish cloudrun` should deploy a more recent SQLite version 959710008  
884910320 https://github.com/simonw/datasette/issues/1401#issuecomment-884910320 https://api.github.com/repos/simonw/datasette/issues/1401 IC_kwDOBm6k_c40vqjw fgregg 536941 2021-07-22T13:26:01Z 2021-07-22T13:26:01Z NONE

ordered lists didn't work either, btw

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 unordered list is not rendering bullet points in description_html on database page 950664971  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);