issue_comments

Table actions

16 rows where "updated_at" is on date 2020-11-29 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, body, reactions, created_at (date), updated_at (date)

author_association 4

user 3

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
735465708 https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735465708 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54 MDEyOklzc3VlQ29tbWVudDczNTQ2NTcwOA== simonw 9599 2020-11-29T22:08:46Z 2020-11-29T22:08:46Z MEMBER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 github-to-sqlite workflows command 753026003  
735464493 https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735464493 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54 MDEyOklzc3VlQ29tbWVudDczNTQ2NDQ5Mw== simonw 9599 2020-11-29T21:57:32Z 2020-11-29T21:57:32Z MEMBER

$ github-to-sqlite workflows github.db simonw/datasette dogsheep/github-to-sqlite

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 github-to-sqlite workflows command 753026003  
735464438 https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735464438 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54 MDEyOklzc3VlQ29tbWVudDczNTQ2NDQzOA== simonw 9599 2020-11-29T21:57:08Z 2020-11-29T21:57:08Z MEMBER

Inspired by this tweet from Michael Heap https://twitter.com/mheap/status/1333108608817631238

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 github-to-sqlite workflows command 753026003  
735447635 https://github.com/simonw/datasette/issues/1114#issuecomment-735447635 https://api.github.com/repos/simonw/datasette/issues/1114 MDEyOklzc3VlQ29tbWVudDczNTQ0NzYzNQ== simonw 9599 2020-11-29T20:16:32Z 2020-11-29T20:17:11Z OWNER

The new Docker container is pushed to Docker Hub too. Here's it in action:

% docker pull datasetteproject/datasette % docker run --rm -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette --version datasette, version 0.52.1 % docker run --rm -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/nps-spatialite.db --load-extension=spatialite INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 --load-extension=spatialite not working with datasetteproject/datasette docker image 752966476  
735447198 https://github.com/simonw/datasette/issues/1115#issuecomment-735447198 https://api.github.com/repos/simonw/datasette/issues/1115 MDEyOklzc3VlQ29tbWVudDczNTQ0NzE5OA== simonw 9599 2020-11-29T20:12:49Z 2020-11-29T20:12:49Z OWNER

``` $ datasette ../shapefile-to-sqlite/nps-spatialite.db Usage: datasette serve [OPTIONS] [FILES]...

Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module.

Try adding the --load-extension=spatialite option.

Read more: https://docs.datasette.io/en/stable/spatialite.html ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 SpatiaLite error could suggest --load-extension=spatialite 752995227  
735444858 https://github.com/simonw/datasette/issues/1099#issuecomment-735444858 https://api.github.com/repos/simonw/datasette/issues/1099 MDEyOklzc3VlQ29tbWVudDczNTQ0NDg1OA== simonw 9599 2020-11-29T19:51:58Z 2020-11-29T19:51:58Z OWNER

My fix in deb0be4ae56f191f121239b29e83dd53b62d6305 for #1098 was to have Datasette deliberately pretend that compound foreign keys don't exist: https://github.com/simonw/datasette/blob/deb0be4ae56f191f121239b29e83dd53b62d6305/datasette/utils/init.py#L470-L495

This workaround will need to be rethought to implement real support for them.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Support linking to compound foreign keys 743371103  
735444698 https://github.com/simonw/datasette/issues/1098#issuecomment-735444698 https://api.github.com/repos/simonw/datasette/issues/1098 MDEyOklzc3VlQ29tbWVudDczNTQ0NDY5OA== simonw 9599 2020-11-29T19:50:14Z 2020-11-29T19:50:14Z OWNER

Demo of the fix: https://latest.datasette.io/fixtures/foreign_key_references (the compound foreign key columns do not link to anything)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Foreign key links break for compound foreign keys 743370900  
735443654 https://github.com/simonw/datasette/issues/1098#issuecomment-735443654 https://api.github.com/repos/simonw/datasette/issues/1098 MDEyOklzc3VlQ29tbWVudDczNTQ0MzY1NA== simonw 9599 2020-11-29T19:41:01Z 2020-11-29T19:41:01Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Foreign key links break for compound foreign keys 743370900  
735443626 https://github.com/simonw/datasette/issues/1114#issuecomment-735443626 https://api.github.com/repos/simonw/datasette/issues/1114 MDEyOklzc3VlQ29tbWVudDczNTQ0MzYyNg== simonw 9599 2020-11-29T19:40:49Z 2020-11-29T19:40:49Z OWNER 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 --load-extension=spatialite not working with datasetteproject/datasette docker image 752966476  
735442226 https://github.com/simonw/datasette/issues/1098#issuecomment-735442226 https://api.github.com/repos/simonw/datasette/issues/1098 MDEyOklzc3VlQ29tbWVudDczNTQ0MjIyNg== simonw 9599 2020-11-29T19:28:04Z 2020-11-29T19:28:04Z OWNER

For the moment I'm going to solve this by teaching Datasette's internal introspection methods - in particular these ones - to ignore compound foreign keys entirely:

https://github.com/simonw/datasette/blob/e800ffcf7cc6a915eb554b369c654f87162575e5/datasette/utils/init.py#L470-L505

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Foreign key links break for compound foreign keys 743370900  
735440555 https://github.com/simonw/datasette/issues/123#issuecomment-735440555 https://api.github.com/repos/simonw/datasette/issues/123 MDEyOklzc3VlQ29tbWVudDczNTQ0MDU1NQ== jsancho-gpl 11912854 2020-11-29T19:12:30Z 2020-11-29T19:12:30Z NONE

datasette-connectors provides an API for making connectors for any file based database. For example, datasette-pytables is a connector for HDF5 files, so now is possible to use this type of files with Datasette.

It'd be nice if Datasette coud provide that API directly, for other file formats and for urls too.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Datasette serve should accept paths/URLs to CSVs and other file formats 275125561  
735436014 https://github.com/simonw/datasette/issues/1114#issuecomment-735436014 https://api.github.com/repos/simonw/datasette/issues/1114 MDEyOklzc3VlQ29tbWVudDczNTQzNjAxNA== danp 2182 2020-11-29T18:33:30Z 2020-11-29T18:33:30Z CONTRIBUTOR

Thank you!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 --load-extension=spatialite not working with datasetteproject/datasette docker image 752966476  
735429041 https://github.com/simonw/datasette/issues/1114#issuecomment-735429041 https://api.github.com/repos/simonw/datasette/issues/1114 MDEyOklzc3VlQ29tbWVudDczNTQyOTA0MQ== simonw 9599 2020-11-29T17:36:18Z 2020-11-29T17:36:18Z OWNER

This is a great catch, thanks.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 --load-extension=spatialite not working with datasetteproject/datasette docker image 752966476  
735356882 https://github.com/simonw/datasette/issues/1102#issuecomment-735356882 https://api.github.com/repos/simonw/datasette/issues/1102 MDEyOklzc3VlQ29tbWVudDczNTM1Njg4Mg== simonw 9599 2020-11-29T07:47:22Z 2020-11-29T07:47:22Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Plugin testing docs should show datasette.client 749289611  
735320736 https://github.com/simonw/datasette/issues/1111#issuecomment-735320736 https://api.github.com/repos/simonw/datasette/issues/1111 MDEyOklzc3VlQ29tbWVudDczNTMyMDczNg== simonw 9599 2020-11-29T02:46:23Z 2020-11-29T02:46:23Z OWNER

This is a really useful bug report, thanks! I agree: more aggressive timeouts on table counts sounds like the right solution here.

I've learned that avoiding count(*) is crucial for handling these larger databases. Datasette has been getting better about that over time but there are still some edge-cases.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Accessing a database's `.json` is slow for very large SQLite files 751195017  
735320499 https://github.com/simonw/datasette/issues/1108#issuecomment-735320499 https://api.github.com/repos/simonw/datasette/issues/1108 MDEyOklzc3VlQ29tbWVudDczNTMyMDQ5OQ== simonw 9599 2020-11-29T02:42:42Z 2020-11-29T02:42:42Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Configure /en/stable/config.html redirect when I ship 0.52 750087350  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);