home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

12 rows where "created_at" is on date 2020-04-16 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, created_at (date), updated_at (date)

issue 5

  • order_by mechanism 4
  • Repos have a big blob of JSON in the organization column 3
  • Topics are missing from repositories 2
  • Mechanism for forcing column-type, over-riding auto-detection 2
  • Improvements to demo instance 1

author_association 3

  • MEMBER 6
  • OWNER 5
  • NONE 1

user 2

  • simonw 11
  • metab0t 1
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
614843406 https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614843406 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 MDEyOklzc3VlQ29tbWVudDYxNDg0MzQwNg== simonw 9599 2020-04-16T19:11:53Z 2020-04-16T19:20:23Z MEMBER

This didn't quite work: the column type is incorrect, so the foreign key relationship isn't sticking:

https://github-to-sqlite.dogsheep.net/github/repos?organization=53015001

[organization] TEXT REFERENCES [users]([id]) - should be INTEGER.

The problem is that if the first repo inserted has no organization it's set to null, which sqlite-utils derives as a TEXT column.

One solution would be to create the column explicitly with a type, but this could get messy.

I think I want a new sqlite-utils feature for this instead.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Repos have a big blob of JSON in the organization column 601330277  
614846059 https://github.com/simonw/sqlite-utils/issues/100#issuecomment-614846059 https://api.github.com/repos/simonw/sqlite-utils/issues/100 MDEyOklzc3VlQ29tbWVudDYxNDg0NjA1OQ== simonw 9599 2020-04-16T19:17:56Z 2020-04-16T19:17:56Z OWNER

Reminder: the current list of arguments for those methods is: https://github.com/simonw/sqlite-utils/blob/13528faa817d79bc3900d3af7473300686b145d7/sqlite_utils/db.py#L951-L967

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for forcing column-type, over-riding auto-detection 601358649  
614845657 https://github.com/simonw/sqlite-utils/issues/100#issuecomment-614845657 https://api.github.com/repos/simonw/sqlite-utils/issues/100 MDEyOklzc3VlQ29tbWVudDYxNDg0NTY1Nw== simonw 9599 2020-04-16T19:17:05Z 2020-04-16T19:17:05Z OWNER

The syntax for explicitly creating a table looks like this: python db["cats"].create({ "id": int, "name": str, "weight": float, }, pk="id") So using a dictionary of column name -> Python type here would be good.

I think it becomes a new optional argument to .insert_all() and .insert() and .upsert_all() and .upsert(). Question is, what should that argument be called?

Options:

  • columns
  • types
  • type_overrides
  • extra_columns

I like the first two best. I'm leaning towards columns - it would serve two purposes: you can use it to ensure extra columns are created, and you can use it to specify the type on extra OR deteced columns.

Maybe something like this:

python db["authors"].insert_all( [{"id": 1, "name": "Sally", "score": "2"}], pk="id", not_null={"name", "score"}, defaults={"score": 1}, columns={"score": int, "weight": float} )

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for forcing column-type, over-riding auto-detection 601358649  
614831842 https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831842 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 MDEyOklzc3VlQ29tbWVudDYxNDgzMTg0Mg== simonw 9599 2020-04-16T18:48:18Z 2020-04-16T18:48:18Z MEMBER

I'm going to make organization another foreign key to the users table just in case it IS possible (maybe with GitHub Enterprise or similar?)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Repos have a big blob of JSON in the organization column 601330277  
614831451 https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831451 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 MDEyOklzc3VlQ29tbWVudDYxNDgzMTQ1MQ== simonw 9599 2020-04-16T18:47:25Z 2020-04-16T18:47:25Z MEMBER

Is it possible for a repo to have an owner that differs from its organization?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Repos have a big blob of JSON in the organization column 601330277  
614810417 https://github.com/dogsheep/github-to-sqlite/issues/25#issuecomment-614810417 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/25 MDEyOklzc3VlQ29tbWVudDYxNDgxMDQxNw== simonw 9599 2020-04-16T18:07:11Z 2020-04-16T18:07:11Z MEMBER

Turns out the main problem was #26 - now fixed.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Improvements to demo instance 601265023  
614795712 https://github.com/dogsheep/github-to-sqlite/issues/26#issuecomment-614795712 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26 MDEyOklzc3VlQ29tbWVudDYxNDc5NTcxMg== simonw 9599 2020-04-16T17:40:27Z 2020-04-16T17:40:27Z MEMBER

Aha! it was missing from the fetch_repo() function.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Topics are missing from repositories 601271612  
614794739 https://github.com/dogsheep/github-to-sqlite/issues/26#issuecomment-614794739 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26 MDEyOklzc3VlQ29tbWVudDYxNDc5NDczOQ== simonw 9599 2020-04-16T17:38:28Z 2020-04-16T17:38:28Z MEMBER

I'm already doing this here: https://github.com/dogsheep/github-to-sqlite/blob/c4aaa50e167cfa9021c7c94260bc3e89e10947bf/github_to_sqlite/utils.py#L246-L250

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Topics are missing from repositories 601271612  
614440032 https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614440032 https://api.github.com/repos/simonw/sqlite-utils/issues/76 MDEyOklzc3VlQ29tbWVudDYxNDQ0MDAzMg== metab0t 10501166 2020-04-16T06:23:29Z 2020-04-16T06:23:29Z NONE

Thanks for your hard work!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
order_by mechanism 549287310  
614400533 https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614400533 https://api.github.com/repos/simonw/sqlite-utils/issues/76 MDEyOklzc3VlQ29tbWVudDYxNDQwMDUzMw== simonw 9599 2020-04-16T03:51:26Z 2020-04-16T03:51:26Z OWNER

Documentation here: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#listing-rows

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
order_by mechanism 549287310  
614400454 https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614400454 https://api.github.com/repos/simonw/sqlite-utils/issues/76 MDEyOklzc3VlQ29tbWVudDYxNDQwMDQ1NA== simonw 9599 2020-04-16T03:51:01Z 2020-04-16T03:51:01Z OWNER

Released in 2.6

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
order_by mechanism 549287310  
614354219 https://github.com/simonw/sqlite-utils/issues/76#issuecomment-614354219 https://api.github.com/repos/simonw/sqlite-utils/issues/76 MDEyOklzc3VlQ29tbWVudDYxNDM1NDIxOQ== simonw 9599 2020-04-16T01:01:34Z 2020-04-16T01:01:34Z OWNER

I think a neat way to do this would be with an optional argument for .rows_where():

python rows = db["table"].rows_where("age > 10", order_by="age desc")

If you want everything you can use this:

python rows = db["table"].rows_where(order_by="age desc")

It's a tiny bit weird calling .rows_where() without a where clause, but I think it makes sense here - especially since .rows is a property that can't take any arguments - though under the hood it actually does this:

https://github.com/simonw/sqlite-utils/blob/ad6ac19470a67867b96cb4c086450b8e4e46bf02/sqlite_utils/db.py#L436-L443

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
order_by mechanism 549287310  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 476.199ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows