issue_comments

@simonw rebased against latest, please let me know if i should drop this PR.

@simonw is this PR of interest to you?

@simonw -- i've created an omega-query that fetched most of what was interesting to me for a single user.

found by poking around in the "Explorer" tab in https://docs.github.com/en/graphql/overview/explorer

note: pagination is still required via first and last but it seems to allow unlimited history.

query MyQuery { __typename user(login: "<user>") { id pinnedItems(first: 100) { edges { node } } pullRequests(first: 100) { nodes { body title state createdAt } } createdAt issues(first: 100) { pageInfo { endCursor startCursor } nodes { title url createdAt body } } issueComments(first: 100) { edges { node { id updatedAt url body } } } repositories(first: 100) { nodes { createdAt description parent { name } pinnedIssues(first: 100) { edges { node { id } } } pinnedDiscussions(first: 100) { edges { node { id } } } } } starredRepositories(first: 100) { edges { node { id } } } } }

it looks like the v4 GraphQL API is the only way to get data beyond 90 days from GitHub.

this is significant change, but may be worth considering in the future.

i have a basic working version at https://github.com/khimaros/github-to-sqlite

this can be tested with github-to-sqlite events.db khimaros/events

caveat: the GitHub API doesn't seem to provide a complete history of events.

it looks like almost all of the memory consumption is coming from json.load().

another direction here may be to use the new "Semantic Location History" data which is already broken down by year and month.

it also provides much more interesting data, such as estimated address, form of travel, etc.