id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
1149402080,PR_kwDODFdgUs4zaUta,70,scrape-dependents: enable paging through package menu option if present,36061055,open,0,,,0,2022-02-24T15:07:25Z,2022-02-24T15:07:25Z,,FIRST_TIME_CONTRIBUTOR,dogsheep/github-to-sqlite/pulls/70,Some repos organize network dependents by a Package toggle. This PR adds the ability to page through those options and scrape underlying dependents.,207052882,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/70/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1071071397,I_kwDODFdgUs4_10Cl,69,View that combines issues and issue comments,9599,open,0,,,1,2021-12-04T00:34:33Z,2021-12-04T00:34:52Z,,MEMBER,,I want to see a reverse chronologically ordered interface onto both issues and comments - essentially a unified log of comments and issues opened across one or multiple projects.,207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1013506559,PR_kwDODFdgUs4skaNS,68,Add support for retrieving teams / members,68329,open,0,,,0,2021-10-01T15:55:02Z,2021-10-01T15:59:53Z,,FIRST_TIME_CONTRIBUTOR,dogsheep/github-to-sqlite/pulls/68,Adds a method for retrieving all the teams within an organisation and all the members in those teams. The latter is stored as a join table `team_members` beteween `teams` and `users`.,207052882,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/68/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
975161924,MDExOlB1bGxSZXF1ZXN0NzE2MzU3OTgy,66,Add --merged-by flag to pull-requests sub command,30531572,open,0,,,1,2021-08-20T00:57:55Z,2021-09-28T21:50:31Z,,FIRST_TIME_CONTRIBUTOR,dogsheep/github-to-sqlite/pulls/66,"## Description

Proposing a solution to the API limitation for `merged_by` in pull_requests. Specifically the following called out in the readme:

```
Note that the merged_by column on the pull_requests table will only be populated for pull requests that are loaded using the --pull-request option - the GitHub API does not return this field for pull requests that are loaded in bulk.
```

This approach might cause larger repos to hit rate limits called out in https://github.com/dogsheep/github-to-sqlite/issues/51 but seems to work well in the repos I tested and included below.

## Old Behavior
- Had to list out the pull-requests individually via multiple `--pull-request` flags

## New Behavior

- `--merged-by` flag for getting 'merge_by' information out of pull-requests without having to specify individual PR numbers.

# Testing

Picking some repo that has more than one merger (datasette only has 1 😉 )

```
$ github-to-sqlite pull-requests ./github.db opnsense/tools --merged-by
$ echo ""select id, url, merged_by from pull_requests;"" | sqlite3 ./github.db 
83533612|https://github.com/opnsense/tools/pull/39|1915288
102632885|https://github.com/opnsense/tools/pull/43|1915288
149114810|https://github.com/opnsense/tools/pull/57|1915288
160394495|https://github.com/opnsense/tools/pull/64|1915288
163308408|https://github.com/opnsense/tools/pull/67|1915288
169723264|https://github.com/opnsense/tools/pull/69|1915288
171381422|https://github.com/opnsense/tools/pull/72|1915288
179938195|https://github.com/opnsense/tools/pull/77|1915288
196233824|https://github.com/opnsense/tools/pull/82|1915288
215289964|https://github.com/opnsense/tools/pull/93|
219696100|https://github.com/opnsense/tools/pull/97|1915288
223664843|https://github.com/opnsense/tools/pull/99|
228446172|https://github.com/opnsense/tools/pull/103|1915288
238930434|https://github.com/opnsense/tools/pull/110|1915288
255507110|https://github.com/opnsense/tools/pull/119|1915288
255980675|https://github.com/opnsense/tools/pull/120|1915288
261906770|https://github.com/opnsense/tools/pull/125|
263800503|https://github.com/opnsense/tools/pull/127|1915288
264038685|https://github.com/opnsense/tools/pull/128|1915288
264696704|https://github.com/opnsense/tools/pull/129|1915288
266660547|https://github.com/opnsense/tools/pull/130|1915288
273120409|https://github.com/opnsense/tools/pull/133|1915288
274370803|https://github.com/opnsense/tools/pull/135|
276600629|https://github.com/opnsense/tools/pull/139|
277303655|https://github.com/opnsense/tools/pull/141|1915288
293033714|https://github.com/opnsense/tools/pull/145|
294827649|https://github.com/opnsense/tools/pull/146|
295140008|https://github.com/opnsense/tools/pull/147|1915288
305690829|https://github.com/opnsense/tools/pull/150|9783985
307077931|https://github.com/opnsense/tools/pull/152|1915288
321782100|https://github.com/opnsense/tools/pull/155|
337265672|https://github.com/opnsense/tools/pull/160|
337267484|https://github.com/opnsense/tools/pull/161|1915288
368251763|https://github.com/opnsense/tools/pull/169|
428262505|https://github.com/opnsense/tools/pull/181|
437557011|https://github.com/opnsense/tools/pull/182|1915288
447079893|https://github.com/opnsense/tools/pull/185|
461822092|https://github.com/opnsense/tools/pull/191|
463290142|https://github.com/opnsense/tools/pull/193|1915288
470112962|https://github.com/opnsense/tools/pull/194|1915288
472644649|https://github.com/opnsense/tools/pull/195|1915288
488696898|https://github.com/opnsense/tools/pull/198|
513289902|https://github.com/opnsense/tools/pull/201|
522530265|https://github.com/opnsense/tools/pull/203|
564443347|https://github.com/opnsense/tools/pull/213|
597579516|https://github.com/opnsense/tools/pull/220|1915288
602860357|https://github.com/opnsense/tools/pull/221|1915288
608744738|https://github.com/opnsense/tools/pull/222|1915288
623279673|https://github.com/opnsense/tools/pull/228|1915288
664656182|https://github.com/opnsense/tools/pull/233|
664781786|https://github.com/opnsense/tools/pull/234|1915288
670683636|https://github.com/opnsense/tools/pull/235|1915288
683150764|https://github.com/opnsense/tools/pull/237|
685016233|https://github.com/opnsense/tools/pull/238|
687099825|https://github.com/opnsense/tools/pull/239|1915288
715705652|https://github.com/opnsense/tools/pull/244|1915288
715721248|https://github.com/opnsense/tools/pull/245|1915288
```
`userid` are now present for those PRs that were merged.

Without the flag the `merged_by` behavior remains missing as expected when get PRs bulk:

```
$ github-to-sqlite pull-requests ./github.db opnsense/tools
$ echo ""select id, url, merged_by from pull_requests;"" | sqlite3 ./github.db 
83533612|https://github.com/opnsense/tools/pull/39|
102632885|https://github.com/opnsense/tools/pull/43|
149114810|https://github.com/opnsense/tools/pull/57|
160394495|https://github.com/opnsense/tools/pull/64|
163308408|https://github.com/opnsense/tools/pull/67|
169723264|https://github.com/opnsense/tools/pull/69|
171381422|https://github.com/opnsense/tools/pull/72|
179938195|https://github.com/opnsense/tools/pull/77|
196233824|https://github.com/opnsense/tools/pull/82|
215289964|https://github.com/opnsense/tools/pull/93|
219696100|https://github.com/opnsense/tools/pull/97|
223664843|https://github.com/opnsense/tools/pull/99|
228446172|https://github.com/opnsense/tools/pull/103|
238930434|https://github.com/opnsense/tools/pull/110|
255507110|https://github.com/opnsense/tools/pull/119|
255980675|https://github.com/opnsense/tools/pull/120|
261906770|https://github.com/opnsense/tools/pull/125|
263800503|https://github.com/opnsense/tools/pull/127|
264038685|https://github.com/opnsense/tools/pull/128|
264696704|https://github.com/opnsense/tools/pull/129|
266660547|https://github.com/opnsense/tools/pull/130|
273120409|https://github.com/opnsense/tools/pull/133|
274370803|https://github.com/opnsense/tools/pull/135|
276600629|https://github.com/opnsense/tools/pull/139|
277303655|https://github.com/opnsense/tools/pull/141|
293033714|https://github.com/opnsense/tools/pull/145|
294827649|https://github.com/opnsense/tools/pull/146|
295140008|https://github.com/opnsense/tools/pull/147|
305690829|https://github.com/opnsense/tools/pull/150|
307077931|https://github.com/opnsense/tools/pull/152|
321782100|https://github.com/opnsense/tools/pull/155|
337265672|https://github.com/opnsense/tools/pull/160|
337267484|https://github.com/opnsense/tools/pull/161|
368251763|https://github.com/opnsense/tools/pull/169|
428262505|https://github.com/opnsense/tools/pull/181|
437557011|https://github.com/opnsense/tools/pull/182|
447079893|https://github.com/opnsense/tools/pull/185|
461822092|https://github.com/opnsense/tools/pull/191|
463290142|https://github.com/opnsense/tools/pull/193|
470112962|https://github.com/opnsense/tools/pull/194|
472644649|https://github.com/opnsense/tools/pull/195|
488696898|https://github.com/opnsense/tools/pull/198|
513289902|https://github.com/opnsense/tools/pull/201|
522530265|https://github.com/opnsense/tools/pull/203|
564443347|https://github.com/opnsense/tools/pull/213|
597579516|https://github.com/opnsense/tools/pull/220|
602860357|https://github.com/opnsense/tools/pull/221|
608744738|https://github.com/opnsense/tools/pull/222|
623279673|https://github.com/opnsense/tools/pull/228|
664656182|https://github.com/opnsense/tools/pull/233|
664781786|https://github.com/opnsense/tools/pull/234|
670683636|https://github.com/opnsense/tools/pull/235|
683150764|https://github.com/opnsense/tools/pull/237|
685016233|https://github.com/opnsense/tools/pull/238|
687099825|https://github.com/opnsense/tools/pull/239|
715705652|https://github.com/opnsense/tools/pull/244|
715721248|https://github.com/opnsense/tools/pull/245|
```

Individual PRs passed via `--pull-request` flag behaves as expected (unchanged):

```
$ github-to-sqlite pull-requests ./github.db opnsense/tools --pull-request 39 --pull-request 237
$ echo ""select id, url, merged_by from pull_requests;"" | sqlite3 ./github.db
83533612|https://github.com/opnsense/tools/pull/39|1915288
683150764|https://github.com/opnsense/tools/pull/237|
```
> Picking 1 PR that has a merged_by (39) and one that does not (237)",207052882,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/66/reactions"", ""total_count"": 3, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",0,
981690086,MDExOlB1bGxSZXF1ZXN0NzIxNjg2NzIx,67,Replacing step ID key with step_id,16374374,open,0,,,0,2021-08-28T01:26:41Z,2021-08-28T01:27:00Z,,FIRST_TIME_CONTRIBUTOR,dogsheep/github-to-sqlite/pulls/67,"Workflows that have an `id` in any step result in the following error when running `workflows`:

e.g.`github-to-sqlite workflows github.db nixos/nixpkgs`

```Traceback (most recent call last):
  File ""/usr/local/bin/github-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1137, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1062, in main
    rv = self.invoke(ctx)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1668, in invoke```Traceback (most recent call last):
  File ""/usr/local/bin/github-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1137, in __call__
    return self.main(*args, **kwargs)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1062, in main
    rv = self.invoke(ctx)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/usr/local/lib/python3.8/dist-packages/click/core.py"", line 763, in invoke
    return __callback(*args, **kwargs)
  File ""/usr/local/lib/python3.8/dist-packages/github_to_sqlite/cli.py"", line 601, in workflows
    utils.save_workflow(db, repo_id, filename, content)
  File ""/usr/local/lib/python3.8/dist-packages/github_to_sqlite/utils.py"", line 865, in save_workflow
    db[""steps""].insert_all(
  File ""/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py"", line 2596, in insert_all
    self.insert_chunk(
  File ""/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py"", line 2378, in insert_chunk
    result = self.db.execute(query, params)
  File ""/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py"", line 419, in execute
    return self.conn.execute(sql, parameters)
sqlite3.IntegrityError: datatype mismatch
```

 - [Information about the ID key in a step for GHA](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsid)
 - [An example workflow from a public repo](https://github.com/NixOS/nixpkgs/blob/b4cc66827745e525ce7bb54659845ac89788a597/.github/workflows/direct-push.yml#L16)

# Changes
I'm proposing that the key for `id` in step is replaced with `step_id` so that it no longer interferes with the table `id` for tracking the record.

Special thanks to @sarcasticadmin @egiffen and @ruebenramirez for helping a bit on this 😄 ",207052882,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/67/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
920636216,MDU6SXNzdWU5MjA2MzYyMTY=,64,"feature: support ""events""",231498,open,0,,,5,2021-06-14T17:42:49Z,2021-06-15T00:48:37Z,,NONE,,"the GitHub API provides the ability to fetch all events for a given user, organization, or repository: https://docs.github.com/en/rest/reference/activity#list-events-for-the-authenticated-user

this would allow users to export all of the issue comments, new issues, etc. that they created. something which is currently missing from the GitHub takeout exports.",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
897212458,MDU6SXNzdWU4OTcyMTI0NTg=,63,Ability to fetch commits from branches other than the default,9599,open,0,,,0,2021-05-20T17:58:08Z,2021-05-20T17:58:08Z,,MEMBER,,This tool is currently almost entirely ignorant of the concept of branches. One example: you can't retrieve commits from any branch other than the default (usually main).,207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/63/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
797097140,MDU6SXNzdWU3OTcwOTcxNDA=,60,Use Data from SQLite in other commands,22578954,open,0,,,3,2021-01-29T18:35:52Z,2021-02-12T18:29:43Z,,CONTRIBUTOR,,"As a total beginner here how could you access data from the sqlite table to run other commands.

What I am thinking is I want to get all the repos in an organization then using the repo list pull all the commit messages for each repo. 

I love this project by the way!",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
797784080,MDU6SXNzdWU3OTc3ODQwODA=,62,Stargazers and workflows commands always require an auth file when using GITHUB_TOKEN ,631242,open,0,,,0,2021-01-31T18:56:05Z,2021-01-31T18:56:05Z,,CONTRIBUTOR,,"Requested fix in https://github.com/dogsheep/github-to-sqlite/pull/59

The stargazers and workflows commands always require an auth file, even when using a `GITHUB_TOKEN`.  Other commands don't require the auth file.

",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/62/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
753000405,MDU6SXNzdWU3NTMwMDA0MDU=,53,Command for fetching file contents,9599,open,0,,,1,2020-11-29T20:31:04Z,2020-11-30T00:36:09Z,,MEMBER,,"Something like this:

    github-to-sqlite files github.db simonw/datasette

This would fetch all files from the `main` branch into a `files` table.

Additional options could handle things like pulling files from a branch or tag, or just pulling files that match a specific glob or that exist in a specific directory.",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/53/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
703218756,MDU6SXNzdWU3MDMyMTg3NTY=,50,Commands for making authenticated API calls,9599,open,0,,,7,2020-09-17T02:39:07Z,2020-10-19T05:01:29Z,,MEMBER,,"Similar to `twitter-to-sqlite fetch`, see https://github.com/dogsheep/twitter-to-sqlite/issues/51",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
703216044,MDU6SXNzdWU3MDMyMTYwNDQ=,49,Feature: gists and starred gists,9599,open,0,,,0,2020-09-17T02:30:52Z,2020-09-17T02:30:52Z,,MEMBER,,https://developer.github.com/v3/gists/#list-starred-gists,207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/49/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
530491074,MDU6SXNzdWU1MzA0OTEwNzQ=,14,Command for importing events,9599,open,0,,,3,2019-11-29T21:28:58Z,2020-04-14T19:38:34Z,,MEMBER,,"Eg from https://api.github.com/users/simonw/events

Docs here: https://developer.github.com/v3/activity/events/#list-events-performed-by-a-user",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
599776345,MDU6SXNzdWU1OTk3NzYzNDU=,24,Feature idea: github-to-sqlite everything ...,9599,open,0,,,0,2020-04-14T18:34:00Z,2020-04-14T18:34:00Z,,MEMBER,,"At the moment if you want to pull all your repos, issues, issues comments etc you have to do it with a sequence of separate commands.

Consider adding a `everything` or `all` command which fetches everything that the tool knows how to fetch, and is designed to be run on a cron in a way that fetches just new stuff each time.",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/24/reactions"", ""total_count"": 7, ""+1"": 7, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,