html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/dogsheep/dogsheep-photos/issues/6#issuecomment-615983393,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6,615983393,MDEyOklzc3VlQ29tbWVudDYxNTk4MzM5Mw==,9599,simonw,2020-04-18T23:53:10Z,2020-04-18T23:53:10Z,MEMBER,"```
$ photos-to-sqlite upload photos3.db ~/Pictures/Photos\ Library.photoslibrary/Masters/2020
Uploading 2.09 GB [##----------------------------------] 6% 00:36:37
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602575575,Add progress bar to upload command,
https://github.com/dogsheep/dogsheep-photos/issues/6#issuecomment-615979923,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6,615979923,MDEyOklzc3VlQ29tbWVudDYxNTk3OTkyMw==,9599,simonw,2020-04-18T23:36:02Z,2020-04-18T23:36:02Z,MEMBER,"I'll use a Click progress bar. To do this I need to first calculate the sum number of bytes in the photos that are going to be uploaded, then run the upload.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602575575,Add progress bar to upload command,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615957385,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615957385,MDEyOklzc3VlQ29tbWVudDYxNTk1NzM4NQ==,9599,simonw,2020-04-18T21:56:16Z,2020-04-18T21:58:11Z,MEMBER,Got this working! I'll do EXIF in a separate ticket #3.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/5#issuecomment-615949574,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/5,615949574,MDEyOklzc3VlQ29tbWVudDYxNTk0OTU3NA==,9599,simonw,2020-04-18T21:06:07Z,2020-04-18T21:06:07Z,MEMBER,"```
$ photos-to-sqlite s3-auth
Create S3 credentials and paste them here:
Access key ID: xxx
Secret access key: yyy
$ cat auth.json
{
""access_key_id"": ""xxx"",
""secret_access_key"": ""yyy""
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602551638,photos-to-sqlite s3-auth command,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615948102,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615948102,MDEyOklzc3VlQ29tbWVudDYxNTk0ODEwMg==,9599,simonw,2020-04-18T20:56:59Z,2020-04-18T20:56:59Z,MEMBER,"I'm going to start with this:
`photos-to-sqlite upload photos.db ~/path/to/directory`
This will scan the provided directory (and all sub-directories) for image files. It will then:
* Calculate a sha256 of the contents of that file
* Upload the file to a key that's `sha256.jpg` or `.heic`
* Upload a `sha256.json` file with the original path to the image
* Add that image to a `uploads` table in `photos.db`
Stretch goal: grab the EXIF data and include that in the `.json` upload AND the `uploads` database table.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947370,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615947370,MDEyOklzc3VlQ29tbWVudDYxNTk0NzM3MA==,9599,simonw,2020-04-18T20:52:13Z,2020-04-18T20:52:13Z,MEMBER,"This is great! I now have a key that can upload photos, and a separate key that can download photos OR generate signed URLs to access those photos.
Next step: a script that starts uploading my photos.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947229,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615947229,MDEyOklzc3VlQ29tbWVudDYxNTk0NzIyOQ==,9599,simonw,2020-04-18T20:51:26Z,2020-04-18T20:51:26Z,MEMBER,"Running the upload again like this resulted in the correct content-type:
```python
client.upload_file(
""/Users/simonw/Desktop/this_is_fine.jpg"",
""dogsheep-photos-simon"",
""this_is_fine.jpg"",
ExtraArgs={
""ContentType"": ""image/jpeg""
}
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615946537,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615946537,MDEyOklzc3VlQ29tbWVudDYxNTk0NjUzNw==,9599,simonw,2020-04-18T20:48:13Z,2020-04-18T20:48:13Z,MEMBER,"How about generating a signed URL?
```python
read_client.generate_presigned_url(
""get_object"",
Params={
""Bucket"": ""dogsheep-photos-simon"",
""Key"": ""this_is_fine.jpg"",
},
ExpiresIn=600
)
```
Gave me https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398
Which does this:
```
~ $ curl -i 'https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'
HTTP/1.1 307 Temporary Redirect
x-amz-bucket-region: us-west-1
x-amz-request-id: E78CD859AEE21D33
x-amz-id-2: 648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=
Location: https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Sat, 18 Apr 2020 20:47:21 GMT
Server: AmazonS3
TemporaryRedirect
Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.dogsheep-photos-simon.s3-us-west-1.amazonaws.comdogsheep-photos-simonE78CD859AEE21D33648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=~ $
```
So it redirects to another URL... which returns this:
```
~ $ curl -i 'https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'
HTTP/1.1 200 OK
x-amz-id-2: XafOl6mswj3yz0GJC9+Ptot1ll5sROVwqsMc10CUUfgpaUANTdIx2GhnONb5d1GVFJ6wlS2j3UY=
x-amz-request-id: 258387C180411AFE
Date: Sat, 18 Apr 2020 20:47:52 GMT
Last-Modified: Sat, 18 Apr 2020 20:37:35 GMT
ETag: ""ee04081c3182a44a1c6944e94012e977""
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 53072
Server: AmazonS3
????JFIF??C
```
So that worked! It did come back with `Content-Type: binary/octet-stream` though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615945056,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615945056,MDEyOklzc3VlQ29tbWVudDYxNTk0NTA1Ng==,9599,simonw,2020-04-18T20:42:41Z,2020-04-18T20:42:41Z,MEMBER,"But... `list_objects` failed for both of my keys (read and write):
![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670798-75c41780-817a-11ea-9907-2cbc4a2e497c.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615944806,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615944806,MDEyOklzc3VlQ29tbWVudDYxNTk0NDgwNg==,9599,simonw,2020-04-18T20:41:39Z,2020-04-18T20:41:39Z,MEMBER,"This worked!
![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670712-d868e380-8179-11ea-82a5-5dfd17356113.png)
And this worked:
![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670777-50370e00-817a-11ea-83cd-18ebf5702878.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615942116,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615942116,MDEyOklzc3VlQ29tbWVudDYxNTk0MjExNg==,9599,simonw,2020-04-18T20:30:56Z,2020-04-18T20:30:56Z,MEMBER,"Next step: attempt a programmatic upload using the `dogsheep-photos-simon-read-write` credentials from a Jupyter notebook.
Also attempt a programmatic bucket listing and read using `dogsheep-photos-simon-read` credentials.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615941746,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615941746,MDEyOklzc3VlQ29tbWVudDYxNTk0MTc0Ng==,9599,simonw,2020-04-18T20:29:36Z,2020-04-18T20:29:36Z,MEMBER,"I'm going to create another user just for Transmit, with full S3 access.
name: `dogsheep-photos-simon-s3-all-access`
Rather than creating a group for that user, I'm trying the ""Attach existing policies directly"" option:
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79670182-03513880-8176-11ea-811a-c80aefb4538a.png)
That user DID work with Transmit. I uploaded a test HEIC image. I used Transmit to copy a signed URL for it.
```
~ $ curl -i 'https://dogsheep-photos-simon.s3.us-west-1.amazonaws.com/IMG_7195.HEIC?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWXFXAI...' | head -n 100
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK
x-amz-id-2: gBOCYqZfbNAnv0R/uJ++qm2NbW5SgD4TapgF9RQjzzeDIThcCz/BkKU+YoxlG4NJHlcmMgAHyh4=
x-amz-request-id: C2FE7FCC3BD53A84
Date: Sat, 18 Apr 2020 20:28:54 GMT
Last-Modified: Sat, 18 Apr 2020 20:13:49 GMT
ETag: ""fe3e081239a123ef745517878c53b854""
Accept-Ranges: bytes
Content-Type: image/heic
Content-Length: 1913097
Server: AmazonS3
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615936880,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615936880,MDEyOklzc3VlQ29tbWVudDYxNTkzNjg4MA==,9599,simonw,2020-04-18T20:04:31Z,2020-04-18T20:04:31Z,MEMBER,"Next step: create two IAM users, one for each of those groups.
https://console.aws.amazon.com/iam/home#/users$new?step=details
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669931-1bc05380-8174-11ea-9657-0e0c6a692d42.png)
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669941-27137f00-8174-11ea-8ce7-249f0d4f96f6.png)
I copied the keys into a secure note in 1password.
Couldn't get into Transmit with them though! https://library.panic.com/transmit/transmit5/iam-roles/ may help.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615935577,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615935577,MDEyOklzc3VlQ29tbWVudDYxNTkzNTU3Nw==,9599,simonw,2020-04-18T19:54:59Z,2020-04-18T19:55:30Z,MEMBER,"Creating IAM groups called `dogsheep-photos-simon-read-write` and `dogsheep-photos-simon-read`: https://console.aws.amazon.com/iam/home#/groups - I created them with no attached policies.
Now I can attach an ""inline policy"" to each one. For the read-write group I go here:
https://console.aws.amazon.com/iam/home#/groups/dogsheep-photos-simon-read-write
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669703-2d086080-8172-11ea-9597-83e0b155193e.png)
Example policies are here: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html
For the read-write one I went with:
```json
{
""Version"": ""2012-10-17"",
""Statement"": [
{
""Effect"": ""Allow"",
""Action"": ""s3:*"",
""Resource"": [
""arn:aws:s3:::dogsheep-photos-simon/*""
]
}
]
}
```
For the read-only policy I'm going to guess that this is appropriate:
```json
{
""Version"": ""2012-10-17"",
""Statement"": [
{
""Effect"": ""Allow"",
""Action"": [
""s3:GetObject*"",
""s3:ListBucket""
],
""Resource"": [
""arn:aws:s3:::dogsheep-photos-simon/*""
]
}
]
}
```
I tried the policy simulator to test this out: https://policysim.aws.amazon.com/home/index.jsp?#groups/dogsheep-photos-simon-read - this worked:
![IAM_Policy_Simulator](https://user-images.githubusercontent.com/9599/79669893-cd12b980-8173-11ea-8dfb-5660ce3652da.png)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615933273,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615933273,MDEyOklzc3VlQ29tbWVudDYxNTkzMzI3Mw==,9599,simonw,2020-04-18T19:37:33Z,2020-04-18T19:37:33Z,MEMBER,"https://console.aws.amazon.com/s3/bucket/create?region=us-west-1
![S3_Management_Console](https://user-images.githubusercontent.com/9599/79669552-33e2a380-8171-11ea-9ab5-5785d34f652a.png)
I created it with no public read-write access. I plan to use signed URLs via a transforming proxy to access images for display on the web.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932204,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615932204,MDEyOklzc3VlQ29tbWVudDYxNTkzMjIwNA==,9599,simonw,2020-04-18T19:29:22Z,2020-04-18T19:34:44Z,MEMBER,I'm going to call my bucket `dogsheep-photos-simon`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932007,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615932007,MDEyOklzc3VlQ29tbWVudDYxNTkzMjAwNw==,9599,simonw,2020-04-18T19:27:55Z,2020-04-18T19:27:55Z,MEMBER,"Research thread: https://twitter.com/simonw/status/1249049694984011776
> I want to build some software that lets people store their own data in their own S3 bucket, but if possible I'd like not to have to teach people the incantations needed to get their bucket setup and minimum-permission credentials figures out
https://testdriven.io/blog/storing-django-static-and-media-files-on-amazon-s3/ looks useful","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/2#issuecomment-615931488,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2,615931488,MDEyOklzc3VlQ29tbWVudDYxNTkzMTQ4OA==,9599,simonw,2020-04-18T19:24:02Z,2020-04-18T19:24:02Z,MEMBER,I made a start on this last week with a https://github.com/simonw/heic-to-jpeg proxy.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533352,Ability to convert HEIC images to JPEG,
https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615886206,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28,615886206,MDEyOklzc3VlQ29tbWVudDYxNTg4NjIwNg==,9599,simonw,2020-04-18T15:04:59Z,2020-04-18T15:04:59Z,MEMBER,"Demo: https://github-to-sqlite.dogsheep.net/github/contributors
Documentation: https://github.com/dogsheep/github-to-sqlite/blob/13f8868fb5efa01c263b24f6dd91c617e6e938e1/README.md#fetching-contributors-to-a-repository ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601333634,Pull repository contributors,
https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615883687,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28,615883687,MDEyOklzc3VlQ29tbWVudDYxNTg4MzY4Nw==,9599,simonw,2020-04-18T14:49:58Z,2020-04-18T14:49:58Z,MEMBER,"That happened trying to pull contributors for `dogsheep/beta` - an empty repository. Turns out it was returning a `204 no content`:
```
~ $ curl -i 'https://api.github.com/repos/dogsheep/beta/contributors'
HTTP/1.1 204 No Content
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601333634,Pull repository contributors,
https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615883040,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28,615883040,MDEyOklzc3VlQ29tbWVudDYxNTg4MzA0MA==,9599,simonw,2020-04-18T14:45:38Z,2020-04-18T14:45:38Z,MEMBER,"```
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
return self.main(*args, **kwargs)
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 782, in main
rv = self.invoke(ctx)
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
return callback(*args, **kwargs)
File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 219, in contributors
utils.save_contributors(db, contributors, repo_full[""id""])
File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 354, in save_contributors
for contributor in contributors:
File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 228, in fetch_contributors
for contributors in paginate(url, headers):
File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 286, in paginate
data = response.json()
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/requests/models.py"", line 898, in json
return complexjson.loads(self.text, **kwargs)
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/json/__init__.py"", line 357, in loads
return _default_decoder.decode(s)
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/json/decoder.py"", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/json/decoder.py"", line 355, in raw_decode
raise JSONDecodeError(""Expecting value"", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601333634,Pull repository contributors,
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-615519409,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27,615519409,MDEyOklzc3VlQ29tbWVudDYxNTUxOTQwOQ==,9599,simonw,2020-04-18T00:19:16Z,2020-04-18T00:19:16Z,MEMBER,"```
$ github-to-sqlite repos b.db dogsheep
$ sqlite3 b.db '.schema repos'
CREATE TABLE [repos] (
[id] INTEGER PRIMARY KEY,
...
[permissions] TEXT,
[organization] INTEGER REFERENCES [users]([id]),
FOREIGN KEY(license) REFERENCES licenses(key)
);
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601330277,Repos have a big blob of JSON in the organization column,
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-615518606,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27,615518606,MDEyOklzc3VlQ29tbWVudDYxNTUxODYwNg==,9599,simonw,2020-04-18T00:14:32Z,2020-04-18T00:14:32Z,MEMBER,https://github.com/simonw/sqlite-utils/issues/100 is done and released in sqlite-utils 2.7.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601330277,Repos have a big blob of JSON in the organization column,
https://github.com/simonw/sqlite-utils/issues/89#issuecomment-615515867,https://api.github.com/repos/simonw/sqlite-utils/issues/89,615515867,MDEyOklzc3VlQ29tbWVudDYxNTUxNTg2Nw==,9599,simonw,2020-04-18T00:00:41Z,2020-04-18T00:00:41Z,OWNER,"Yes pleas, I'd love to see that pull request!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",573578548,Ability to customize columns used by extracts= feature,