html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615957385,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615957385,MDEyOklzc3VlQ29tbWVudDYxNTk1NzM4NQ==,9599,2020-04-18T21:56:16Z,2020-04-18T21:58:11Z,MEMBER,Got this working! I'll do EXIF in a separate ticket #3.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615948102,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615948102,MDEyOklzc3VlQ29tbWVudDYxNTk0ODEwMg==,9599,2020-04-18T20:56:59Z,2020-04-18T20:56:59Z,MEMBER,"I'm going to start with this:
`photos-to-sqlite upload photos.db ~/path/to/directory`
This will scan the provided directory (and all sub-directories) for image files. It will then:
* Calculate a sha256 of the contents of that file
* Upload the file to a key that's `sha256.jpg` or `.heic`
* Upload a `sha256.json` file with the original path to the image
* Add that image to a `uploads` table in `photos.db`
Stretch goal: grab the EXIF data and include that in the `.json` upload AND the `uploads` database table.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947370,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615947370,MDEyOklzc3VlQ29tbWVudDYxNTk0NzM3MA==,9599,2020-04-18T20:52:13Z,2020-04-18T20:52:13Z,MEMBER,"This is great! I now have a key that can upload photos, and a separate key that can download photos OR generate signed URLs to access those photos.
Next step: a script that starts uploading my photos.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947229,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615947229,MDEyOklzc3VlQ29tbWVudDYxNTk0NzIyOQ==,9599,2020-04-18T20:51:26Z,2020-04-18T20:51:26Z,MEMBER,"Running the upload again like this resulted in the correct content-type:
```python
client.upload_file(
""/Users/simonw/Desktop/this_is_fine.jpg"",
""dogsheep-photos-simon"",
""this_is_fine.jpg"",
ExtraArgs={
""ContentType"": ""image/jpeg""
}
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615946537,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615946537,MDEyOklzc3VlQ29tbWVudDYxNTk0NjUzNw==,9599,2020-04-18T20:48:13Z,2020-04-18T20:48:13Z,MEMBER,"How about generating a signed URL?
```python
read_client.generate_presigned_url(
""get_object"",
Params={
""Bucket"": ""dogsheep-photos-simon"",
""Key"": ""this_is_fine.jpg"",
},
ExpiresIn=600
)
```
Gave me https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398
Which does this:
```
~ $ curl -i 'https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'
HTTP/1.1 307 Temporary Redirect
x-amz-bucket-region: us-west-1
x-amz-request-id: E78CD859AEE21D33
x-amz-id-2: 648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=
Location: https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Sat, 18 Apr 2020 20:47:21 GMT
Server: AmazonS3
TemporaryRedirect
Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.dogsheep-photos-simon.s3-us-west-1.amazonaws.comdogsheep-photos-simonE78CD859AEE21D33648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=~ $
```
So it redirects to another URL... which returns this:
```
~ $ curl -i 'https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'
HTTP/1.1 200 OK
x-amz-id-2: XafOl6mswj3yz0GJC9+Ptot1ll5sROVwqsMc10CUUfgpaUANTdIx2GhnONb5d1GVFJ6wlS2j3UY=
x-amz-request-id: 258387C180411AFE
Date: Sat, 18 Apr 2020 20:47:52 GMT
Last-Modified: Sat, 18 Apr 2020 20:37:35 GMT
ETag: ""ee04081c3182a44a1c6944e94012e977""
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 53072
Server: AmazonS3
????JFIF??C
```
So that worked! It did come back with `Content-Type: binary/octet-stream` though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615945056,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615945056,MDEyOklzc3VlQ29tbWVudDYxNTk0NTA1Ng==,9599,2020-04-18T20:42:41Z,2020-04-18T20:42:41Z,MEMBER,"But... `list_objects` failed for both of my keys (read and write):
![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670798-75c41780-817a-11ea-9907-2cbc4a2e497c.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615944806,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615944806,MDEyOklzc3VlQ29tbWVudDYxNTk0NDgwNg==,9599,2020-04-18T20:41:39Z,2020-04-18T20:41:39Z,MEMBER,"This worked!
![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670712-d868e380-8179-11ea-82a5-5dfd17356113.png)
And this worked:
![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670777-50370e00-817a-11ea-83cd-18ebf5702878.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615942116,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615942116,MDEyOklzc3VlQ29tbWVudDYxNTk0MjExNg==,9599,2020-04-18T20:30:56Z,2020-04-18T20:30:56Z,MEMBER,"Next step: attempt a programmatic upload using the `dogsheep-photos-simon-read-write` credentials from a Jupyter notebook.
Also attempt a programmatic bucket listing and read using `dogsheep-photos-simon-read` credentials.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615941746,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615941746,MDEyOklzc3VlQ29tbWVudDYxNTk0MTc0Ng==,9599,2020-04-18T20:29:36Z,2020-04-18T20:29:36Z,MEMBER,"I'm going to create another user just for Transmit, with full S3 access.
name: `dogsheep-photos-simon-s3-all-access`
Rather than creating a group for that user, I'm trying the ""Attach existing policies directly"" option:
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79670182-03513880-8176-11ea-811a-c80aefb4538a.png)
That user DID work with Transmit. I uploaded a test HEIC image. I used Transmit to copy a signed URL for it.
```
~ $ curl -i 'https://dogsheep-photos-simon.s3.us-west-1.amazonaws.com/IMG_7195.HEIC?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWXFXAI...' | head -n 100
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK
x-amz-id-2: gBOCYqZfbNAnv0R/uJ++qm2NbW5SgD4TapgF9RQjzzeDIThcCz/BkKU+YoxlG4NJHlcmMgAHyh4=
x-amz-request-id: C2FE7FCC3BD53A84
Date: Sat, 18 Apr 2020 20:28:54 GMT
Last-Modified: Sat, 18 Apr 2020 20:13:49 GMT
ETag: ""fe3e081239a123ef745517878c53b854""
Accept-Ranges: bytes
Content-Type: image/heic
Content-Length: 1913097
Server: AmazonS3
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615936880,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615936880,MDEyOklzc3VlQ29tbWVudDYxNTkzNjg4MA==,9599,2020-04-18T20:04:31Z,2020-04-18T20:04:31Z,MEMBER,"Next step: create two IAM users, one for each of those groups.
https://console.aws.amazon.com/iam/home#/users$new?step=details
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669931-1bc05380-8174-11ea-9657-0e0c6a692d42.png)
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669941-27137f00-8174-11ea-8ce7-249f0d4f96f6.png)
I copied the keys into a secure note in 1password.
Couldn't get into Transmit with them though! https://library.panic.com/transmit/transmit5/iam-roles/ may help.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615935577,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615935577,MDEyOklzc3VlQ29tbWVudDYxNTkzNTU3Nw==,9599,2020-04-18T19:54:59Z,2020-04-18T19:55:30Z,MEMBER,"Creating IAM groups called `dogsheep-photos-simon-read-write` and `dogsheep-photos-simon-read`: https://console.aws.amazon.com/iam/home#/groups - I created them with no attached policies.
Now I can attach an ""inline policy"" to each one. For the read-write group I go here:
https://console.aws.amazon.com/iam/home#/groups/dogsheep-photos-simon-read-write
![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669703-2d086080-8172-11ea-9597-83e0b155193e.png)
Example policies are here: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html
For the read-write one I went with:
```json
{
""Version"": ""2012-10-17"",
""Statement"": [
{
""Effect"": ""Allow"",
""Action"": ""s3:*"",
""Resource"": [
""arn:aws:s3:::dogsheep-photos-simon/*""
]
}
]
}
```
For the read-only policy I'm going to guess that this is appropriate:
```json
{
""Version"": ""2012-10-17"",
""Statement"": [
{
""Effect"": ""Allow"",
""Action"": [
""s3:GetObject*"",
""s3:ListBucket""
],
""Resource"": [
""arn:aws:s3:::dogsheep-photos-simon/*""
]
}
]
}
```
I tried the policy simulator to test this out: https://policysim.aws.amazon.com/home/index.jsp?#groups/dogsheep-photos-simon-read - this worked:
![IAM_Policy_Simulator](https://user-images.githubusercontent.com/9599/79669893-cd12b980-8173-11ea-8dfb-5660ce3652da.png)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615933273,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615933273,MDEyOklzc3VlQ29tbWVudDYxNTkzMzI3Mw==,9599,2020-04-18T19:37:33Z,2020-04-18T19:37:33Z,MEMBER,"https://console.aws.amazon.com/s3/bucket/create?region=us-west-1
![S3_Management_Console](https://user-images.githubusercontent.com/9599/79669552-33e2a380-8171-11ea-9ab5-5785d34f652a.png)
I created it with no public read-write access. I plan to use signed URLs via a transforming proxy to access images for display on the web.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932204,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615932204,MDEyOklzc3VlQ29tbWVudDYxNTkzMjIwNA==,9599,2020-04-18T19:29:22Z,2020-04-18T19:34:44Z,MEMBER,I'm going to call my bucket `dogsheep-photos-simon`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,