github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615957385 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615957385 | MDEyOklzc3VlQ29tbWVudDYxNTk1NzM4NQ== | 9599 | 2020-04-18T21:56:16Z | 2020-04-18T21:58:11Z | MEMBER | Got this working! I'll do EXIF in a separate ticket #3. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615948102 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615948102 | MDEyOklzc3VlQ29tbWVudDYxNTk0ODEwMg== | 9599 | 2020-04-18T20:56:59Z | 2020-04-18T20:56:59Z | MEMBER | I'm going to start with this: `photos-to-sqlite upload photos.db ~/path/to/directory` This will scan the provided directory (and all sub-directories) for image files. It will then: * Calculate a sha256 of the contents of that file * Upload the file to a key that's `sha256.jpg` or `.heic` * Upload a `sha256.json` file with the original path to the image * Add that image to a `uploads` table in `photos.db` Stretch goal: grab the EXIF data and include that in the `.json` upload AND the `uploads` database table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947370 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615947370 | MDEyOklzc3VlQ29tbWVudDYxNTk0NzM3MA== | 9599 | 2020-04-18T20:52:13Z | 2020-04-18T20:52:13Z | MEMBER | This is great! I now have a key that can upload photos, and a separate key that can download photos OR generate signed URLs to access those photos. Next step: a script that starts uploading my photos. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947229 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615947229 | MDEyOklzc3VlQ29tbWVudDYxNTk0NzIyOQ== | 9599 | 2020-04-18T20:51:26Z | 2020-04-18T20:51:26Z | MEMBER | Running the upload again like this resulted in the correct content-type: ```python client.upload_file( "/Users/simonw/Desktop/this_is_fine.jpg", "dogsheep-photos-simon", "this_is_fine.jpg", ExtraArgs={ "ContentType": "image/jpeg" } ) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615946537 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615946537 | MDEyOklzc3VlQ29tbWVudDYxNTk0NjUzNw== | 9599 | 2020-04-18T20:48:13Z | 2020-04-18T20:48:13Z | MEMBER | How about generating a signed URL? ```python read_client.generate_presigned_url( "get_object", Params={ "Bucket": "dogsheep-photos-simon", "Key": "this_is_fine.jpg", }, ExpiresIn=600 ) ``` Gave me https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398 Which does this: ``` ~ $ curl -i 'https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398' HTTP/1.1 307 Temporary Redirect x-amz-bucket-region: us-west-1 x-amz-request-id: E78CD859AEE21D33 x-amz-id-2: 648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU= Location: https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398 Content-Type: application/xml Transfer-Encoding: chunked Date: Sat, 18 Apr 2020 20:47:21 GMT Server: AmazonS3 <?xml version="1.0" encoding="UTF-8"?> <Error><Code>TemporaryRedirect</Code><Message>Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.</Message><Endpoint>dogsheep-photos-simon.s3-us-west-1.amazonaws.com</Endpoint><Bucket>dogsheep-photos-simon</Bucket><RequestId>E78CD859AEE21D33</RequestId><HostId>648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=</HostId></Error>~ $ ``` So it redirects to another URL... which returns this: ``` ~ $ curl -i 'https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398' HTTP/1.1 200 OK x-amz-id-2: XafOl6mswj3yz0GJC9+Ptot1ll5sROVwqsMc10CUUfgpaUANTdIx2GhnONb5d1GVFJ6wlS2j3UY= x-amz-request-id: 258387C180411AFE Date: Sat, 18 Apr 2020 20:47:52 GMT Last-Modified: Sat, 18 Apr 2020 20:37:35 GMT E… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615945056 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615945056 | MDEyOklzc3VlQ29tbWVudDYxNTk0NTA1Ng== | 9599 | 2020-04-18T20:42:41Z | 2020-04-18T20:42:41Z | MEMBER | But... `list_objects` failed for both of my keys (read and write): ![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670798-75c41780-817a-11ea-9907-2cbc4a2e497c.png) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615944806 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615944806 | MDEyOklzc3VlQ29tbWVudDYxNTk0NDgwNg== | 9599 | 2020-04-18T20:41:39Z | 2020-04-18T20:41:39Z | MEMBER | This worked! ![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670712-d868e380-8179-11ea-82a5-5dfd17356113.png) And this worked: ![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670777-50370e00-817a-11ea-83cd-18ebf5702878.png) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615942116 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615942116 | MDEyOklzc3VlQ29tbWVudDYxNTk0MjExNg== | 9599 | 2020-04-18T20:30:56Z | 2020-04-18T20:30:56Z | MEMBER | Next step: attempt a programmatic upload using the `dogsheep-photos-simon-read-write` credentials from a Jupyter notebook. Also attempt a programmatic bucket listing and read using `dogsheep-photos-simon-read` credentials. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615941746 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615941746 | MDEyOklzc3VlQ29tbWVudDYxNTk0MTc0Ng== | 9599 | 2020-04-18T20:29:36Z | 2020-04-18T20:29:36Z | MEMBER | I'm going to create another user just for Transmit, with full S3 access. name: `dogsheep-photos-simon-s3-all-access` Rather than creating a group for that user, I'm trying the "Attach existing policies directly" option: ![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79670182-03513880-8176-11ea-811a-c80aefb4538a.png) That user DID work with Transmit. I uploaded a test HEIC image. I used Transmit to copy a signed URL for it. ``` ~ $ curl -i 'https://dogsheep-photos-simon.s3.us-west-1.amazonaws.com/IMG_7195.HEIC?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWXFXAI...' | head -n 100 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK x-amz-id-2: gBOCYqZfbNAnv0R/uJ++qm2NbW5SgD4TapgF9RQjzzeDIThcCz/BkKU+YoxlG4NJHlcmMgAHyh4= x-amz-request-id: C2FE7FCC3BD53A84 Date: Sat, 18 Apr 2020 20:28:54 GMT Last-Modified: Sat, 18 Apr 2020 20:13:49 GMT ETag: "fe3e081239a123ef745517878c53b854" Accept-Ranges: bytes Content-Type: image/heic Content-Length: 1913097 Server: AmazonS3 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615936880 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615936880 | MDEyOklzc3VlQ29tbWVudDYxNTkzNjg4MA== | 9599 | 2020-04-18T20:04:31Z | 2020-04-18T20:04:31Z | MEMBER | Next step: create two IAM users, one for each of those groups. https://console.aws.amazon.com/iam/home#/users$new?step=details ![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669931-1bc05380-8174-11ea-9657-0e0c6a692d42.png) ![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669941-27137f00-8174-11ea-8ce7-249f0d4f96f6.png) I copied the keys into a secure note in 1password. Couldn't get into Transmit with them though! https://library.panic.com/transmit/transmit5/iam-roles/ may help. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615935577 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615935577 | MDEyOklzc3VlQ29tbWVudDYxNTkzNTU3Nw== | 9599 | 2020-04-18T19:54:59Z | 2020-04-18T19:55:30Z | MEMBER | Creating IAM groups called `dogsheep-photos-simon-read-write` and `dogsheep-photos-simon-read`: https://console.aws.amazon.com/iam/home#/groups - I created them with no attached policies. Now I can attach an "inline policy" to each one. For the read-write group I go here: https://console.aws.amazon.com/iam/home#/groups/dogsheep-photos-simon-read-write ![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669703-2d086080-8172-11ea-9597-83e0b155193e.png) Example policies are here: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html For the read-write one I went with: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": [ "arn:aws:s3:::dogsheep-photos-simon/*" ] } ] } ``` For the read-only policy I'm going to guess that this is appropriate: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject*", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::dogsheep-photos-simon/*" ] } ] } ``` I tried the policy simulator to test this out: https://policysim.aws.amazon.com/home/index.jsp?#groups/dogsheep-photos-simon-read - this worked: ![IAM_Policy_Simulator](https://user-images.githubusercontent.com/9599/79669893-cd12b980-8173-11ea-8dfb-5660ce3652da.png) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615933273 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615933273 | MDEyOklzc3VlQ29tbWVudDYxNTkzMzI3Mw== | 9599 | 2020-04-18T19:37:33Z | 2020-04-18T19:37:33Z | MEMBER | https://console.aws.amazon.com/s3/bucket/create?region=us-west-1 ![S3_Management_Console](https://user-images.githubusercontent.com/9599/79669552-33e2a380-8171-11ea-9ab5-5785d34f652a.png) I created it with no public read-write access. I plan to use signed URLs via a transforming proxy to access images for display on the web. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932204 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615932204 | MDEyOklzc3VlQ29tbWVudDYxNTkzMjIwNA== | 9599 | 2020-04-18T19:29:22Z | 2020-04-18T19:34:44Z | MEMBER | I'm going to call my bucket `dogsheep-photos-simon`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 |