github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/dogsheep/github-to-sqlite/issues/26#issuecomment-614794739 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26 | 614794739 | MDEyOklzc3VlQ29tbWVudDYxNDc5NDczOQ== | 9599 | 2020-04-16T17:38:28Z | 2020-04-16T17:38:28Z | MEMBER | I'm already doing this here: https://github.com/dogsheep/github-to-sqlite/blob/c4aaa50e167cfa9021c7c94260bc3e89e10947bf/github_to_sqlite/utils.py#L246-L250 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
601271612 | |
https://github.com/dogsheep/github-to-sqlite/issues/29#issuecomment-616883275 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/29 | 616883275 | MDEyOklzc3VlQ29tbWVudDYxNjg4MzI3NQ== | 9599 | 2020-04-21T00:43:28Z | 2020-04-21T00:43:28Z | MEMBER | I'm copying repo from issue, which surprisingly is a string, not an integer ID. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
603617013 | |
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861042050 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64 | 861042050 | MDEyOklzc3VlQ29tbWVudDg2MTA0MjA1MA== | 9599 | 2021-06-14T22:45:42Z | 2021-06-14T22:45:42Z | MEMBER | I'm definitely interested in supporting events in this tool - see #14. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
920636216 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902327457 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 | 902327457 | IC_kwDODEm0Qs41yGyh | 9599 | 2021-08-19T23:53:25Z | 2021-08-19T23:53:25Z | MEMBER | I'm getting this too. Looking into it now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
907645813 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549226399 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | 549226399 | MDEyOklzc3VlQ29tbWVudDU0OTIyNjM5OQ== | 9599 | 2019-11-04T05:11:57Z | 2019-11-04T05:11:57Z | MEMBER | I'm going to add a `hash` column to `search_runs` to support that. It's going to be the sha1 hash of the key-ordered JSON of the search arguments used by that run. Then `--since` can look for an identical hash and use it to identify the highest last fetched tweet to use in `since_id`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
488833975 | |
https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886140431 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2 | 886140431 | IC_kwDODtX3eM400W4P | 9599 | 2021-07-25T03:12:57Z | 2021-07-25T03:12:57Z | MEMBER | I'm going to build a general-purpose `hacker-new-to-sqlite search ...` command, where one of the options is to search within the URL. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
952179830 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932204 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615932204 | MDEyOklzc3VlQ29tbWVudDYxNTkzMjIwNA== | 9599 | 2020-04-18T19:29:22Z | 2020-04-18T19:34:44Z | MEMBER | I'm going to call my bucket `dogsheep-photos-simon`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/2#issuecomment-527954898 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2 | 527954898 | MDEyOklzc3VlQ29tbWVudDUyNzk1NDg5OA== | 9599 | 2019-09-04T15:31:46Z | 2019-09-04T15:31:46Z | MEMBER | I'm going to call this `twitter-to-sqlite user-timeline` to reflect the language used to describe the API endpoint: https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
488833698 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615941746 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615941746 | MDEyOklzc3VlQ29tbWVudDYxNTk0MTc0Ng== | 9599 | 2020-04-18T20:29:36Z | 2020-04-18T20:29:36Z | MEMBER | I'm going to create another user just for Transmit, with full S3 access. name: `dogsheep-photos-simon-s3-all-access` Rather than creating a group for that user, I'm trying the "Attach existing policies directly" option: ![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79670182-03513880-8176-11ea-811a-c80aefb4538a.png) That user DID work with Transmit. I uploaded a test HEIC image. I used Transmit to copy a signed URL for it. ``` ~ $ curl -i 'https://dogsheep-photos-simon.s3.us-west-1.amazonaws.com/IMG_7195.HEIC?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWXFXAI...' | head -n 100 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK x-amz-id-2: gBOCYqZfbNAnv0R/uJ++qm2NbW5SgD4TapgF9RQjzzeDIThcCz/BkKU+YoxlG4NJHlcmMgAHyh4= x-amz-request-id: C2FE7FCC3BD53A84 Date: Sat, 18 Apr 2020 20:28:54 GMT Last-Modified: Sat, 18 Apr 2020 20:13:49 GMT ETag: "fe3e081239a123ef745517878c53b854" Accept-Ranges: bytes Content-Type: image/heic Content-Length: 1913097 Server: AmazonS3 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/github-to-sqlite/issues/21#issuecomment-602896434 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/21 | 602896434 | MDEyOklzc3VlQ29tbWVudDYwMjg5NjQzNA== | 9599 | 2020-03-23T22:43:37Z | 2020-03-23T22:43:37Z | MEMBER | I'm going to do this now to help figure out the latest error in #13: ``` Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.8.2/x64/bin/github-to-sqlite", line 11, in <module> load_entry_point('github-to-sqlite', 'console_scripts', 'github-to-sqlite')() File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py", line 237, in commits utils.save_commits(db, commits, repo_full["id"]) File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 345, in save_commits for commit in commits: File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 207, in fetch_commits if stop_when(commit): File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py", line 224, in stop_when db["commits"].get(commit["sha"]) TypeError: string indices must be integers ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
586561727 | |
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602861730 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | 602861730 | MDEyOklzc3VlQ29tbWVudDYwMjg2MTczMA== | 9599 | 2020-03-23T21:19:16Z | 2020-03-23T21:19:42Z | MEMBER | I'm going to do this with a scheduled GitHub Action in this repo. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
521275281 | |
https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-542876047 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3 | 542876047 | MDEyOklzc3VlQ29tbWVudDU0Mjg3NjA0Nw== | 9599 | 2019-10-16T20:23:36Z | 2019-10-16T20:23:36Z | MEMBER | I'm going to go with `--since=1d/2w/3h` for this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
487600595 | |
https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685966361 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7 | 685966361 | MDEyOklzc3VlQ29tbWVudDY4NTk2NjM2MQ== | 9599 | 2020-09-02T20:03:29Z | 2020-09-02T20:03:41Z | MEMBER | I'm going to implement the first version of this as an indexed integer `category` column which has 1 for "about me" and 2 for "liked by me" - and space for other category numerals in the future, albeit a row can only belong to one category. I'll think about a full tagging system separately. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
691265198 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601876051 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | 601876051 | MDEyOklzc3VlQ29tbWVudDYwMTg3NjA1MQ== | 9599 | 2020-03-20T19:31:43Z | 2020-03-20T19:31:43Z | MEMBER | I'm going to keep `--screen_name` working for backwards compatibility. Using `hidden=True` on it should hide it from the help output. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
585282212 | |
https://github.com/dogsheep/dogsheep-photos/issues/13#issuecomment-620309185 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/13 | 620309185 | MDEyOklzc3VlQ29tbWVudDYyMDMwOTE4NQ== | 9599 | 2020-04-28T00:39:45Z | 2020-04-28T00:39:45Z | MEMBER | I'm going to leave this until I have the mechanism for associating a live photo video with the photo. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
607888367 | |
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831842 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 | 614831842 | MDEyOklzc3VlQ29tbWVudDYxNDgzMTg0Mg== | 9599 | 2020-04-16T18:48:18Z | 2020-04-16T18:48:18Z | MEMBER | I'm going to make `organization` another foreign key to the `users` table just in case it IS possible (maybe with GitHub Enterprise or similar?) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
601330277 | |
https://github.com/dogsheep/dogsheep-beta/issues/25#issuecomment-695108895 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25 | 695108895 | MDEyOklzc3VlQ29tbWVudDY5NTEwODg5NQ== | 9599 | 2020-09-18T22:11:32Z | 2020-09-18T22:11:32Z | MEMBER | I'm going to make this a new plugin configuration setting, `template_debug`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
704685890 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615509803 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37 | 615509803 | MDEyOklzc3VlQ29tbWVudDYxNTUwOTgwMw== | 9599 | 2020-04-17T23:36:40Z | 2020-04-17T23:36:40Z | MEMBER | I'm going to print a warning to stderr, skip and continue - because if you have 100 screen names and only one of them is invalid you should still execute for the other 99. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
585353598 | |
https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-623723026 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15 | 623723026 | MDEyOklzc3VlQ29tbWVudDYyMzcyMzAyNg== | 9599 | 2020-05-04T21:41:30Z | 2020-05-04T21:41:30Z | MEMBER | I'm going to put these in a table called `apple_photos_scores` - I'll also pull in the following columns from the `ZGENERICASSET` table: * `ZOVERALLAESTHETICSCORE` * `ZCURATIONSCORE` * `ZHIGHLIGHTVISIBILITYSCORE` * `ZPROMOTIONSCORE` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
612151767 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601875405 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | 601875405 | MDEyOklzc3VlQ29tbWVudDYwMTg3NTQwNQ== | 9599 | 2020-03-20T19:30:09Z | 2020-03-20T19:32:33Z | MEMBER | I'm going to redesign the command a little. Right now it defaults to fetching the authenticated user's timeline and allows `--screen_name` to be used to fetch for one other user instead. I think the design of `users-lookup` is much better. I'm going to copy that: ``` twitter-to-sqlite user-timeline users.db simonw cleopaws twitter-to-sqlite user-timeline users.db 12497 3166449535 --ids ``` It will also accept `--since` and `--sql` and `--alter` and `--since_id`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
585282212 | |
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631227245 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26 | 631227245 | MDEyOklzc3VlQ29tbWVudDYzMTIyNzI0NQ== | 9599 | 2020-05-20T04:21:38Z | 2020-05-20T04:21:38Z | MEMBER | I'm going to release 0.4 now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
621444763 | |
https://github.com/dogsheep/github-to-sqlite/issues/58#issuecomment-746734412 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58 | 746734412 | MDEyOklzc3VlQ29tbWVudDc0NjczNDQxMg== | 9599 | 2020-12-16T17:58:56Z | 2020-12-16T17:58:56Z | MEMBER | I'm going to rewrite those `<a href="#filtering-tables">` links to `<a href="#user-content-filtering-tables">` - but only if a corresponding `id="user-content-filtering-tables"` element exists. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
769150394 | |
https://github.com/dogsheep/healthkit-to-sqlite/issues/5#issuecomment-513514978 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5 | 513514978 | MDEyOklzc3VlQ29tbWVudDUxMzUxNDk3OA== | 9599 | 2019-07-21T02:55:12Z | 2019-07-21T02:55:12Z | MEMBER | I'm going to show this by default. Users can pass `-s` or `--silent` to disable the progress bar. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
470691622 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693773191 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693773191 | MDEyOklzc3VlQ29tbWVudDY5Mzc3MzE5MQ== | 9599 | 2020-09-17T02:39:26Z | 2020-09-17T02:39:26Z | MEMBER | I'm going to start with `github-to-sqlite get` and `github-to-sqlite post` - I may add `put` and suchlike later on. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615948102 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | 615948102 | MDEyOklzc3VlQ29tbWVudDYxNTk0ODEwMg== | 9599 | 2020-04-18T20:56:59Z | 2020-04-18T20:56:59Z | MEMBER | I'm going to start with this: `photos-to-sqlite upload photos.db ~/path/to/directory` This will scan the provided directory (and all sub-directories) for image files. It will then: * Calculate a sha256 of the contents of that file * Upload the file to a key that's `sha256.jpg` or `.heic` * Upload a `sha256.json` file with the original path to the image * Add that image to a `uploads` table in `photos.db` Stretch goal: grab the EXIF data and include that in the `.json` upload AND the `uploads` database table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533539 | |
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695856398 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26 | 695856398 | MDEyOklzc3VlQ29tbWVudDY5NTg1NjM5OA== | 9599 | 2020-09-21T00:22:20Z | 2020-09-21T00:22:20Z | MEMBER | I'm going to try for keyset pagination sorted by relevance just as a learning exercise. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705215230 | |
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623193947 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | 623193947 | MDEyOklzc3VlQ29tbWVudDYyMzE5Mzk0Nw== | 9599 | 2020-05-03T22:36:17Z | 2020-05-03T22:36:17Z | MEMBER | I'm going to use [osxphotos](https://github.com/RhetTbull/osxphotos) for this. Since I've already got code to upload photos and insert them into a table based on their `sha256` hash, my first go at this will be to import data using the tool and foreign-key it to the `sha256` hash in the existing table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
602533300 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-662626901 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48 | 662626901 | MDEyOklzc3VlQ29tbWVudDY2MjYyNjkwMQ== | 9599 | 2020-07-22T18:54:53Z | 2020-07-22T18:54:53Z | MEMBER | I'm going to use a GitHub Action to run `npx markdown-toc README.md -i` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
663976976 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-607010791 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10 | 607010791 | MDEyOklzc3VlQ29tbWVudDYwNzAxMDc5MQ== | 9599 | 2020-04-01T03:45:48Z | 2020-04-01T03:45:48Z | MEMBER | I'm happy with the recent work I did on this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
492297930 | |
https://github.com/dogsheep/github-to-sqlite/issues/21#issuecomment-602920163 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/21 | 602920163 | MDEyOklzc3VlQ29tbWVudDYwMjkyMDE2Mw== | 9599 | 2020-03-23T23:48:22Z | 2020-03-23T23:48:22Z | MEMBER | I'm happy with this pattern: https://github.com/dogsheep/github-to-sqlite/blob/f78c4e9baaf0970ffab266ba780df7240aae9f32/github_to_sqlite/utils.py#L4-L18 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
586561727 | |
https://github.com/dogsheep/dogsheep-beta/issues/18#issuecomment-688623097 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18 | 688623097 | MDEyOklzc3VlQ29tbWVudDY4ODYyMzA5Nw== | 9599 | 2020-09-08T05:15:51Z | 2020-09-08T05:15:51Z | MEMBER | I'm inclined to go with the first, simpler option. I have longer term plans for efficient incremental index updates based on clever trickery with triggers. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
695553522 | |
https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-747034481 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29 | 747034481 | MDEyOklzc3VlQ29tbWVudDc0NzAzNDQ4MQ== | 9599 | 2020-12-16T21:17:05Z | 2020-12-16T21:17:05Z | MEMBER | I'm just going to add `q` for the moment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
724759588 | |
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623000814 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4 | 623000814 | MDEyOklzc3VlQ29tbWVudDYyMzAwMDgxNA== | 9599 | 2020-05-02T19:15:23Z | 2020-05-02T19:15:23Z | MEMBER | I'm not going to do the `--sql` bit just yet. I have patterns for working around this for other commands which are working fine: https://github.com/dogsheep/github-to-sqlite/blob/d00a53061556dc403c166b443d141c4e1adbd64a/.github/workflows/deploy-demo.yml#L53-L70 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
493670730 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-542854749 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20 | 542854749 | MDEyOklzc3VlQ29tbWVudDU0Mjg1NDc0OQ== | 9599 | 2019-10-16T19:26:01Z | 2019-10-16T19:26:01Z | MEMBER | I'm not going to do this for "accounts that have followed me" and "new accounts that I have followed" - instead I will recommend running the `friend_ids` and `followers_ids` commands on a daily basis since that data doesn't really change much by the hour. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
506268945 | |
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790372621 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | 790372621 | MDEyOklzc3VlQ29tbWVudDc5MDM3MjYyMQ== | 9599 | 2021-03-04T07:01:18Z | 2021-03-04T07:01:18Z | MEMBER | I'm not sure if it would work, but there is an alternative pattern for showing a progress bar against a really large file that I've used in `healthkit-to-sqlite` - you set the progress bar size to the size of the file in bytes, then update a counter as you read the file. https://github.com/dogsheep/healthkit-to-sqlite/blob/3eb2b06bfe3b4faaf10e9cf9dfcb28e3d16c14ff/healthkit_to_sqlite/cli.py#L24-L57 and https://github.com/dogsheep/healthkit-to-sqlite/blob/3eb2b06bfe3b4faaf10e9cf9dfcb28e3d16c14ff/healthkit_to_sqlite/utils.py#L4-L19 (the `progress_callback()` bit) is where that happens. It can be a bit of a convoluted pattern, and I'm not at all sure it would work for `mbox` files since it looks like that library has other reasons it needs to do a file scan rather than streaming it through one chunk of bytes at a time. So I imagine this would not work here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
813880401 | |
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790695126 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | 790695126 | MDEyOklzc3VlQ29tbWVudDc5MDY5NTEyNg== | 9599 | 2021-03-04T15:20:42Z | 2021-03-04T15:20:42Z | MEMBER | I'm not sure why but my most recent import, when displayed in Datasette, looks like this: <img width="574" alt="mbox__mbox_emails__753_446_rows" src="https://user-images.githubusercontent.com/9599/109985836-0ab00080-7cba-11eb-97d5-0631a0835b61.png"> Sorting by `id` in the opposite order gives me the data I would expect - so it looks like a bunch of null/blank messages are being imported at some point and showing up first due to ID ordering. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
813880401 | |
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686688963 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9 | 686688963 | MDEyOklzc3VlQ29tbWVudDY4NjY4ODk2Mw== | 9599 | 2020-09-03T18:42:59Z | 2020-09-03T18:42:59Z | MEMBER | I'm pleased with how this works now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
691521965 | |
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623807568 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16 | 623807568 | MDEyOklzc3VlQ29tbWVudDYyMzgwNzU2OA== | 9599 | 2020-05-05T02:56:06Z | 2020-05-05T02:56:06Z | MEMBER | I'm pretty sure this is what I'm after. The `groups` table has what looks like identified labels in the rows with category = 2025: <img width="1122" alt="words__groups__2_528_rows_where_where_category___2025" src="https://user-images.githubusercontent.com/9599/81031361-e0df4080-8e40-11ea-9060-6d850aa52140.png"> Then there's a `ga` table that maps groups to assets: <img width="304" alt="words__ga__633_653_rows" src="https://user-images.githubusercontent.com/9599/81031387-f48aa700-8e40-11ea-9a3d-da23903be928.png"> And an `assets` table which looks like it has one row for every one of my photos: <img width="645" alt="words__assets__40_419_rows" src="https://user-images.githubusercontent.com/9599/81031402-04a28680-8e41-11ea-8047-e9199d068563.png"> One major challenge: these UUIDs are split into two integer numbers, `uuid_0` and `uuid_1` - but the main photos database uses regular UUIDs like this: ![image](https://user-images.githubusercontent.com/9599/81031481-39164280-8e41-11ea-983b-005ced641a18.png) I need to figure out how to match up these two different UUID representations. I asked on Twitter if anyone has any ideas: https://twitter.com/simonw/status/1257500689019703296 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
612287234 | |
https://github.com/dogsheep/github-to-sqlite/issues/16#issuecomment-601979709 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16 | 601979709 | MDEyOklzc3VlQ29tbWVudDYwMTk3OTcwOQ== | 9599 | 2020-03-21T02:20:08Z | 2020-03-21T02:20:08Z | MEMBER | I'm pretty sure this means I need to upgrade the underlying `sqlite-utils` dependency. I saw the same bug in `twitter-to-sqlite`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
546051181 | |
https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514496725 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7 | 514496725 | MDEyOklzc3VlQ29tbWVudDUxNDQ5NjcyNQ== | 9599 | 2019-07-24T06:20:59Z | 2019-07-24T06:20:59Z | MEMBER | I'm using https://pypi.org/project/memory-profiler/ to explore this in more detail: ``` $ pip install memory-profiler matplotlib ``` Then: ``` $ mprof run healthkit-to-sqlite ~/Downloads/healthkit-export.zip healthkit.db $ mprof plot ``` <img width="1128" alt="Screen Shot 2019-07-24 at 8 17 06 AM" src="https://user-images.githubusercontent.com/9599/61769849-f3294a00-adeb-11e9-9026-02c24209530f.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
472097220 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607003655 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | 607003655 | MDEyOklzc3VlQ29tbWVudDYwNzAwMzY1NQ== | 9599 | 2020-04-01T03:18:00Z | 2020-04-01T03:18:00Z | MEMBER | I've got this working for the `user-timeline` command. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
590666760 | |
https://github.com/dogsheep/evernote-to-sqlite/pull/10#issuecomment-777839351 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/10 | 777839351 | MDEyOklzc3VlQ29tbWVudDc3NzgzOTM1MQ== | 9599 | 2021-02-11T22:37:55Z | 2021-02-11T22:37:55Z | MEMBER | I've merged these changes by hand now, thanks! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
770712149 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645599881 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47 | 645599881 | MDEyOklzc3VlQ29tbWVudDY0NTU5OTg4MQ== | 9599 | 2020-06-17T20:13:48Z | 2020-06-17T20:13:48Z | MEMBER | I've now figured out how to compile specific SQLite versions to help replicate this problem: https://github.com/simonw/til/blob/master/sqlite/ld-preload.md Next step: replicate the problem! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
639542974 | |
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695879237 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26 | 695879237 | MDEyOklzc3VlQ29tbWVudDY5NTg3OTIzNw== | 9599 | 2020-09-21T02:53:29Z | 2020-09-21T02:53:29Z | MEMBER | If previous page ended at `2018-02-11T16:32:53+00:00`: ```sql select search_index.rowid, search_index.type, search_index.key, search_index.title, search_index.category, search_index.timestamp, search_index.search_1 from search_index where date("timestamp") = '2018-02-11' and timestamp < '2018-02-11T16:32:53+00:00' order by search_index.timestamp desc, rowid limit 41 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705215230 | |
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686154627 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9 | 686154627 | MDEyOklzc3VlQ29tbWVudDY4NjE1NDYyNw== | 9599 | 2020-09-03T00:19:22Z | 2020-09-03T00:19:22Z | MEMBER | If this performs well enough (100 displayed items will be 100 extra `display_sql` calls) then I'll go with this as the design for the feature. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
691521965 | |
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686158454 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9 | 686158454 | MDEyOklzc3VlQ29tbWVudDY4NjE1ODQ1NA== | 9599 | 2020-09-03T00:32:42Z | 2020-09-03T00:32:42Z | MEMBER | If this turns out to be too inefficient I could add a `display` text column to the `search_index` table which is designed to be populated with arbitrary JSON by the indexing query, which can then be used to render the template fragment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
691521965 | |
https://github.com/dogsheep/hacker-news-to-sqlite/issues/3#issuecomment-886237834 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3 | 886237834 | IC_kwDODtX3eM400uqK | 9599 | 2021-07-25T18:05:32Z | 2021-07-25T18:05:32Z | MEMBER | If you hit the endpoint for a comment that's part of a thread you get that comment and its recursive children: https://hn.algolia.com/api/v1/items/27941552 You can tell that it's not the top-level because the `parent_id` isn't `null`. You can use `story_id` to figure out what the top-level item is. ```json { "id": 27941552, "created_at": "2021-07-24T15:08:39.000Z", "created_at_i": 1627139319, "type": "comment", "author": "nine_k", "title": null, "url": null, "text": "<p>I wish ...", "points": null, "parent_id": 27941108, "story_id": 27941108 } ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
952189173 | |
https://github.com/dogsheep/apple-notes-to-sqlite/issues/7#issuecomment-1462691466 | https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7 | 1462691466 | IC_kwDOJHON9s5XLuaK | 9599 | 2023-03-09T19:59:52Z | 2023-03-09T19:59:52Z | MEMBER | Improved script: ```zsh osascript -e 'tell application "Notes" set allFolders to folders repeat with aFolder in allFolders set folderId to id of aFolder set folderName to name of aFolder set folderContainer to container of aFolder if class of folderContainer is folder then set folderContainerId to id of folderContainer else set folderContainerId to "" end if log "ID: " & folderId log "Name: " & folderName log "Container: " & folderContainerId log " " end repeat end tell ' ``` ``` ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6113 Name: Blog posts Container: ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p698 Name: JSK Container: ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p7995 Name: Nested inside blog posts Container: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6113 ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p3526 Name: New Folder Container: ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p3839 Name: New Folder 1 Container: ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p2 Name: Notes Container: ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6059 Name: Quick Notes Container: ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p7283 Name: UK Christmas 2022 Container: ``` I filtered out things where the parent was an account and not a folder using `if class of folderContainer is folder then`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1617769847 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-544335363 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20 | 544335363 | MDEyOklzc3VlQ29tbWVudDU0NDMzNTM2Mw== | 9599 | 2019-10-21T03:32:04Z | 2019-10-21T03:32:04Z | MEMBER | In case anyone is interested, here's an extract from the crontab I'm running these under at the moment: ``` 1,11,21,31,41,51 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite user-timeline /home/ubuntu/twitter.db -a /home/ubuntu/auth.json --since 2,7,12,17,22,27,32,37,42,47,52,57 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite home-timeline /home/ubuntu/timeline.db -a /home/ubuntu/auth.json --since 6,16,26,36,46,56 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite favorites /home/ubuntu/twitter.db -a /home/ubuntu/auth.json --stop_after=50 ``` | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
506268945 | |
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623806085 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16 | 623806085 | MDEyOklzc3VlQ29tbWVudDYyMzgwNjA4NQ== | 9599 | 2020-05-05T02:47:18Z | 2020-05-05T02:47:18Z | MEMBER | In https://github.com/RhetTbull/osxphotos/issues/121#issuecomment-623249263 Rhet Turnbull spotted a table called `ZSCENEIDENTIFIER` which looked like it might have the right data, but the columns in it aren't particularly helpful: ``` Z_PK,Z_ENT,Z_OPT,ZSCENEIDENTIFIER,ZASSETATTRIBUTES,ZCONFIDENCE 8,49,1,731,5,0.11834716796875 9,49,1,684,6,0.0233648251742125 10,49,1,1702,1,0.026153564453125 ``` I love the look of those confidence scores, but what do the numbers mean? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
612287234 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541119834 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18 | 541119834 | MDEyOklzc3VlQ29tbWVudDU0MTExOTgzNA== | 9599 | 2019-10-11T15:51:22Z | 2019-10-11T16:51:33Z | MEMBER | In order to support multiple user timelines being saved in the same database, I'm going to import the tweets into the `tweets` table AND add a new `timeline_tweets` table recording that a specific tweet showed up in a specific user's timeline. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
505928530 | |
https://github.com/dogsheep/apple-notes-to-sqlite/issues/4#issuecomment-1462556829 | https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/4 | 1462556829 | IC_kwDOJHON9s5XLNid | 9599 | 2023-03-09T18:20:56Z | 2023-03-09T18:20:56Z | MEMBER | In terms of the UI: I'm tempted to say that the default behaviour is for it to run until it sees a note that it already knows about AND that has matching update/created dates, and then stop. You can do a full import again ignoring that logic with `apple-notes-to-sqlite notes.db --full`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1616429236 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607019151 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | 607019151 | MDEyOklzc3VlQ29tbWVudDYwNzAxOTE1MQ== | 9599 | 2020-04-01T04:11:10Z | 2020-04-01T04:11:10Z | MEMBER | In testing this collects a LOT of data. I'm going to skip tracking favourites_count and statuses_count and just track followers, friends and listed instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
590669793 | |
https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735464438 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54 | 735464438 | MDEyOklzc3VlQ29tbWVudDczNTQ2NDQzOA== | 9599 | 2020-11-29T21:57:08Z | 2020-11-29T21:57:08Z | MEMBER | Inspired by this tweet from Michael Heap https://twitter.com/mheap/status/1333108608817631238 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
753026003 | |
https://github.com/dogsheep/dogsheep-photos/issues/22#issuecomment-626941278 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22 | 626941278 | MDEyOklzc3VlQ29tbWVudDYyNjk0MTI3OA== | 9599 | 2020-05-11T20:25:58Z | 2020-05-11T20:25:58Z | MEMBER | Interesting - do you know if there's anything the `exiftool` process handles that `ExifReader` doesn't? I'm actually just going to extract a subset of the EXIF data at first - since the original photo files will always be available I don't feel the need to get everything out for the first step. My plan is to use EXIF to help support photo collections that aren't in Apple Photos - I'm going to build a database table keyed by the `sha256` of each photo that extracts the camera make, lens, a few settings (ISO, aperture etc) and the GPS lat/lon. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
615626118 | |
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831451 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 | 614831451 | MDEyOklzc3VlQ29tbWVudDYxNDgzMTQ1MQ== | 9599 | 2020-04-16T18:47:25Z | 2020-04-16T18:47:25Z | MEMBER | Is it possible for a repo to have an `owner` that differs from its `organization`? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
601330277 | |
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461262577 | https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2 | 1461262577 | IC_kwDOJHON9s5XGRjx | 9599 | 2023-03-09T04:30:00Z | 2023-03-09T04:30:00Z | MEMBER | It doesn't have tests yet. I guess I'll need to mock `subprocess` to test this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1616354999 | |
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695855723 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26 | 695855723 | MDEyOklzc3VlQ29tbWVudDY5NTg1NTcyMw== | 9599 | 2020-09-21T00:16:52Z | 2020-09-21T00:17:53Z | MEMBER | It feels a bit weird to implement keyset pagination against results sorted by `rank` because the ranks could change substantially if the search index gets updated while the user is paginating. I may just ignore that though. If you want reliable pagination you can get it by sorting by date. Maybe it doesn't even make sense to offer pagination if you sort by relevance? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705215230 | |
https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550783316 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | 550783316 | MDEyOklzc3VlQ29tbWVudDU1MDc4MzMxNg== | 9599 | 2019-11-07T05:16:56Z | 2019-11-07T05:34:29Z | MEMBER | It looks like Apple changed the location of these in iOS 13 - they are now in separate `.gpx` files: ![2FF70E95-CDEE-4241-A5C5-EE95A862E519](https://user-images.githubusercontent.com/9599/68362042-be12e000-00da-11ea-8925-7397410332d8.png) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
519038979 | |
https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-623723687 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15 | 623723687 | MDEyOklzc3VlQ29tbWVudDYyMzcyMzY4Nw== | 9599 | 2020-05-04T21:43:06Z | 2020-05-04T21:43:06Z | MEMBER | It looks like I can map the photos I'm importing to these tables using the `ZUUID` column on `ZGENERICASSET` to get a `Z_PK` which then maps to the rows in `ZGENERICASSET`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
612151767 | |
https://github.com/dogsheep/pocket-to-sqlite/issues/3#issuecomment-684424396 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3 | 684424396 | MDEyOklzc3VlQ29tbWVudDY4NDQyNDM5Ng== | 9599 | 2020-09-01T06:17:45Z | 2020-09-01T06:17:45Z | MEMBER | It looks like I could ignore the `image` column and synthesize a unique key from the data in the `images` column using `$item_id/$image_id`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
503243784 | |
https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-906635938 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13 | 906635938 | IC_kwDOEhK-wc42Ciqi | 9599 | 2021-08-26T18:18:27Z | 2021-08-26T18:18:27Z | MEMBER | It looks like I was using the round-trip to dump the `<?xml version="1.0" encoding="UTF-8" standalone="no"?>` and `<!DOCTYPE` prefixes. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
978743426 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095641 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27 | 549095641 | MDEyOklzc3VlQ29tbWVudDU0OTA5NTY0MQ== | 9599 | 2019-11-03T01:12:58Z | 2019-11-03T01:12:58Z | MEMBER | It looks like Twitter really want you to subscribe to a premium API for this kind of thing and consume retweets via webhooks: https://developer.twitter.com/en/docs/accounts-and-users/subscribe-account-activity/api-reference I'm going to give up on this for the moment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
514459062 | |
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623865250 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16 | 623865250 | MDEyOklzc3VlQ29tbWVudDYyMzg2NTI1MA== | 9599 | 2020-05-05T05:38:16Z | 2020-05-05T05:38:16Z | MEMBER | It looks like `groups.content_string` often has a null byte in it. I should clean this up as part of the import. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
612287234 | |
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790378658 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | 790378658 | MDEyOklzc3VlQ29tbWVudDc5MDM3ODY1OA== | 9599 | 2021-03-04T07:12:48Z | 2021-03-04T07:12:48Z | MEMBER | It looks like the `body` is being loaded into a BLOB column - so in Datasette default it looks like this: <img width="1650" alt="mbox__mbox_emails__753_446_rows" src="https://user-images.githubusercontent.com/9599/109924808-b4b96980-7c75-11eb-8c9e-307f2ae32d5a.png"> If I `datasette install datasette-render-binary` and then try again I get this: <img width="1487" alt="mbox__mbox_emails__753_446_rows" src="https://user-images.githubusercontent.com/9599/109924944-ea5e5280-7c75-11eb-9a32-404f3d68455f.png"> It would be great if we could store the `body` as unicode text instead. May have to do something clever to decode it based on some kind of charset header? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
813880401 | |
https://github.com/dogsheep/apple-notes-to-sqlite/issues/4#issuecomment-1462554175 | https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/4 | 1462554175 | IC_kwDOJHON9s5XLM4_ | 9599 | 2023-03-09T18:19:34Z | 2023-03-09T18:19:34Z | MEMBER | It looks like the iteration order is most-recently-modified-first - I tried editing a note a bit further back in my notes app and it was the first one output by `apple-notes-to-sqlite --dump`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1616429236 | |
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622133422 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34 | 622133422 | MDEyOklzc3VlQ29tbWVudDYyMjEzMzQyMg== | 9599 | 2020-04-30T21:48:39Z | 2020-04-30T21:48:39Z | MEMBER | It looks like the only option is to scrape them. I'll do that and then replace with an API as soon as one becomes available. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
610408908 | |
https://github.com/dogsheep/evernote-to-sqlite/issues/6#issuecomment-706785201 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6 | 706785201 | MDEyOklzc3VlQ29tbWVudDcwNjc4NTIwMQ== | 9599 | 2020-10-11T23:29:39Z | 2020-10-11T23:29:39Z | MEMBER | It looks to me like each of those `<item>` blocks has a number of guesses in order of confidence: ```xml <item x="215" y="190" w="187" h="39"> <t w="57">wonders,</t> <t w="55">wanders,</t> <t w="52">wonders ?</t> <t w="45">wonders</t> <t w="42">wonders.</t> </item> ``` So maybe the best approach here is to just take the first `t` element within each `item`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
718949182 | |
https://github.com/dogsheep/healthkit-to-sqlite/issues/5#issuecomment-513625406 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5 | 513625406 | MDEyOklzc3VlQ29tbWVudDUxMzYyNTQwNg== | 9599 | 2019-07-22T03:20:16Z | 2019-07-22T03:20:16Z | MEMBER | It now renders like this: ``` Importing from HealthKit [#-----------------------------------] 5% 00:01:33 ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
470691622 | |
https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735483604 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46 | 735483604 | MDEyOklzc3VlQ29tbWVudDczNTQ4MzYwNA== | 9599 | 2020-11-30T00:26:50Z | 2020-11-30T00:26:50Z | MEMBER | It seems like there's a lot missing from that - those aren't particularly interesting given the data that is returned. From the docs at https://docs.github.com/en/free-pro-team@latest/rest/reference/pulls#reviews it looks like each review consists of multiple comments, and the comments are where the useful material is - https://docs.github.com/en/free-pro-team@latest/rest/reference/pulls#list-comments-for-a-pull-request-review `github-to-sqlite get https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48/reviews/503368921/comments --accept 'application/vnd.github.v3+json'` ```json [ { "id": 500603838, "node_id": "MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDUwMDYwMzgzOA==", "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500603838", "pull_request_review_id": 503368921, "diff_hunk": "@@ -0,0 +1,370 @@\n+[\n+ {\n+ \"url\": \"https://api.github.com/repos/simonw/datasette/pulls/571\",\n+ \"id\": 313384926,\n+ \"node_id\": \"MDExOlB1bGxSZXF1ZXN0MzEzMzg0OTI2\",\n+ \"html_url\": \"https://github.com/simonw/datasette/pull/571\",\n+ \"diff_url\": \"https://github.com/simonw/datasette/pull/571.diff\",\n+ \"patch_url\": \"https://github.com/simonw/datasette/pull/571.patch\",\n+ \"issue_url\": \"https://api.github.com/repos/simonw/datasette/issues/571\",\n+ \"number\": 571,\n+ \"state\": \"closed\",\n+ \"locked\": false,\n+ \"title\": \"detect_fts now works with alternative table escaping\",\n+ \"user\": {\n+ \"login\": \"simonw\",\n+ \"id\": 9599,\n+ \"node_id\": \"MDQ6VXNlcjk1OTk=\",\n+ \"avatar_url\": \"https://avatars0.githubusercontent.com/u/9599?v=4\",\n+ \"gravatar_id\": \"\",\n+ \"url\": \"https://api.github.com/users/simonw\",\n+ \"html_url\": \"https://github.com/simonw\",\n+ \"followers_url\": \"https://api.github.com/users/simonw/followers\",\n+ \"following_ur… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
664485022 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/17#issuecomment-541112108 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17 | 541112108 | MDEyOklzc3VlQ29tbWVudDU0MTExMjEwOA== | 9599 | 2019-10-11T15:30:15Z | 2019-10-11T15:30:15Z | MEMBER | It should delete the tables entirely. That way it will work even if the table schema has changed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
505674949 | |
https://github.com/dogsheep/github-to-sqlite/issues/37#issuecomment-622461948 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37 | 622461948 | MDEyOklzc3VlQ29tbWVudDYyMjQ2MTk0OA== | 9599 | 2020-05-01T16:36:42Z | 2020-05-01T16:36:42Z | MEMBER | It should only create views if the underlying tables exist. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
610843136 | |
https://github.com/dogsheep/github-to-sqlite/issues/35#issuecomment-622213950 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35 | 622213950 | MDEyOklzc3VlQ29tbWVudDYyMjIxMzk1MA== | 9599 | 2020-05-01T02:09:04Z | 2020-05-01T02:09:04Z | MEMBER | It sped up this query a lot - 2.5s down to 300ms: ```sql select repos.full_name, json_object( 'href', 'https://github.com/' || repos.full_name || '/issues/' || issues.number, 'label', '#' || issues.number ) as issue, issues.title, users.login, users.id, issues.state, issues.locked, issues.assignee, issues.milestone, issues.comments, issues.created_at, issues.updated_at, issues.closed_at, issues.author_association, issues.pull_request, issues.repo, issues.type from issues join repos on repos.id = issues.repo join users on issues.user = users.id where issues.state = 'open' and issues.user not in (9599, 27856297) and not exists ( select id from issue_comments where issue_comments.user = 9599 and issues.id = issue_comments.issue ) order by issues.updated_at desc; ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
610511450 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549096321 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | 549096321 | MDEyOklzc3VlQ29tbWVudDU0OTA5NjMyMQ== | 9599 | 2019-11-03T01:27:55Z | 2019-11-03T01:28:17Z | MEMBER | It would be neat if this could support `--since`, with that argument automatically finding the maximum tweet ID from a previous search that used the same exact arguments (using the `search_runs` table). | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
488833975 | |
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693794700 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50 | 693794700 | MDEyOklzc3VlQ29tbWVudDY5Mzc5NDcwMA== | 9599 | 2020-09-17T04:02:39Z | 2020-09-17T04:02:39Z | MEMBER | It would be useful if you could pass an `--accept` option to this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703218756 | |
https://github.com/dogsheep/github-to-sqlite/issues/79#issuecomment-1498167714 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/79 | 1498167714 | IC_kwDODFdgUs5ZTDmi | 9599 | 2023-04-05T21:12:55Z | 2023-04-05T21:12:55Z | MEMBER | It's a rate limiting problem: https://github.com/dogsheep/github-to-sqlite/actions/runs/4622674009/jobs/8175633155 ``` File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py", line 407, in commits utils.save_commits(db, commits, repo_full["id"]) File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 635, in save_commits for commit in commits: File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 417, in fetch_commits for commits in paginate(url, headers): File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 478, in paginate raise GitHubError.from_response(response) github_to_sqlite.utils.GitHubError: ('API rate limit exceeded for user ID 9599.', 403) Error: Process completed with exit code 1. ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1570375808 | |
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695856967 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26 | 695856967 | MDEyOklzc3VlQ29tbWVudDY5NTg1Njk2Nw== | 9599 | 2020-09-21T00:26:59Z | 2020-09-21T00:26:59Z | MEMBER | It's a shame Datasette doesn't currently have an easy way to implement sorted-by-rank keyset-paginated using a TableView or QueryView. I'll have to do this using the custom SQL query constructed in the plugin: https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L8-L43 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
705215230 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-543273540 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | 543273540 | MDEyOklzc3VlQ29tbWVudDU0MzI3MzU0MA== | 9599 | 2019-10-17T17:12:51Z | 2019-10-17T17:12:51Z | MEMBER | Just importing tweets here isn't enough - how are we supposed to know which tweets were imported by which search? So I think the right thing to do here is to also create a `search_runs` table, which records each individual run of this tool (with a timestamp and the search terms used). Then have a `search_runs_tweets` m2m table which shows which Tweets were found by that search. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
488833975 | |
https://github.com/dogsheep/pocket-to-sqlite/issues/2#issuecomment-605338322 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2 | 605338322 | MDEyOklzc3VlQ29tbWVudDYwNTMzODMyMg== | 9599 | 2020-03-27T22:18:02Z | 2020-03-27T22:18:02Z | MEMBER | Just needs documentation now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
503234169 | |
https://github.com/dogsheep/pocket-to-sqlite/issues/11#issuecomment-1221621529 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11 | 1221621529 | IC_kwDODLZ_YM5I0HcZ | 9599 | 2022-08-21T21:10:15Z | 2022-08-21T21:11:26Z | MEMBER | Just saw that's what's implemented here already! - #7 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1345452427 | |
https://github.com/dogsheep/dogsheep-photos/issues/18#issuecomment-624364557 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/18 | 624364557 | MDEyOklzc3VlQ29tbWVudDYyNDM2NDU1Nw== | 9599 | 2020-05-05T23:49:18Z | 2020-05-05T23:49:18Z | MEMBER | Label is `macos-latest` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
612860758 | |
https://github.com/dogsheep/dogsheep-beta/issues/31#issuecomment-748426663 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/31 | 748426663 | MDEyOklzc3VlQ29tbWVudDc0ODQyNjY2Mw== | 9599 | 2020-12-19T06:14:06Z | 2020-12-19T06:14:06Z | MEMBER | Looks like I already do that here: https://github.com/dogsheep/dogsheep-beta/blob/9ba4401017ac24ffa3bc1db38e0910ea49de7616/dogsheep_beta/__init__.py#L141-L146 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
771316301 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694551646 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694551646 | MDEyOklzc3VlQ29tbWVudDY5NDU1MTY0Ng== | 9599 | 2020-09-17T23:22:48Z | 2020-09-17T23:22:48Z | MEMBER | Looks like its happening in a Jinja fragment template for one of the results: ``` /Users/simon/Dropbox/Development/dogsheep-beta/dogsheep_beta/__init__.py(169)process_results() -> output = compiled.render({**result, **{"json": json}}) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/asyncsupport.py(71)render() -> return original_render(self, *args, **kwargs) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(1090)render() -> self.environment.handle_exception() /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(832)handle_exception() -> reraise(*rewrite_traceback_stack(source=source)) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/_compat.py(28)reraise() -> raise value.with_traceback(tb) <template>(5)top-level template code() > /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py(341)loads() -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/swarm-to-sqlite/issues/8#issuecomment-669241341 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/8 | 669241341 | MDEyOklzc3VlQ29tbWVudDY2OTI0MTM0MQ== | 9599 | 2020-08-05T14:55:14Z | 2020-08-05T14:55:14Z | MEMBER | Looks like there's a column that's not consistently there, so the table got created without it. Easiest fix is to add `alter=True` to this line: https://github.com/dogsheep/swarm-to-sqlite/blob/f4a82633da927cde672c9d9af92930bfca2e3ddf/swarm_to_sqlite/utils.py#L94 That will cause `sqlite-utils` to notice if there's a missing column and add it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
648245071 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-543271000 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10 | 543271000 | MDEyOklzc3VlQ29tbWVudDU0MzI3MTAwMA== | 9599 | 2019-10-17T17:05:59Z | 2019-10-17T17:05:59Z | MEMBER | Looks like tqdm already does a TTY check here: https://github.com/tqdm/tqdm/blob/89b73bdc30c099c5b53725806e7edf3a121c9b3a/tqdm/std.py#L889-L890 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
492297930 | |
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790380839 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | 790380839 | MDEyOklzc3VlQ29tbWVudDc5MDM4MDgzOQ== | 9599 | 2021-03-04T07:17:05Z | 2021-03-04T07:17:05Z | MEMBER | Looks like you're doing this: ```python elif message.get_content_type() == "text/plain": body = message.get_payload(decode=True) ``` So presumably that decodes to a unicode string? I imagine the reason the column is a `BLOB` for me is that `sqlite-utils` determines the column type based on the first batch of items - https://github.com/simonw/sqlite-utils/blob/09c3386f55f766b135b6a1c00295646c4ae29bec/sqlite_utils/db.py#L1927-L1928 - and I got unlucky and had something in my first batch that wasn't a unicode string. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
813880401 | |
https://github.com/dogsheep/twitter-to-sqlite/pull/33#issuecomment-601871930 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/33 | 601871930 | MDEyOklzc3VlQ29tbWVudDYwMTg3MTkzMA== | 9599 | 2020-03-20T19:21:41Z | 2020-03-20T19:21:41Z | MEMBER | Made obsolete by https://github.com/dogsheep/twitter-to-sqlite/commit/d3887d1535c3edd59d3ebf6aea4495bacd82e59b | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
561469252 | |
https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554162 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45 | 660554162 | MDEyOklzc3VlQ29tbWVudDY2MDU1NDE2Mg== | 9599 | 2020-07-18T22:56:58Z | 2020-07-18T22:56:58Z | MEMBER | Manually fixing the database: ``` $ wget 'https://github-to-sqlite.dogsheep.net/github.db' --2020-07-18 15:52:33-- https://github-to-sqlite.dogsheep.net/github.db Resolving github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)... 172.217.5.115 Connecting to github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)|172.217.5.115|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 14626816 (14M) [application/octet-stream] Saving to: ‘github.db’ github.db 100%[============================================================================>] 13.95M 1.22MB/s in 18s 2020-07-18 15:52:53 (773 KB/s) - ‘github.db’ saved [14626816/14626816] $ sqlite3 github.db SQLite version 3.28.0 2019-04-15 14:49:49 Enter ".help" for usage hints. sqlite> drop table tags; sqlite> ^D $ github-to-sqlite tags github.db simonw/datasette simonw/sqlite-utils dogsheep/healthkit-to-sqlite dogsheep/swarm-to-sqlite dogsheep/twitter-to-sqlite dogsheep/inaturalist-to-sqlite dogsheep/google-takeout-to-sqlite dogsheep/github-to-sqlite dogsheep/genome-to-sqlite dogsheep/pocket-to-sqlite dogsheep/hacker-news-to-sqlite dogsheep/dogsheep-photos $ sqlite-utils tables github.db --counts [{"table": "users", "count": 4048}, {"table": "repos", "count": 210}, ... {"table": "stars", "count": 4140}, {"table": "tags", "count": 188}] $ sqlite-utils rows github.db tags [{"repo": 107914493, "name": "0.45", "sha": "f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb"}, {"repo": 107914493, "name": "0.45a5", "sha": "676bb64c877d73f8ff496cef4632f5a8a5a9283c"}, ... ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
660429601 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902329884 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 | 902329884 | IC_kwDODEm0Qs41yHYc | 9599 | 2021-08-20T00:01:05Z | 2021-08-20T00:01:05Z | MEMBER | Maybe Click changed something which meant that this broke things when it didn't used to? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
907645813 | |
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686153967 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9 | 686153967 | MDEyOklzc3VlQ29tbWVudDY4NjE1Mzk2Nw== | 9599 | 2020-09-03T00:17:16Z | 2020-09-03T00:17:55Z | MEMBER | Maybe I can take advantage of https://sqlite.org/np1queryprob.html here - I could define a SQL query for fetching the "display" version of each item, and include a Jinja template fragment in the configuration as well. Maybe something like this: ```yaml photos.db: photos_with_apple_metadata: sql: |- select sha256 as key, 'Photo in ' || coalesce(place_city, 'unknown') as title, ( select group_concat(normalized_string, ' ') from labels where labels.uuid = photos_with_apple_metadata.uuid ) as search_1, date as timestamp, 1 as category from photos_with_apple_metadata display_sql: |- select sha256, place_city, date from photos_with_apple_metadata where sha256 = :key display: |- <img src="https://photos.simonwillison.net/i/{{ display.sha256 }}.jpeg?w=600"> <p>Taken in {{ display.place_city }} on {{ display.date }}</p> ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
691521965 | |
https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776680 | https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5 | 706776680 | MDEyOklzc3VlQ29tbWVudDcwNjc3NjY4MA== | 9599 | 2020-10-11T22:22:16Z | 2020-10-11T22:22:16Z | MEMBER | Maybe the best way do this is with a custom route, `/-/evernote/note-id` - that way I can clean the HTML and resolve the other things in the `<en-note>` structure without using `render_cell()` and the like. My concern about using `render_cell()` is that it could lead to weird security problems when combined with `?sql=` queries. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
718938889 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/12#issuecomment-542858025 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12 | 542858025 | MDEyOklzc3VlQ29tbWVudDU0Mjg1ODAyNQ== | 9599 | 2019-10-16T19:35:31Z | 2019-10-16T19:36:09Z | MEMBER | Maybe this means I need an `upgrade` command to apply these kinds of migrations? Total feature creep! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
503053800 | |
https://github.com/dogsheep/dogsheep-beta/issues/3#issuecomment-684250044 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3 | 684250044 | MDEyOklzc3VlQ29tbWVudDY4NDI1MDA0NA== | 9599 | 2020-09-01T05:01:09Z | 2020-09-01T05:01:23Z | MEMBER | Maybe this starts out as a custom templated canned query. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
689810340 | |
https://github.com/dogsheep/apple-notes-to-sqlite/issues/1#issuecomment-1461226402 | https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/1 | 1461226402 | IC_kwDOJHON9s5XGIui | 9599 | 2023-03-09T03:47:11Z | 2023-03-09T03:47:11Z | MEMBER | Me: > Write applescript to loop through all of my Apple Notes and output their contents ChatGPT: ```applescript tell application "Notes" repeat with eachNote in every note set noteTitle to the name of eachNote set noteBody to the body of eachNote display dialog noteTitle & return & return & noteBody end repeat end tell ``` Me: > How can I turn that into a zsh script which will output the results to my terminal ChatGPT: ```zsh #!/usr/bin/zsh osascript -e 'tell application "Notes" repeat with eachNote in every note set noteTitle to the name of eachNote set noteBody to the body of eachNote set output to noteTitle & "\n" & noteBody & "\n" display dialog output log output end repeat end tell' ``` I had to change that first line to `#!/bin/zsh` - then when I ran it it spammed my screen with dialog boxes. Me: > I want to log the notes contents to the terminal, not display it in a dialog ChatGPT: ```zsh #!/bin/zsh osascript -e 'tell application "Notes" repeat with eachNote in every note set noteTitle to the name of eachNote set noteBody to the body of eachNote set output to noteTitle & "\n" & noteBody & "\n" log output end repeat end tell' ``` This really helps! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1616347574 | |
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653947916 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41 | 653947916 | MDEyOklzc3VlQ29tbWVudDY1Mzk0NzkxNg== | 9599 | 2020-07-05T22:40:47Z | 2020-07-05T22:40:47Z | MEMBER | Might be that it's not got enough RAM. I'll try deploying to a larger instance. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
651159727 | |
https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-712266834 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29 | 712266834 | MDEyOklzc3VlQ29tbWVudDcxMjI2NjgzNA== | 9599 | 2020-10-19T16:01:23Z | 2020-10-19T16:01:23Z | MEMBER | Might just be a documented pattern for how to configure this in YAML templates. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
724759588 | |
https://github.com/dogsheep/github-to-sqlite/issues/5#issuecomment-549094229 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/5 | 549094229 | MDEyOklzc3VlQ29tbWVudDU0OTA5NDIyOQ== | 9599 | 2019-11-03T00:44:03Z | 2019-11-03T00:44:03Z | MEMBER | Might not need an incomplete boolean - may be possible to handle this with `alter=True` and then by filtering for users with null values in certain columns. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
493671014 | |
https://github.com/dogsheep/twitter-to-sqlite/issues/23#issuecomment-543222239 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/23 | 543222239 | MDEyOklzc3VlQ29tbWVudDU0MzIyMjIzOQ== | 9599 | 2019-10-17T15:12:33Z | 2019-10-17T15:12:33Z | MEMBER | Migrations will run only if you open a database that previously existed (as opposed to opening a brand new empty database). This means that the first time you run a command against a fresh database, migrations will not run and the `migrations` table will not be created. The _second_ time you run any command against that database the migrations will execute and populate the `migrations` table. This also means that each migration needs to be able to sanity check the database to see if it should run or not. If it should NOT run, it will do nothing but still be marked as having executed by adding to the `migrations` table. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
508190730 | |
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626388764 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21 | 626388764 | MDEyOklzc3VlQ29tbWVudDYyNjM4ODc2NA== | 9599 | 2020-05-10T20:58:52Z | 2020-05-10T20:58:52Z | MEMBER | More from the debugger: ``` > /Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py(614)place() -> self._place = PlaceInfo5(self._info["reverse_geolocation"]) ``` And: ``` > /Users/simon/Dropbox/Development/photos-to-sqlite/photos_to_sqlite/utils.py(91)osxphoto_to_row() -> place = photo.place ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
615474990 | |
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694553579 | https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24 | 694553579 | MDEyOklzc3VlQ29tbWVudDY5NDU1MzU3OQ== | 9599 | 2020-09-17T23:28:37Z | 2020-09-17T23:28:37Z | MEMBER | More investigation in pdb: ``` (dogsheep-beta) dogsheep-beta % datasette . --get '/-/beta?q=pycon&sort=oldest' --pdb > /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py(341)loads() -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' (Pdb) list 336 if s.startswith('\ufeff'): 337 raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)", 338 s, 0) 339 else: 340 if not isinstance(s, (bytes, bytearray)): 341 -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' 342 f'not {s.__class__.__name__}') 343 s = s.decode(detect_encoding(s), 'surrogatepass') 344 345 if "encoding" in kw: 346 import warnings (Pdb) bytes <class 'bytes'> (Pdb) locals()['s'] Undefined (Pdb) type(locals()['s']) <class 'jinja2.runtime.Undefined'> ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
703970814 | |
https://github.com/dogsheep/dogsheep-photos/issues/19#issuecomment-624406285 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19 | 624406285 | MDEyOklzc3VlQ29tbWVudDYyNDQwNjI4NQ== | 9599 | 2020-05-06T02:10:03Z | 2020-05-06T02:10:03Z | MEMBER | Most annoying part of this is the difficulty of actually showing a photo. Maybe I need to run a local proxy that I can link to? A custom Datasette plugin perhaps? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
613002220 |