github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/dogsheep/twitter-to-sqlite/issues/50#issuecomment-690860653 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/50 | 690860653 | MDEyOklzc3VlQ29tbWVudDY5MDg2MDY1Mw== | 370930 | 2020-09-11T04:04:08Z | 2020-09-11T04:04:08Z | CONTRIBUTOR | There's probably a nicer way of doing (hence this is a comment rather than a PR), but this appears to fix it: ```diff --- a/twitter_to_sqlite/utils.py +++ b/twitter_to_sqlite/utils.py @@ -181,6 +181,7 @@ def fetch_timeline( args["tweet_mode"] = "extended" min_seen_id = None num_rate_limit_errors = 0 + seen_count = 0 while True: if min_seen_id is not None: args["max_id"] = min_seen_id - 1 @@ -208,6 +209,7 @@ def fetch_timeline( yield tweet min_seen_id = min(t["id"] for t in tweets) max_seen_id = max(t["id"] for t in tweets) + seen_count += len(tweets) if last_since_id is not None: max_seen_id = max((last_since_id, max_seen_id)) last_since_id = max_seen_id @@ -217,7 +219,9 @@ def fetch_timeline( replace=True, ) if stop_after is not None: - break + if seen_count >= stop_after: + break + args["count"] = min(args["count"], stop_after - seen_count) time.sleep(sleep) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
698791218 | |
https://github.com/simonw/datasette/issues/962#issuecomment-691250299 | https://api.github.com/repos/simonw/datasette/issues/962 | 691250299 | MDEyOklzc3VlQ29tbWVudDY5MTI1MDI5OQ== | 9599 | 2020-09-11T18:33:50Z | 2020-09-11T18:33:50Z | OWNER | Since this is purely a debugging option I'm going to allow myself not to write a unit test for it! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
699622046 | |
https://github.com/simonw/datasette/issues/947#issuecomment-691318133 | https://api.github.com/repos/simonw/datasette/issues/947 | 691318133 | MDEyOklzc3VlQ29tbWVudDY5MTMxODEzMw== | 9599 | 2020-09-11T21:23:40Z | 2020-09-11T21:23:40Z | OWNER | I'm going to use exit code 1 for any errors, be they 500 or 404. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
684111953 | |
https://github.com/simonw/datasette/issues/782#issuecomment-691323302 | https://api.github.com/repos/simonw/datasette/issues/782 | 691323302 | MDEyOklzc3VlQ29tbWVudDY5MTMyMzMwMg== | 9599 | 2020-09-11T21:38:27Z | 2020-09-11T21:40:04Z | OWNER | Another idea: the default output could be the list of dicts: ```json [ { "pk1": "a", "pk2": "a", "pk3": "a", "content": "a-a-a" }, ... ] ``` BUT... I could include pagination information in the HTTP headers - as seen in the WordPress REST API or the GitHub API: ``` ~ % curl -s -i 'https://api.github.com/repos/simonw/datasette/commits' | head -n 40 HTTP/1.1 200 OK server: GitHub.com date: Fri, 11 Sep 2020 21:37:46 GMT content-type: application/json; charset=utf-8 status: 200 OK cache-control: public, max-age=60, s-maxage=60 vary: Accept, Accept-Encoding, Accept, X-Requested-With etag: W/"71c99379743513394e880c6306b66bf9" last-modified: Fri, 11 Sep 2020 21:32:54 GMT x-github-media-type: github.v3; format=json link: <https://api.github.com/repositories/107914493/commits?page=2>; rel="next", <https://api.github.com/repositories/107914493/commits?page=44>; rel="last" access-control-expose-headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset access-control-allow-origin: * strict-transport-security: max-age=31536000; includeSubdomains; preload x-frame-options: deny x-content-type-options: nosniff x-xss-protection: 1; mode=block referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin content-security-policy: default-src 'none' X-Ratelimit-Limit: 60 X-Ratelimit-Remaining: 55 X-Ratelimit-Reset: 1599863850 X-Ratelimit-Used: 5 Accept-Ranges: bytes Content-Length: 118240 X-GitHub-Request-Id: EC76:0EAD:313F40:5291A4:5F5BEE37 [ { "sha": "d02f6151dae073135a22d0123e8abdc6cbef7c50", "node_id": "MDY6Q29tbWl0MTA3OTE0NDkzOmQwMmY2MTUxZGFlMDczMTM1YTIyZDAxMjNlOGFiZGM2Y2JlZjdjNTA=", "commit": { ``` Alternative shapes would provide the pagination information (and other extensions) in the JSON, e.g.: `/squirrels/squirrels.json?_shape=paginated` ```jsonā¦ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 |