home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 606998669

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions issue performed_via_github_app
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606998669 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 606998669 MDEyOklzc3VlQ29tbWVudDYwNjk5ODY2OQ== 9599 2020-04-01T02:57:36Z 2020-04-01T02:57:36Z MEMBER

The tricky thing here is thinking about the interaction between the recorded since_id and a desire to run the initial import.

The first time you run twitter-to-sqlite user-timeline db.db username we want to fetch as many tweets from that user as possible - probably around 3,200 before the API limitations cut us off.

We need to record the maximum ID from those as the since_id - which we will see on the very first page we paginate through. That way next time we run the command with --since we will only fetch new tweets.

But what happens if our initial import is cancelled after only a few tweets? We risk never pulling in the rest of the tweets.

Not sure if I need to solve this at all or if I should instead trust users to run the command a second time without --since if they think they didn't retrieve anything the first time.

I had considered letting --stop_after= over-ride --since but that doesn't actually make sense - if you send a since_id to the Twitter API you'll never get back more tweets than exist after that ID, so the --stop_after would not make a meaningful difference.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
590666760  
Powered by Datasette · Queries took 1.356ms · About: github-to-sqlite