{"id": 520508502, "node_id": "MDU6SXNzdWU1MjA1MDg1MDI=", "number": 31, "title": "\"friends\" command (similar to \"followers\")", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-11-09T20:20:20Z", "updated_at": "2022-09-20T05:05:03Z", "closed_at": "2020-02-07T07:03:28Z", "author_association": "MEMBER", "pull_request": null, "body": "Current list of commands:\r\n```\r\n followers Save followers for specified user (defaults to...\r\n followers-ids Populate followers table with IDs of account followers\r\n friends-ids Populate followers table with IDs of account friends\r\n```\r\nObvious omission here is `friends`, which would be powered by `https://api.twitter.com/1.1/friends/list.json`: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friends-list", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1088816961, "node_id": "I_kwDODEm0Qs5A5gdB", "number": 62, "title": "KeyError: 'created_at' for private accounts?", "user": {"value": 6764957, "label": "swyxio"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-12-26T17:51:51Z", "updated_at": "2022-03-12T02:36:32Z", "closed_at": "2022-02-24T18:10:18Z", "author_association": "NONE", "pull_request": null, "body": "hey Simon!\r\n\r\ni was running `twitter-to-sqlite user-timeline twitter.db` for [my private alt](https://twitter.com/swyxio) and ran into this error:\r\n\r\n\r\n\r\n\r\n\r\n\r\n
\r\n\r\n\r\n\r\n```bash\r\nTraceback (most recent call last):\r\n File \"/Users/swyx/Work/datasette/env/bin/twitter-to-sqlite\", line 8, in \r\n sys.exit(cli())\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py\", line 1128, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py\", line 1053, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py\", line 1659, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py\", line 1395, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py\", line 754, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/cli.py\", line 291, in user_timeline\r\n profile = utils.get_profile(db, session, **kwargs)\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py\", line 133, in get_profile\r\n save_users(db, [profile])\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py\", line 453, in save_users\r\n transform_user(user)\r\n File \"/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py\", line 285, in transform_user\r\n user[\"created_at\"] = parser.parse(user[\"created_at\"])\r\nKeyError: 'created_at'\r\n```\r\n\r\n \r\n\r\n\r\nthis looks awfully like #37 but it can't be, because i'm authed into my account and obviously i have perms to read my own account. wonder if there's any diagnostic methods i should apply here? just filing an issue for others to find while i investigate.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/62/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 984939366, "node_id": "MDU6SXNzdWU5ODQ5MzkzNjY=", "number": 58, "title": "Error: Use either --since or --since_id, not both - still broken", "user": {"value": 42904, "label": "rubenv"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-09-01T09:45:28Z", "updated_at": "2021-09-21T17:37:41Z", "closed_at": "2021-09-21T17:37:41Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Hi Simon,\r\n\r\nIt appears the fix for #57 doesn't fix things for me:\r\n\r\n```\r\n$ twitter-to-sqlite --version\r\ntwitter-to-sqlite, version 0.21.4\r\n$ python --version\r\nPython 3.9.6\r\n```\r\n\r\n```\r\n$ twitter-to-sqlite home-timeline -a twitter-auth.json twitter/timeline.db --since\r\nImporting tweets\r\nError: Use either --since or --since_id, not both\r\n```\r\n\r\nIs there any way I can help debug this?", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/58/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 984942782, "node_id": "MDExOlB1bGxSZXF1ZXN0NzI0MzE3NjUw", "number": 59, "title": "Fix for since_id bug, closes #58", "user": {"value": 42904, "label": "rubenv"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-09-01T09:49:09Z", "updated_at": "2021-09-21T17:37:40Z", "closed_at": "2021-09-21T17:37:40Z", "author_association": "CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/59", "body": "Fixes remaining instances of this bug", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/59/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 779211940, "node_id": "MDExOlB1bGxSZXF1ZXN0NTQ5MjA0MDYz", "number": 55, "title": "Fix archive imports", "user": {"value": 21148, "label": "jacobian"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-01-05T15:54:48Z", "updated_at": "2021-08-20T00:02:49Z", "closed_at": "2021-08-20T00:02:49Z", "author_association": "CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/55", "body": "This fixes the issues discussed in #54", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/55/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 681575714, "node_id": "MDExOlB1bGxSZXF1ZXN0NDY5OTQ0OTk5", "number": 49, "title": "Document the use of --stop_after with favorites, refs #20", "user": {"value": 370930, "label": "mikepqr"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-08-19T06:10:52Z", "updated_at": "2021-08-20T00:02:11Z", "closed_at": "2021-08-20T00:02:11Z", "author_association": "CONTRIBUTOR", "pull_request": "dogsheep/twitter-to-sqlite/pulls/49", "body": "(I discovered this trawling the issues for how to use --since with favorites)", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/49/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 907645813, "node_id": "MDU6SXNzdWU5MDc2NDU4MTM=", "number": 57, "title": "Error: Use either --since or --since_id, not both", "user": {"value": 42904, "label": "rubenv"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-05-31T18:11:04Z", "updated_at": "2021-08-20T00:01:31Z", "closed_at": "2021-08-20T00:01:31Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "I'm using the following command:\r\n\r\n```\r\ntwitter-to-sqlite user-timeline -a twitter-auth.json twitter/tweets.db --since\r\n```\r\n\r\nWhich gives the following error:\r\n```\r\nError: Use either --since or --since_id, not both\r\n```\r\n\r\nRunning without `--since`.\r\n\r\n```\r\nTraceback (most recent call last):\r\n File \"/usr/local/bin/twitter-to-sqlite\", line 8, in \r\n sys.exit(cli())\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1137, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1062, in main\r\n rv = self.invoke(ctx)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1668, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1404, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 763, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/twitter_to_sqlite/cli.py\", line 317, in user_timeline\r\n for tweet in bar:\r\n File \"/usr/local/lib/python3.9/site-packages/click/_termui_impl.py\", line 328, in generator\r\n for rv in self.iter:\r\n File \"/usr/local/lib/python3.9/site-packages/twitter_to_sqlite/utils.py\", line 234, in fetch_user_timeline\r\n yield from fetch_timeline(\r\n File \"/usr/local/lib/python3.9/site-packages/twitter_to_sqlite/utils.py\", line 202, in fetch_timeline\r\n raise Exception(str(tweets[\"errors\"]))\r\nException: [{'code': 44, 'message': 'since_id parameter is invalid.'}]\r\n```\r\n\r\n```\r\nPython 3.9.5\r\ntwitter-to-sqlite, version 0.21.3\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57/reactions\", \"total_count\": 4, \"+1\": 4, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 796736607, "node_id": "MDU6SXNzdWU3OTY3MzY2MDc=", "number": 56, "title": "Not all quoted statuses get fetched?", "user": {"value": 42315895, "label": "gsajko"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-01-29T09:48:44Z", "updated_at": "2021-02-03T10:36:36Z", "closed_at": "2021-02-03T10:36:36Z", "author_association": "NONE", "pull_request": null, "body": "\r\n\r\n\r\nIn my database I have 13300 quote tweets, but eta 3600 have `quoted_status` empty.\r\n\r\nI fetched some of them using `https://api.twitter.com/1.1/statuses/show.json?id=xx` and they did have ids of quoted tweets.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 771324837, "node_id": "MDU6SXNzdWU3NzEzMjQ4Mzc=", "number": 53, "title": "--since support for favorites", "user": {"value": 27, "label": "anotherjesse"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-12-19T07:08:23Z", "updated_at": "2020-12-19T07:47:11Z", "closed_at": "2020-12-19T07:47:11Z", "author_association": "NONE", "pull_request": null, "body": "Having support for `--since` for updating your favorites would be ideal as the api is both slow and it only returns ~3k most recent favorites.\r\n\r\nhttps://twittercommunity.com/t/cant-get-all-favorite-tweets-by-rest-api/22007/3\r\n\r\nThe api seems to take an optional `since_id` parameter - https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/api-reference/get-favorites-list", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/53/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 745393298, "node_id": "MDU6SXNzdWU3NDUzOTMyOTg=", "number": 52, "title": "Discussion: Adding support for fetching only fresh tweets", "user": {"value": 4169772, "label": "fatihky"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-11-18T07:01:48Z", "updated_at": "2020-11-18T07:12:45Z", "closed_at": "2020-11-18T07:12:45Z", "author_association": "NONE", "pull_request": null, "body": "I think it'd be very useful if this tool has an option like `--incremental` to fetch only newer tweets. This way operations could complete very fast in sequential runs. I'd want to try to implement this feature if it seems OK for this tool's purpose. ", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/52/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 663976976, "node_id": "MDU6SXNzdWU2NjM5NzY5NzY=", "number": 48, "title": "Add a table of contents to the README", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-07-22T18:54:33Z", "updated_at": "2020-07-23T17:46:07Z", "closed_at": "2020-07-22T19:03:02Z", "author_association": "MEMBER", "pull_request": null, "body": "Using https://github.com/jonschlinkert/markdown-toc", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 610284471, "node_id": "MDU6SXNzdWU2MTAyODQ0NzE=", "number": 46, "title": "Error running 'search' for the first time", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-30T18:11:20Z", "updated_at": "2020-04-30T18:11:58Z", "closed_at": "2020-04-30T18:11:58Z", "author_association": "MEMBER", "pull_request": null, "body": "```\r\n% twitter-to-sqlite search infodemic.db '#infodemic'\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/bin/twitter-to-sqlite\", line 11, in \r\n load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')()\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/simon/.local/share/virtualenvs/twitter-to-sqlite-PBRUqIv6/lib/python3.7/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py\", line 867, in search\r\n for tweet in tweets:\r\n File \"/Users/simon/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py\", line 165, in fetch_timeline\r\n [since_type_id, since_key],\r\nsqlite3.OperationalError: no such table: since_ids\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/46/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 602176870, "node_id": "MDU6SXNzdWU2MDIxNzY4NzA=", "number": 43, "title": "\"twitter-to-sqlite lists\" command for retrieving a user's owned lists", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-04-17T19:08:59Z", "updated_at": "2020-04-17T23:48:28Z", "closed_at": "2020-04-17T23:30:39Z", "author_association": "MEMBER", "pull_request": null, "body": "https://developer.twitter.com/en/docs/accounts-and-users/create-manage-lists/api-reference/get-lists-ownerships\r\n\r\n`https://api.twitter.com/1.1/lists/ownerships.json `", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/43/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 585353598, "node_id": "MDU6SXNzdWU1ODUzNTM1OTg=", "number": 37, "title": "Handle \"User not found\" error", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-03-20T22:14:32Z", "updated_at": "2020-04-17T23:43:46Z", "closed_at": "2020-04-17T23:43:46Z", "author_association": "MEMBER", "pull_request": null, "body": "While running `user-timeline` I got this bug (because a screen name I asked for didn't exist):\r\n```\r\n File \"/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py\", line 185, in transform_user\r\n user[\"created_at\"] = parser.parse(user[\"created_at\"])\r\nKeyError: 'created_at'\r\n>>> import pdb\r\n>>> pdb.pm()\r\n> /Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py(185)transform_user()\r\n-> user[\"created_at\"] = parser.parse(user[\"created_at\"])\r\n(Pdb) user\r\n{'errors': [{'code': 50, 'message': 'User not found.'}]}\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 602173589, "node_id": "MDU6SXNzdWU2MDIxNzM1ODk=", "number": 42, "title": "Error running user-timeline with --sql and --ids together", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-17T19:02:06Z", "updated_at": "2020-04-17T23:34:40Z", "closed_at": "2020-04-17T23:34:40Z", "author_association": "MEMBER", "pull_request": null, "body": "```\r\n$ twitter-to-sqlite user-timeline tweets.db --sql='select id from users' --ids\r\nTraceback (most recent call last):\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/bin/twitter-to-sqlite\", line 11, in \r\n load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')()\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 764, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 717, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 1137, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 956, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 555, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py\", line 284, in user_timeline\r\n \"@{:\" + str(max(len(identifier) for identifier in identifiers)) + \"}\"\r\n File \"/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py\", line 284, in \r\n \"@{:\" + str(max(len(identifier) for identifier in identifiers)) + \"}\"\r\nTypeError: object of type 'int' has no len()\r\n```\r\nBut this DID work - casting to strings:\r\n```\r\n$ twitter-to-sqlite user-timeline tweets.db --sql='select \"\" || id from users' --ids\r\n... this worked ...\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/42/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 602181581, "node_id": "MDU6SXNzdWU2MDIxODE1ODE=", "number": 44, "title": "tweet[\"source\"] can be an empty string", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-17T19:18:26Z", "updated_at": "2020-04-17T22:01:44Z", "closed_at": "2020-04-17T22:01:44Z", "author_association": "MEMBER", "pull_request": null, "body": "Got this excepion:\r\n```\r\n File \"/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py\", line 641, in extract_and_save_source\r\n details = m.groupdict()\r\nAttributeError: 'NoneType' object has no attribute 'groupdict'\r\n```\r\nI traced it back to this tweet: https://twitter.com/osder/status/578712651393576960\r\n```\r\n(Pdb) source_re\r\nre.compile('.*?)\".*?>(?P.*?)')\r\n(Pdb) locals()['source']\r\n''\r\n(Pdb) u\r\n> /Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py(393)save_tweets()\r\n-> tweet[\"source\"] = extract_and_save_source(db, tweet[\"source\"])\r\n(Pdb) tweet\r\n{'created_at': '2015-03-20T00:20:22+00:00', 'id': 578712651393576960, 'full_text': '@osder', 'truncated': False, 'display_text_range': [0, 6], 'source': '', 'in_reply_to_status_id': 578712521382715392, 'in_reply_to_user_id': 1545741, 'in_reply_to_screen_name': 'osder', 'geo': None, 'coordinates': None, 'place': None, 'contributors': None, 'is_quote_status': False, 'retweet_count': 0, 'favorite_count': 0, 'favorited': False, 'retweeted': False, 'lang': 'und', 'user': 1545741}\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/44/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 591613579, "node_id": "MDU6SXNzdWU1OTE2MTM1Nzk=", "number": 41, "title": "Bug: recorded a since_id for None, None", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-04-01T04:29:43Z", "updated_at": "2020-04-01T04:31:11Z", "closed_at": "2020-04-01T04:31:11Z", "author_association": "MEMBER", "pull_request": null, "body": "This shouldn't happen in the `since_ids` table (relates to #39):\r\n\r\n![\"twitter__since_ids__2_rows\"](\"https://user-images.githubusercontent.com/9599/78099419-b0fed200-7396-11ea-87d8-992a171d6fab.png\")
\r\n", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/41/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 590669793, "node_id": "MDU6SXNzdWU1OTA2Njk3OTM=", "number": 40, "title": "Feature: record history of follower counts", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-03-30T23:32:28Z", "updated_at": "2020-04-01T04:13:05Z", "closed_at": "2020-04-01T04:13:05Z", "author_association": "MEMBER", "pull_request": null, "body": "We currently over-write the follower count every time we import a tweet (when we import that user profile again):\r\n\r\nhttps://github.com/dogsheep/twitter-to-sqlite/blob/810cb2af5a175837204389fd7f4b5721f8b325ab/twitter_to_sqlite/utils.py#L293-L294\r\n\r\nIt would be neat if we noticed if that user's follower count (and maybe other counts?) had changed since we last saved them and recorded that change in a separate history table. This would be an inexpensive way of building up rough charts of follower count over time.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 492297930, "node_id": "MDU6SXNzdWU0OTIyOTc5MzA=", "number": 10, "title": "Rethink progress bars for various commands", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2019-09-11T15:06:47Z", "updated_at": "2020-04-01T03:45:48Z", "closed_at": "2020-04-01T03:45:48Z", "author_association": "MEMBER", "pull_request": null, "body": "Progress bars and the `--silent` option are implemented inconsistently across commands at the moment.\r\n\r\nThis is made more challenging by the fact that for many operations the total length is not known.\r\n\r\nhttps://click.palletsprojects.com/en/7.x/api/#click.progressbar", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 590666760, "node_id": "MDU6SXNzdWU1OTA2NjY3NjA=", "number": 39, "title": "--since feature can be confused by retweets", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 11, "created_at": "2020-03-30T23:25:33Z", "updated_at": "2020-04-01T03:45:16Z", "closed_at": "2020-04-01T03:45:16Z", "author_association": "MEMBER", "pull_request": null, "body": "If you run `twitter-to-sqlite user-timeline ... --since` it's supposed to fetch Tweets those specific users tweeted since last time the command was run.\r\n\r\nIt does this by seeking out the max ID of their previous tweets:\r\n\r\nhttps://github.com/dogsheep/twitter-to-sqlite/blob/810cb2af5a175837204389fd7f4b5721f8b325ab/twitter_to_sqlite/cli.py#L305-L311\r\n\r\nBUT... this has a nasty flaw: if another account had retweeted one of their recent tweets the retweeted-tweet will have been loaded into the database - so we may treat that as the most recent since ID and miss a bunch of their tweets!", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 490803176, "node_id": "MDU6SXNzdWU0OTA4MDMxNzY=", "number": 8, "title": "--sql and --attach options for feeding commands from SQL queries", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-09-08T20:35:49Z", "updated_at": "2020-03-20T23:13:01Z", "closed_at": "2020-03-20T23:13:01Z", "author_association": "MEMBER", "pull_request": null, "body": "Say you want to fetch Twitter profiles for a list of accounts that are stored in another database:\r\n\r\n $ twitter-to-sqlite users-lookup users.db --attach attending.db \\\r\n --sql \"select Twitter from attending.attendes where Twitter is not null\"\r\n\r\nThe SQL query you feed in is expected to return a list of screen names suitable for processing further by the command.\r\n\r\nShould be supported by all three of:\r\n\r\n- [x] `twitter-to-sqlite users-lookup`\r\n- [x] `twitter-to-sqlite user-timeline`\r\n- [x] `twitter-to-sqlite followers` and `friends`\r\n\r\nThe `--attach` option allows other SQLite databases to be attached to the connection. Without it the SQL query will have to read from the single attached database.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 585306847, "node_id": "MDU6SXNzdWU1ODUzMDY4NDc=", "number": 36, "title": "twitter-to-sqlite followers/friends --sql / --attach", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-20T20:20:33Z", "updated_at": "2020-03-20T23:12:38Z", "closed_at": "2020-03-20T23:12:38Z", "author_association": "MEMBER", "pull_request": null, "body": "Split from #8. The `friends` and `followers` commands don't yet support `--sql` and `--attach`.\r\n\r\n(`friends-ids` and `followers-ids` do though).", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/36/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 585359363, "node_id": "MDU6SXNzdWU1ODUzNTkzNjM=", "number": 38, "title": "Screen name display for user-timeline is uneven", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-03-20T22:30:23Z", "updated_at": "2020-03-20T22:37:17Z", "closed_at": "2020-03-20T22:37:17Z", "author_association": "MEMBER", "pull_request": null, "body": "```\r\nCDPHE [####################################] 67\r\nCHFSKy [####################################] 3216\r\nDHSWI [####################################] 41\r\nDPHHSMT [####################################] 742\r\nDelaware_DHSS [####################################] 3231\r\nDhhsNevada [####################################] 639\r\n```\r\nI could format them to match the length of the longest screen name instead.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/38/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 585282212, "node_id": "MDU6SXNzdWU1ODUyODIyMTI=", "number": 35, "title": "twitter-to-sqlite user-timeline [screen_names] --sql / --attach", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 5, "created_at": "2020-03-20T19:26:07Z", "updated_at": "2020-03-20T20:17:00Z", "closed_at": "2020-03-20T20:16:35Z", "author_association": "MEMBER", "pull_request": null, "body": "Split from #8.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 561469252, "node_id": "MDExOlB1bGxSZXF1ZXN0MzcyMjczNjA4", "number": 33, "title": "Upgrade to sqlite-utils 2.2.1", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-02-07T07:32:12Z", "updated_at": "2020-03-20T19:21:42Z", "closed_at": "2020-03-20T19:21:41Z", "author_association": "MEMBER", "pull_request": "dogsheep/twitter-to-sqlite/pulls/33", "body": "", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/33/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 585266763, "node_id": "MDU6SXNzdWU1ODUyNjY3NjM=", "number": 34, "title": "IndexError running user-timeline command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-03-20T18:54:08Z", "updated_at": "2020-03-20T19:20:52Z", "closed_at": "2020-03-20T19:20:37Z", "author_association": "MEMBER", "pull_request": null, "body": "```\r\n$ twitter-to-sqlite user-timeline data.db --screen_name Allen_Joines\r\nTraceback (most recent call last):\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/bin/twitter-to-sqlite\", line 11, in \r\n load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')()\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 764, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 717, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 1137, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 956, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py\", line 555, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py\", line 256, in user_timeline\r\n utils.save_tweets(db, chunk)\r\n File \"/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py\", line 289, in save_tweets\r\n db[\"users\"].upsert(user, pk=\"id\", alter=True)\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1128, in upsert\r\n conversions=conversions,\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1157, in upsert_all\r\n upsert=True,\r\n File \"/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1096, in insert_all\r\n row = list(self.rows_where(\"rowid = ?\", [self.last_rowid]))[0]\r\nIndexError: list index out of range\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 561454071, "node_id": "MDU6SXNzdWU1NjE0NTQwNzE=", "number": 32, "title": "Documentation for \" favorites\" command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-02-07T06:50:11Z", "updated_at": "2020-02-07T06:59:10Z", "closed_at": "2020-02-07T06:59:10Z", "author_association": "MEMBER", "pull_request": null, "body": "It looks like I forgot to document this one in the README.\r\n\r\nhttps://github.com/dogsheep/twitter-to-sqlite/blob/6ebd482619bd94180e54bb7b56549c413077d329/twitter_to_sqlite/cli.py#L183-L194", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/32/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 518725064, "node_id": "MDU6SXNzdWU1MTg3MjUwNjQ=", "number": 29, "title": "`import` command fails on empty files", "user": {"value": 21148, "label": "jacobian"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-11-06T20:34:26Z", "updated_at": "2019-11-09T20:33:38Z", "closed_at": "2019-11-09T19:36:36Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "If a file in the export is empty (in my case it was `account-suspensions.js`), `twitter-to-sqlite import` fails:\r\n\r\n```\r\n$ twitter-to-sqlite import twitter.db ~/Downloads/twitter-2019-11-06-926f4f3be4b3b1fcb1aa387c40cd14f7c8aaf9bbcdb2d78ac14d9989add501bb.zip\r\nTraceback (most recent call last):\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/bin/twitter-to-sqlite\", line 10, in \r\n sys.exit(cli())\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 764, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 717, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 1137, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 956, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 555, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/twitter_to_sqlite/cli.py\", line 627, in import_\r\n archive.import_from_file(db, filename, content)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/twitter_to_sqlite/archive.py\", line 224, in import_from_file\r\n db[table_name].upsert_all(rows, hash_id=\"pk\")\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1113, in upsert_all\r\n extracts=extracts,\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/sqlite_utils/db.py\", line 980, in insert_all\r\n first_record = next(records)\r\nStopIteration\r\n```\r\n\r\nThis appears to be because `db.upsert_all` is called with no rows -- I think? \r\n\r\nI hacked around this by modifying `import_from_file` to have an `if rows:` clause:\r\n\r\n```\r\n for table, rows in to_insert.items():\r\n if rows:\r\n table_name = \"archive_{}\".format(table.replace(\"-\", \"_\"))\r\n ...\r\n```\r\n\r\nI'm happy to work up a real PR if that's the right approach, but I'm not sure it is.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 515658861, "node_id": "MDU6SXNzdWU1MTU2NTg4NjE=", "number": 28, "title": "Add indexes to followers table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-10-31T18:40:22Z", "updated_at": "2019-11-09T20:15:42Z", "closed_at": "2019-11-09T20:11:48Z", "author_association": "MEMBER", "pull_request": null, "body": "`select follower_id from following where followed_id = 12497` takes over a second for me at the moment.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/28/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 518739697, "node_id": "MDU6SXNzdWU1MTg3Mzk2OTc=", "number": 30, "title": "`followers` fails because `transform_user` is called twice", "user": {"value": 21148, "label": "jacobian"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-11-06T20:44:52Z", "updated_at": "2019-11-09T20:15:28Z", "closed_at": "2019-11-09T19:55:52Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "Trying to run `twitter-to-sqlite followers` errors out:\r\n\r\n```\r\nTraceback (most recent call last):\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/bin/twitter-to-sqlite\", line 10, in \r\n sys.exit(cli())\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 764, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 717, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 1137, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 956, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/click/core.py\", line 555, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/twitter_to_sqlite/cli.py\", line 130, in followers\r\n go(bar.update)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/twitter_to_sqlite/cli.py\", line 116, in go\r\n utils.save_users(db, [profile])\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/twitter_to_sqlite/utils.py\", line 302, in save_users\r\n transform_user(user)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/twitter_to_sqlite/utils.py\", line 181, in transform_user\r\n user[\"created_at\"] = parser.parse(user[\"created_at\"])\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/dateutil/parser/_parser.py\", line 1374, in parse\r\n return DEFAULTPARSER.parse(timestr, **kwargs)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/dateutil/parser/_parser.py\", line 646, in parse\r\n res, skipped_tokens = self._parse(timestr, **kwargs)\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/dateutil/parser/_parser.py\", line 725, in _parse\r\n l = _timelex.split(timestr) # Splits the timestr into tokens\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/dateutil/parser/_parser.py\", line 207, in split\r\n return list(cls(s))\r\n File \"/Users/jacob/Library/Caches/pypoetry/virtualenvs/jkm-dogsheep-ezLnyXZS-py3.7/lib/python3.7/site-packages/dateutil/parser/_parser.py\", line 76, in __init__\r\n '{itype}'.format(itype=instream.__class__.__name__))\r\nTypeError: Parser must be a string or character stream, not datetime\r\n```\r\n\r\nThis appears to be because https://github.com/dogsheep/twitter-to-sqlite/blob/master/twitter_to_sqlite/cli.py#L111 calls `transform_user`, and then https://github.com/dogsheep/twitter-to-sqlite/blob/master/twitter_to_sqlite/cli.py#L116 calls `transform_user` again, which fails because the user is already transformed.\r\n\r\nI was able to work around this by commenting out https://github.com/dogsheep/twitter-to-sqlite/blob/master/twitter_to_sqlite/cli.py#L116. \r\n\r\nShall I work up a patch for that, or is there a better approach?", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/30/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 488833975, "node_id": "MDU6SXNzdWU0ODg4MzM5NzU=", "number": 3, "title": "Command for running a search and saving tweets for that search", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-09-03T21:29:56Z", "updated_at": "2019-11-04T05:31:56Z", "closed_at": "2019-11-04T05:31:16Z", "author_association": "MEMBER", "pull_request": null, "body": " $ twitter-to-sqlite search dogsheep", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 514459062, "node_id": "MDU6SXNzdWU1MTQ0NTkwNjI=", "number": 27, "title": "retweets-of-me command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-10-30T07:43:01Z", "updated_at": "2019-11-03T01:12:58Z", "closed_at": "2019-11-03T01:12:58Z", "author_association": "MEMBER", "pull_request": null, "body": "https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-retweets_of_me", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 513074501, "node_id": "MDU6SXNzdWU1MTMwNzQ1MDE=", "number": 26, "title": "Command for importing mentions timeline", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-10-28T03:14:27Z", "updated_at": "2019-10-30T02:36:13Z", "closed_at": "2019-10-30T02:20:47Z", "author_association": "MEMBER", "pull_request": null, "body": "https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-mentions_timeline\r\n\r\nAlmost identical to home-timeline #18 but it uses `https://api.twitter.com/1.1/statuses/mentions_timeline.json` instead.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/26/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 506268945, "node_id": "MDU6SXNzdWU1MDYyNjg5NDU=", "number": 20, "title": "--since support for various commands for refresh-by-cron", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-10-13T03:40:46Z", "updated_at": "2019-10-21T03:32:04Z", "closed_at": "2019-10-16T19:26:11Z", "author_association": "MEMBER", "pull_request": null, "body": "I want to run a cron that updates my Twitter database every X minutes.\r\n\r\nIt should be able to retrieve the following without needing to paginate through everything:\r\n\r\n- [x] Tweets I have tweeted\r\n- [x] My home timeline (see #19)\r\n- [x] Tweets I have favourited\r\n\r\nIt would be nice if this could be standardized across all commands as a `--since` option.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 508190730, "node_id": "MDU6SXNzdWU1MDgxOTA3MzA=", "number": 23, "title": "Extremely simple migration system", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-10-17T02:13:57Z", "updated_at": "2019-10-17T16:57:17Z", "closed_at": "2019-10-17T16:57:17Z", "author_association": "MEMBER", "pull_request": null, "body": "Needed for #12. This is going to be an incredibly simple version of the Django migration system.\r\n\r\n* A `migrations` table, keeping track of which migrations were applied (and when)\r\n* A `migrate()` function which applies any pending migrations\r\n* A `MIGRATIONS` constant which is a list of functions to be applied\r\n\r\nThe function names will be detected and used as the names of the migrations.\r\n\r\nEvery time you run the CLI tool it will call the `migrate()` function before doing anything else.\r\n\r\nNeeds to take into account that there might be no tables at all. As such, migration functions should sanity check that the tables they are going to work on actually exist.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/23/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 508578780, "node_id": "MDU6SXNzdWU1MDg1Nzg3ODA=", "number": 25, "title": "Ensure migrations don't accidentally create foreign key twice", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-10-17T16:08:50Z", "updated_at": "2019-10-17T16:56:47Z", "closed_at": "2019-10-17T16:56:47Z", "author_association": "MEMBER", "pull_request": null, "body": "Is it possible for these lines to run against a database table that already has these foreign keys?\r\n\r\nhttps://github.com/dogsheep/twitter-to-sqlite/blob/c9295233f219c446fa2085cace987067488a31b9/twitter_to_sqlite/migrations.py#L21-L22", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/25/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 508553387, "node_id": "MDExOlB1bGxSZXF1ZXN0MzI5MzI0MzY4", "number": 24, "title": "Tweet source extraction and new migration system", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-10-17T15:24:56Z", "updated_at": "2019-10-17T15:49:29Z", "closed_at": "2019-10-17T15:49:24Z", "author_association": "MEMBER", "pull_request": "dogsheep/twitter-to-sqlite/pulls/24", "body": "Closes #12 and #23", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/24/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 503053800, "node_id": "MDU6SXNzdWU1MDMwNTM4MDA=", "number": 12, "title": "Extract \"source\" into a separate lookup table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-10-06T05:17:23Z", "updated_at": "2019-10-17T15:49:24Z", "closed_at": "2019-10-17T15:49:24Z", "author_association": "MEMBER", "pull_request": null, "body": "It's pretty bulky and ugly at the moment:\r\n\r\n![\"trump__tweets__1_820_rows\"](\"https://user-images.githubusercontent.com/9599/66264630-df23a080-e7bd-11e9-9154-403c2e69f841.png\")
\r\n", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 506087267, "node_id": "MDU6SXNzdWU1MDYwODcyNjc=", "number": 19, "title": "since_id support for home-timeline", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-10-11T22:48:24Z", "updated_at": "2019-10-16T19:13:06Z", "closed_at": "2019-10-16T19:12:46Z", "author_association": "MEMBER", "pull_request": null, "body": "Currently every time you run `home-timeline` we pull all 800 available tweets. We should offer to support `since_id` (which can be provided or can be pulled directly from the database) in order to work more efficiently if this command is executed e.g. on a cron.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 508024032, "node_id": "MDU6SXNzdWU1MDgwMjQwMzI=", "number": 22, "title": "Ability to import from uncompressed archive or from specific files", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-10-16T18:31:57Z", "updated_at": "2019-10-16T18:53:36Z", "closed_at": "2019-10-16T18:53:36Z", "author_association": "MEMBER", "pull_request": null, "body": "Currently you can only import like this:\r\n\r\n $ twitter-to-sqlite import path-to-twitter.zip\r\n\r\nIt would be useful if you could import from a folder that was decompressed from that zip:\r\n\r\n $ twitter-to-sqlite import path-to-twitter/\r\n\r\nAND from individual files within that folder - since that would allow you to e.g. selectively import certain files:\r\n\r\n $ twitter-to-sqlite import path-to-twitter/favorites.js path-to-twitter/tweets.js", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/22/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 506432572, "node_id": "MDU6SXNzdWU1MDY0MzI1NzI=", "number": 21, "title": "Fix & escapes in tweet text", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-10-14T03:37:28Z", "updated_at": "2019-10-15T18:48:16Z", "closed_at": "2019-10-15T18:48:16Z", "author_association": "MEMBER", "pull_request": null, "body": "![\"twitter__tweets__21_773_rows_where_sorted_by_id_descending\"](\"https://user-images.githubusercontent.com/9599/66728360-38f91b80-edf9-11e9-95b5-ce6d097fe18e.png\")
\r\n\r\nShouldn't be storing `&` here.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/21/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 503244410, "node_id": "MDU6SXNzdWU1MDMyNDQ0MTA=", "number": 14, "title": "When importing favorites, record which user favorited them", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-10-07T05:45:11Z", "updated_at": "2019-10-14T03:30:25Z", "closed_at": "2019-10-14T03:30:25Z", "author_association": "MEMBER", "pull_request": null, "body": "This code currently just dumps them into the `tweets` table without recording who it was who had favorited them.\r\n\r\nhttps://github.com/dogsheep/twitter-to-sqlite/blob/436a170d74ec70903d1b4ca430c2c6b6435cdfcc/twitter_to_sqlite/cli.py#L152-L157", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/14/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 505928530, "node_id": "MDU6SXNzdWU1MDU5Mjg1MzA=", "number": 18, "title": "Command to import home-timeline", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-10-11T15:47:54Z", "updated_at": "2019-10-11T16:51:33Z", "closed_at": "2019-10-11T16:51:12Z", "author_association": "MEMBER", "pull_request": null, "body": "Feature request: https://twitter.com/johankj/status/1182563563136868352\r\n> Would it be possible to save all tweets in my timeline from the last X days? I would love to see how big a percentage some users are of my daily timeline as a metric on whether I should unfollow them/move them to a list.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 505674949, "node_id": "MDU6SXNzdWU1MDU2NzQ5NDk=", "number": 17, "title": "import command should empty all archive-* tables first", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-10-11T06:58:43Z", "updated_at": "2019-10-11T15:40:08Z", "closed_at": "2019-10-11T15:40:08Z", "author_association": "MEMBER", "pull_request": null, "body": "Can have a CLI option for NOT doing that.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 488835586, "node_id": "MDU6SXNzdWU0ODg4MzU1ODY=", "number": 4, "title": "Command for importing data from a Twitter Export file", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-09-03T21:34:13Z", "updated_at": "2019-10-11T06:45:02Z", "closed_at": "2019-10-11T06:45:02Z", "author_association": "MEMBER", "pull_request": null, "body": "Twitter lets you export all of your data as an archive file: https://twitter.com/settings/your_twitter_data\r\n\r\nA command for importing this data into SQLite would be extremely useful.\r\n\r\n $ twitter-to-sqlite import twitter.db path-to-archive.zip\r\n", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/4/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 505666744, "node_id": "MDExOlB1bGxSZXF1ZXN0MzI3MDUxNjcz", "number": 15, "title": "twitter-to-sqlite import command, refs #4", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-10-11T06:37:14Z", "updated_at": "2019-10-11T06:45:01Z", "closed_at": "2019-10-11T06:45:01Z", "author_association": "MEMBER", "pull_request": "dogsheep/twitter-to-sqlite/pulls/15", "body": "", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/15/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 503085013, "node_id": "MDU6SXNzdWU1MDMwODUwMTM=", "number": 13, "title": "statuses-lookup command", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-10-06T11:00:20Z", "updated_at": "2019-10-07T00:33:49Z", "closed_at": "2019-10-07T00:31:44Z", "author_association": "MEMBER", "pull_request": null, "body": "For bulk retrieving tweets by their ID.\r\n\r\nhttps://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-lookup\r\n\r\nRate limit is 900/15 minutes (1 call per second) but each call can pull up to 100 IDs, so we can pull 6,000 per minute.\r\n\r\nShould support `--SQL` and `--attach` #8 ", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/13/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 503045221, "node_id": "MDU6SXNzdWU1MDMwNDUyMjE=", "number": 11, "title": "Commands for recording real-time tweets from the streaming API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-10-06T03:09:30Z", "updated_at": "2019-10-06T04:54:17Z", "closed_at": "2019-10-06T04:48:31Z", "author_association": "MEMBER", "pull_request": null, "body": "https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter\r\n\r\nWe can support tracking keywords and following specific users.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/11/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 491791152, "node_id": "MDU6SXNzdWU0OTE3OTExNTI=", "number": 9, "title": "followers-ids and friends-ids subcommands", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-09-10T16:58:15Z", "updated_at": "2019-09-10T17:36:55Z", "closed_at": "2019-09-10T17:36:55Z", "author_association": "MEMBER", "pull_request": null, "body": "These will import follower and friendship IDs into the following tables, using these APIs:\r\n\r\nhttps://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-followers-ids\r\nhttps://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friends-ids", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/9/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 490798130, "node_id": "MDU6SXNzdWU0OTA3OTgxMzA=", "number": 7, "title": "users-lookup command for fetching users", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-09-08T19:47:59Z", "updated_at": "2019-09-08T20:32:13Z", "closed_at": "2019-09-08T20:32:13Z", "author_association": "MEMBER", "pull_request": null, "body": "https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-users-lookup\r\n```\r\nhttps://api.twitter.com/1.1/users/lookup.json?user_id=783214,6253282\r\nhttps://api.twitter.com/1.1/users/lookup.json?screen_name=simonw,cleopaws\r\n```\r\nCLI design:\r\n```\r\n$ twitter-to-sqlite users-lookup simonw cleopaws\r\n$ twitter-to-sqlite users-lookup 783214 6253282 --ids\r\n```", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/7/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 489419782, "node_id": "MDU6SXNzdWU0ODk0MTk3ODI=", "number": 6, "title": "Extract extended_entities into a media table", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-09-04T21:59:10Z", "updated_at": "2019-09-04T22:08:01Z", "closed_at": "2019-09-04T22:08:01Z", "author_association": "MEMBER", "pull_request": null, "body": "![\"cleo__select___from_tweets_order_by_id_limit_101\"](\"https://user-images.githubusercontent.com/9599/64294799-880f7f00-cf24-11e9-83b8-75c1ee54182c.png\")
\r\n", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/6/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 488833136, "node_id": "MDU6SXNzdWU0ODg4MzMxMzY=", "number": 1, "title": "Imported followers should go in \"users\", relationships in \"following\"", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-09-03T21:27:37Z", "updated_at": "2019-09-04T20:23:04Z", "closed_at": "2019-09-04T20:23:04Z", "author_association": "MEMBER", "pull_request": null, "body": "Right now `twitter-to-sqlite followers` dumps everything in a `followers` table, and doesn't actually record which account they are following!\r\n\r\nIt should instead save them all in a global `users` table and then set up m2m relationships in a `following` table. This also means it should create a record for the specified user in order to record both sides of each relationship.", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/1/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 488833698, "node_id": "MDU6SXNzdWU0ODg4MzM2OTg=", "number": 2, "title": "\"twitter-to-sqlite user-timeline\" command for pulling tweets by a specific user", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2019-09-03T21:29:12Z", "updated_at": "2019-09-04T20:02:11Z", "closed_at": "2019-09-04T20:02:11Z", "author_association": "MEMBER", "pull_request": null, "body": "Twitter only allows up to 3,200 tweets to be retrieved from https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html\r\n\r\nI'm going to do:\r\n\r\n $ twitter-to-sqlite tweets simonw\r\n\r\n", "repo": {"value": 206156866, "label": "twitter-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}