issue_comments
525 rows where author_association = "MEMBER" sorted by html_url descending
This data as json, CSV (advanced)
Suggested facets: issue_url, body, reactions, created_at (date), updated_at (date)
user 1
- simonw 525
id | html_url ▲ | issue_url | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
924209583 | https://github.com/dogsheep/twitter-to-sqlite/pull/59#issuecomment-924209583 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/59 | IC_kwDODEm0Qs43FlGv | simonw 9599 | 2021-09-21T17:37:34Z | 2021-09-21T17:37:34Z | MEMBER | Thanks for this! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fix for since_id bug, closes #58 984942782 | |
902330301 | https://github.com/dogsheep/twitter-to-sqlite/pull/49#issuecomment-902330301 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/49 | IC_kwDODEm0Qs41yHe9 | simonw 9599 | 2021-08-20T00:01:56Z | 2021-08-20T00:01:56Z | MEMBER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document the use of --stop_after with favorites, refs #20 681575714 | |
601871930 | https://github.com/dogsheep/twitter-to-sqlite/pull/33#issuecomment-601871930 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/33 | MDEyOklzc3VlQ29tbWVudDYwMTg3MTkzMA== | simonw 9599 | 2020-03-20T19:21:41Z | 2020-03-20T19:21:41Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to sqlite-utils 2.2.1 561469252 | ||
530028567 | https://github.com/dogsheep/twitter-to-sqlite/issues/9#issuecomment-530028567 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/9 | MDEyOklzc3VlQ29tbWVudDUzMDAyODU2Nw== | simonw 9599 | 2019-09-10T16:59:25Z | 2019-09-10T16:59:25Z | MEMBER | By default in SQLite foreign key constraints are not enforced (you need to run We will take advantage of this - even though the In the future we may add a command that can backfill missing user records. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
followers-ids and friends-ids subcommands 491791152 | |
531404891 | https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-531404891 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDUzMTQwNDg5MQ== | simonw 9599 | 2019-09-13T22:01:57Z | 2019-09-13T22:01:57Z | MEMBER | I also wrote about this in https://simonwillison.net/2019/Sep/13/weeknotestwitter-sqlite-datasette-rure/ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--sql and --attach options for feeding commands from SQL queries 490803176 | |
530417631 | https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-530417631 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDUzMDQxNzYzMQ== | simonw 9599 | 2019-09-11T14:52:44Z | 2019-09-14T19:09:22Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--sql and --attach options for feeding commands from SQL queries 490803176 | |
529240286 | https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-529240286 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDUyOTI0MDI4Ng== | simonw 9599 | 2019-09-08T20:48:33Z | 2019-09-08T20:48:33Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--sql and --attach options for feeding commands from SQL queries 490803176 | |
529239307 | https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-529239307 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDUyOTIzOTMwNw== | simonw 9599 | 2019-09-08T20:36:49Z | 2019-09-08T20:36:49Z | MEMBER |
If you omit the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--sql and --attach options for feeding commands from SQL queries 490803176 | |
902329884 | https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902329884 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 | IC_kwDODEm0Qs41yHYc | simonw 9599 | 2021-08-20T00:01:05Z | 2021-08-20T00:01:05Z | MEMBER | Maybe Click changed something which meant that this broke things when it didn't used to? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error: Use either --since or --since_id, not both 907645813 | |
902329455 | https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902329455 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 | IC_kwDODEm0Qs41yHRv | simonw 9599 | 2021-08-19T23:59:56Z | 2021-08-19T23:59:56Z | MEMBER | This looks like the bug to me:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error: Use either --since or --since_id, not both 907645813 | |
902328760 | https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902328760 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 | IC_kwDODEm0Qs41yHG4 | simonw 9599 | 2021-08-19T23:57:41Z | 2021-08-19T23:57:41Z | MEMBER | Weird, added debug code and got this: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error: Use either --since or --since_id, not both 907645813 | |
902328369 | https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902328369 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 | IC_kwDODEm0Qs41yHAx | simonw 9599 | 2021-08-19T23:56:26Z | 2021-08-19T23:56:26Z | MEMBER | https://developer.twitter.com/en/docs/twitter-api/v1/tweets/timelines/api-reference/get-statuses-user_timeline says the API has been replaced by the new v2 one, but it should still work - and the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error: Use either --since or --since_id, not both 907645813 | |
902327457 | https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902327457 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 | IC_kwDODEm0Qs41yGyh | simonw 9599 | 2021-08-19T23:53:25Z | 2021-08-19T23:53:25Z | MEMBER | I'm getting this too. Looking into it now. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error: Use either --since or --since_id, not both 907645813 | |
769957751 | https://github.com/dogsheep/twitter-to-sqlite/issues/56#issuecomment-769957751 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56 | MDEyOklzc3VlQ29tbWVudDc2OTk1Nzc1MQ== | simonw 9599 | 2021-01-29T17:59:40Z | 2021-01-29T17:59:40Z | MEMBER | This is interesting - how did you create that initial table? Was this using the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Not all quoted statuses get fetched? 796736607 | |
527684202 | https://github.com/dogsheep/twitter-to-sqlite/issues/5#issuecomment-527684202 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDUyNzY4NDIwMg== | simonw 9599 | 2019-09-03T23:56:28Z | 2019-09-03T23:56:28Z | MEMBER | I previously used betamax here: https://github.com/simonw/github-contents/blob/master/test_github_contents.py |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Write tests that simulate the Twitter API 488874815 | |
663143160 | https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-663143160 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48 | MDEyOklzc3VlQ29tbWVudDY2MzE0MzE2MA== | simonw 9599 | 2020-07-23T17:46:07Z | 2020-07-23T17:46:07Z | MEMBER | Frustratingly, these links don't work on PyPI: https://pypi.org/project/twitter-to-sqlite/ There's an issue about that here: https://github.com/pypa/readme_renderer/issues/169 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add a table of contents to the README 663976976 | |
662630868 | https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-662630868 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48 | MDEyOklzc3VlQ29tbWVudDY2MjYzMDg2OA== | simonw 9599 | 2020-07-22T19:03:02Z | 2020-07-22T19:03:02Z | MEMBER | Done! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add a table of contents to the README 663976976 | |
662626901 | https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-662626901 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48 | MDEyOklzc3VlQ29tbWVudDY2MjYyNjkwMQ== | simonw 9599 | 2020-07-22T18:54:53Z | 2020-07-22T18:54:53Z | MEMBER | I'm going to use a GitHub Action to run |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add a table of contents to the README 663976976 | |
645599881 | https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645599881 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47 | MDEyOklzc3VlQ29tbWVudDY0NTU5OTg4MQ== | simonw 9599 | 2020-06-17T20:13:48Z | 2020-06-17T20:13:48Z | MEMBER | I've now figured out how to compile specific SQLite versions to help replicate this problem: https://github.com/simonw/til/blob/master/sqlite/ld-preload.md Next step: replicate the problem! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fall back to FTS4 if FTS5 is not available 639542974 | |
645512127 | https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645512127 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47 | MDEyOklzc3VlQ29tbWVudDY0NTUxMjEyNw== | simonw 9599 | 2020-06-17T17:24:22Z | 2020-06-17T17:24:22Z | MEMBER | That means your version of SQLite is old enough that it doesn't support the FTS5 extension. Could you share what operating system you're running, and what the output is that you get from running this?
I can teach this tool to fall back on FTS4 if FTS5 isn't available. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fall back to FTS4 if FTS5 is not available 639542974 | |
616029262 | https://github.com/dogsheep/twitter-to-sqlite/issues/45#issuecomment-616029262 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/45 | MDEyOklzc3VlQ29tbWVudDYxNjAyOTI2Mg== | simonw 9599 | 2020-04-19T04:39:21Z | 2020-04-19T04:39:21Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use raise_for_status() everywhere 602619330 | ||
615513491 | https://github.com/dogsheep/twitter-to-sqlite/issues/43#issuecomment-615513491 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/43 | MDEyOklzc3VlQ29tbWVudDYxNTUxMzQ5MQ== | simonw 9599 | 2020-04-17T23:48:28Z | 2020-04-17T23:48:28Z | MEMBER | Released in 0.21. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"twitter-to-sqlite lists" command for retrieving a user's owned lists 602176870 | |
607019151 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607019151 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNzAxOTE1MQ== | simonw 9599 | 2020-04-01T04:11:10Z | 2020-04-01T04:11:10Z | MEMBER | In testing this collects a LOT of data. I'm going to skip tracking favourites_count and statuses_count and just track followers, friends and listed instead. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
607011972 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011972 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNzAxMTk3Mg== | simonw 9599 | 2020-04-01T03:49:02Z | 2020-04-01T03:50:01Z | MEMBER | I want the datetime value to look like
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
607011421 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011421 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNzAxMTQyMQ== | simonw 9599 | 2020-04-01T03:47:37Z | 2020-04-01T03:55:08Z | MEMBER | Actually a single table with a In which case the compound primary key would be |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
606307376 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-606307376 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNjMwNzM3Ng== | simonw 9599 | 2020-03-30T23:35:40Z | 2020-03-30T23:39:15Z | MEMBER | I think five separate tables:
Each with the following structure:
I'm tempted to have a compound primary key here - user, datetime |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
606307019 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-606307019 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNjMwNzAxOQ== | simonw 9599 | 2020-03-30T23:34:27Z | 2020-03-30T23:34:27Z | MEMBER | The count properties available for a user are:
May as well track history for all of them? Should be pretty cheap to store. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
540879620 | https://github.com/dogsheep/twitter-to-sqlite/issues/4#issuecomment-540879620 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/4 | MDEyOklzc3VlQ29tbWVudDU0MDg3OTYyMA== | simonw 9599 | 2019-10-11T02:59:16Z | 2019-10-11T02:59:16Z | MEMBER | Also import ad preferences and all that other junk. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing data from a Twitter Export file 488835586 | |
527682713 | https://github.com/dogsheep/twitter-to-sqlite/issues/4#issuecomment-527682713 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/4 | MDEyOklzc3VlQ29tbWVudDUyNzY4MjcxMw== | simonw 9599 | 2019-09-03T23:48:57Z | 2019-09-03T23:48:57Z | MEMBER | One interesting challenge here is that the JSON format for tweets in the archive is subtly different from the JSON format currently returned by the API. If we want to keep the tweets in the same database table (which feels like the right thing to me) we'll need to handle this. One thing we can do is have a column for We can also ensure that tweets from the API always over-write the version that came from the archive (using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing data from a Twitter Export file 488835586 | |
607010634 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607010634 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNzAxMDYzNA== | simonw 9599 | 2020-04-01T03:45:16Z | 2020-04-01T03:45:16Z | MEMBER | OK, fix is applied to everything now. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
607003655 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607003655 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNzAwMzY1NQ== | simonw 9599 | 2020-04-01T03:18:00Z | 2020-04-01T03:18:00Z | MEMBER | I've got this working for the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606998669 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606998669 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjk5ODY2OQ== | simonw 9599 | 2020-04-01T02:57:36Z | 2020-04-01T02:57:36Z | MEMBER | The tricky thing here is thinking about the interaction between the recorded since_id and a desire to run the initial import. The first time you run We need to record the maximum ID from those as the But what happens if our initial import is cancelled after only a few tweets? We risk never pulling in the rest of the tweets. Not sure if I need to solve this at all or if I should instead trust users to run the command a second time without I had considered letting |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606850453 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850453 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg1MDQ1Mw== | simonw 9599 | 2020-03-31T20:14:58Z | 2020-04-01T03:03:50Z | MEMBER | Actually I'll hard-code the population of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606850008 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850008 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg1MDAwOA== | simonw 9599 | 2020-03-31T20:13:59Z | 2020-04-01T00:23:00Z | MEMBER | Table design for type | key | since_id --- | --- | --- 1 | 124324 | 2347239847293 2 | 99ff9cefff5cbfd804f7cd43e2b27ced8addbe8d | 2125947927344 Primary compound key on
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606844521 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606844521 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg0NDUyMQ== | simonw 9599 | 2020-03-31T20:01:39Z | 2020-03-31T20:01:39Z | MEMBER | I think |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606843224 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606843224 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg0MzIyNA== | simonw 9599 | 2020-03-31T19:59:11Z | 2020-03-31T20:06:32Z | MEMBER | Or... have a single
That last example would use the hash generated here: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606824992 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606824992 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjgyNDk5Mg== | simonw 9599 | 2020-03-31T19:24:23Z | 2020-03-31T19:24:23Z | MEMBER | The
All of them use the same Question: should I have a table for each of those four methods or a single table that is used by them all? I'm leaning towards four separate tables. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606309165 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606309165 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjMwOTE2NQ== | simonw 9599 | 2020-03-30T23:41:31Z | 2020-03-30T23:41:31Z | MEMBER | I like the separate |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606305701 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606305701 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjMwNTcwMQ== | simonw 9599 | 2020-03-30T23:30:27Z | 2020-03-30T23:30:27Z | MEMBER | A better alternative would be to maintain a separate table with the last seen since value for when we ran |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606304837 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606304837 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjMwNDgzNw== | simonw 9599 | 2020-03-30T23:27:50Z | 2020-03-30T23:29:31Z | MEMBER | One option would be something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
601939656 | https://github.com/dogsheep/twitter-to-sqlite/issues/38#issuecomment-601939656 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/38 | MDEyOklzc3VlQ29tbWVudDYwMTkzOTY1Ng== | simonw 9599 | 2020-03-20T22:37:04Z | 2020-03-20T22:37:04Z | MEMBER | New formatting:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Screen name display for user-timeline is uneven 585359363 | |
615510361 | https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615510361 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37 | MDEyOklzc3VlQ29tbWVudDYxNTUxMDM2MQ== | simonw 9599 | 2020-04-17T23:38:27Z | 2020-04-17T23:38:27Z | MEMBER | That's a bit tricky since I'd have to rewrite the internals of a bunch of other commands. For the moment I'll exit the script with an error but at least it will be a decent error! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle "User not found" error 585353598 | |
615509803 | https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615509803 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37 | MDEyOklzc3VlQ29tbWVudDYxNTUwOTgwMw== | simonw 9599 | 2020-04-17T23:36:40Z | 2020-04-17T23:36:40Z | MEMBER | I'm going to print a warning to stderr, skip and continue - because if you have 100 screen names and only one of them is invalid you should still execute for the other 99. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle "User not found" error 585353598 | |
615509578 | https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615509578 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37 | MDEyOklzc3VlQ29tbWVudDYxNTUwOTU3OA== | simonw 9599 | 2020-04-17T23:36:00Z | 2020-04-17T23:36:00Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle "User not found" error 585353598 | |
601892338 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601892338 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg5MjMzOA== | simonw 9599 | 2020-03-20T20:17:00Z | 2020-03-20T20:17:00Z | MEMBER | Released as 0.18. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601890795 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601890795 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg5MDc5NQ== | simonw 9599 | 2020-03-20T20:13:08Z | 2020-03-20T20:13:08Z | MEMBER | Needs documentation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601876051 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601876051 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg3NjA1MQ== | simonw 9599 | 2020-03-20T19:31:43Z | 2020-03-20T19:31:43Z | MEMBER | I'm going to keep |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601875524 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601875524 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg3NTUyNA== | simonw 9599 | 2020-03-20T19:30:27Z | 2020-03-20T19:30:27Z | MEMBER | I can give it a snazzier progress bar to, as requested by #10. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601875405 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601875405 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg3NTQwNQ== | simonw 9599 | 2020-03-20T19:30:09Z | 2020-03-20T19:32:33Z | MEMBER | I'm going to redesign the command a little. Right now it defaults to fetching the authenticated user's timeline and allows I think the design of
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601871579 | https://github.com/dogsheep/twitter-to-sqlite/issues/34#issuecomment-601871579 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34 | MDEyOklzc3VlQ29tbWVudDYwMTg3MTU3OQ== | simonw 9599 | 2020-03-20T19:20:52Z | 2020-03-20T19:20:52Z | MEMBER | Cause was the redefining of how |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
IndexError running user-timeline command 585266763 | |
601861908 | https://github.com/dogsheep/twitter-to-sqlite/issues/34#issuecomment-601861908 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34 | MDEyOklzc3VlQ29tbWVudDYwMTg2MTkwOA== | simonw 9599 | 2020-03-20T18:56:44Z | 2020-03-20T18:56:44Z | MEMBER | Could this be a bug in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
IndexError running user-timeline command 585266763 | |
552135263 | https://github.com/dogsheep/twitter-to-sqlite/issues/31#issuecomment-552135263 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31 | MDEyOklzc3VlQ29tbWVudDU1MjEzNTI2Mw== | simonw 9599 | 2019-11-09T20:38:35Z | 2019-11-09T20:38:35Z | MEMBER | Command still needs documentation and a bit more testing. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"friends" command (similar to "followers") 520508502 | |
552133468 | https://github.com/dogsheep/twitter-to-sqlite/issues/30#issuecomment-552133468 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/30 | MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ2OA== | simonw 9599 | 2019-11-09T20:15:27Z | 2019-11-09T20:15:27Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`followers` fails because `transform_user` is called twice 518739697 | ||
552131798 | https://github.com/dogsheep/twitter-to-sqlite/issues/30#issuecomment-552131798 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/30 | MDEyOklzc3VlQ29tbWVudDU1MjEzMTc5OA== | simonw 9599 | 2019-11-09T19:54:45Z | 2019-11-09T19:54:45Z | MEMBER | Good catch - not sure how that bug crept in. Removing line 116 looks like the right fix to me. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`followers` fails because `transform_user` is called twice 518739697 | |
549228535 | https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549228535 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0OTIyODUzNQ== | simonw 9599 | 2019-11-04T05:31:55Z | 2019-11-04T05:31:55Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for running a search and saving tweets for that search 488833975 | ||
549226399 | https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549226399 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0OTIyNjM5OQ== | simonw 9599 | 2019-11-04T05:11:57Z | 2019-11-04T05:11:57Z | MEMBER | I'm going to add a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for running a search and saving tweets for that search 488833975 | |
549096321 | https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549096321 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0OTA5NjMyMQ== | simonw 9599 | 2019-11-03T01:27:55Z | 2019-11-03T01:28:17Z | MEMBER | It would be neat if this could support |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for running a search and saving tweets for that search 488833975 | |
543290744 | https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-543290744 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0MzI5MDc0NA== | simonw 9599 | 2019-10-17T17:57:14Z | 2019-10-17T17:57:14Z | MEMBER | I have a working command now. I'm going to ship it early because it could do with some other people trying it out. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for running a search and saving tweets for that search 488833975 | |
543273540 | https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-543273540 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0MzI3MzU0MA== | simonw 9599 | 2019-10-17T17:12:51Z | 2019-10-17T17:12:51Z | MEMBER | Just importing tweets here isn't enough - how are we supposed to know which tweets were imported by which search? So I think the right thing to do here is to also create a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for running a search and saving tweets for that search 488833975 | |
541493242 | https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-541493242 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0MTQ5MzI0Mg== | simonw 9599 | 2019-10-14T03:35:36Z | 2019-10-14T03:35:36Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for running a search and saving tweets for that search 488833975 | ||
552133449 | https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552133449 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29 | MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ0OQ== | simonw 9599 | 2019-11-09T20:15:15Z | 2019-11-09T20:15:15Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`import` command fails on empty files 518725064 | ||
552129921 | https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552129921 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29 | MDEyOklzc3VlQ29tbWVudDU1MjEyOTkyMQ== | simonw 9599 | 2019-11-09T19:30:42Z | 2019-11-09T19:30:42Z | MEMBER | Confirmed, that seems to fix it:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`import` command fails on empty files 518725064 | |
552129686 | https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552129686 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29 | MDEyOklzc3VlQ29tbWVudDU1MjEyOTY4Ng== | simonw 9599 | 2019-11-09T19:27:39Z | 2019-11-09T19:27:39Z | MEMBER | I think this is fixed by the latest version of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`import` command fails on empty files 518725064 | |
552133488 | https://github.com/dogsheep/twitter-to-sqlite/issues/28#issuecomment-552133488 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/28 | MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ4OA== | simonw 9599 | 2019-11-09T20:15:42Z | 2019-11-09T20:15:42Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add indexes to followers table 515658861 | ||
549095641 | https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095641 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDU0OTA5NTY0MQ== | simonw 9599 | 2019-11-03T01:12:58Z | 2019-11-03T01:12:58Z | MEMBER | It looks like Twitter really want you to subscribe to a premium API for this kind of thing and consume retweets via webhooks: https://developer.twitter.com/en/docs/accounts-and-users/subscribe-account-activity/api-reference I'm going to give up on this for the moment. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
retweets-of-me command 514459062 | |
549095463 | https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095463 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDU0OTA5NTQ2Mw== | simonw 9599 | 2019-11-03T01:10:52Z | 2019-11-03T01:10:52Z | MEMBER | I imagine it won't, since the data I would be recording and then passing to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
retweets-of-me command 514459062 | |
549095317 | https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095317 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDU0OTA5NTMxNw== | simonw 9599 | 2019-11-03T01:08:10Z | 2019-11-03T01:08:10Z | MEMBER | Hmm... one thing that could be useful is that I'm not sure if the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
retweets-of-me command 514459062 | |
549095217 | https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095217 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDU0OTA5NTIxNw== | simonw 9599 | 2019-11-03T01:06:25Z | 2019-11-03T01:06:25Z | MEMBER | Wow, that It looks like this needs to be combined with this API - https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-retweets-id - to fetch the details of up to 100 recent users who actually DID retweet an individual status. But that has a one-every-12-seconds rate limit on it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
retweets-of-me command 514459062 | |
547713287 | https://github.com/dogsheep/twitter-to-sqlite/issues/26#issuecomment-547713287 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/26 | MDEyOklzc3VlQ29tbWVudDU0NzcxMzI4Nw== | simonw 9599 | 2019-10-30T02:36:13Z | 2019-10-30T02:36:13Z | MEMBER | Shipped this in 0.13: https://github.com/dogsheep/twitter-to-sqlite/releases/tag/0.13 See also this Twitter thread: https://twitter.com/simonw/status/1189369677509623809 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing mentions timeline 513074501 | |
543266947 | https://github.com/dogsheep/twitter-to-sqlite/issues/25#issuecomment-543266947 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/25 | MDEyOklzc3VlQ29tbWVudDU0MzI2Njk0Nw== | simonw 9599 | 2019-10-17T16:56:06Z | 2019-10-17T16:56:06Z | MEMBER | I wrote a test that proves that this is a problem. Should be an easy fix though. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ensure migrations don't accidentally create foreign key twice 508578780 | |
543265058 | https://github.com/dogsheep/twitter-to-sqlite/issues/25#issuecomment-543265058 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/25 | MDEyOklzc3VlQ29tbWVudDU0MzI2NTA1OA== | simonw 9599 | 2019-10-17T16:51:12Z | 2019-10-17T16:51:12Z | MEMBER | This migration function only runs if there is a table called I think this can happen if the database has just been freshly created (by a command that fetches the user's user timeline for example) and is then run a SECOND time. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ensure migrations don't accidentally create foreign key twice 508578780 | |
543222239 | https://github.com/dogsheep/twitter-to-sqlite/issues/23#issuecomment-543222239 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/23 | MDEyOklzc3VlQ29tbWVudDU0MzIyMjIzOQ== | simonw 9599 | 2019-10-17T15:12:33Z | 2019-10-17T15:12:33Z | MEMBER | Migrations will run only if you open a database that previously existed (as opposed to opening a brand new empty database). This means that the first time you run a command against a fresh database, migrations will not run and the This also means that each migration needs to be able to sanity check the database to see if it should run or not. If it should NOT run, it will do nothing but still be marked as having executed by adding to the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extremely simple migration system 508190730 | |
543217890 | https://github.com/dogsheep/twitter-to-sqlite/issues/23#issuecomment-543217890 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/23 | MDEyOklzc3VlQ29tbWVudDU0MzIxNzg5MA== | simonw 9599 | 2019-10-17T15:03:10Z | 2019-10-17T15:03:10Z | MEMBER | Thinking about this further: the concept of migrations may end up being in direct conflict with the I'm going to forge ahead anyway and build this because I think it will be an interesting exploration, but it's very likely this turns out to be a bad idea in the long run! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extremely simple migration system 508190730 | |
542333836 | https://github.com/dogsheep/twitter-to-sqlite/issues/21#issuecomment-542333836 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/21 | MDEyOklzc3VlQ29tbWVudDU0MjMzMzgzNg== | simonw 9599 | 2019-10-15T18:00:48Z | 2019-10-15T18:00:48Z | MEMBER | I'll use |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fix & escapes in tweet text 506432572 | |
544335363 | https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-544335363 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20 | MDEyOklzc3VlQ29tbWVudDU0NDMzNTM2Mw== | simonw 9599 | 2019-10-21T03:32:04Z | 2019-10-21T03:32:04Z | MEMBER | In case anyone is interested, here's an extract from the crontab I'm running these under at the moment:
|
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since support for various commands for refresh-by-cron 506268945 | |
542854749 | https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-542854749 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20 | MDEyOklzc3VlQ29tbWVudDU0Mjg1NDc0OQ== | simonw 9599 | 2019-10-16T19:26:01Z | 2019-10-16T19:26:01Z | MEMBER | I'm not going to do this for "accounts that have followed me" and "new accounts that I have followed" - instead I will recommend running the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since support for various commands for refresh-by-cron 506268945 | |
541388038 | https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-541388038 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20 | MDEyOklzc3VlQ29tbWVudDU0MTM4ODAzOA== | simonw 9599 | 2019-10-13T05:31:58Z | 2019-10-13T05:31:58Z | MEMBER | For favourites a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since support for various commands for refresh-by-cron 506268945 | |
527990908 | https://github.com/dogsheep/twitter-to-sqlite/issues/2#issuecomment-527990908 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDUyNzk5MDkwOA== | simonw 9599 | 2019-09-04T16:57:24Z | 2019-09-04T16:57:24Z | MEMBER | I just tried this using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"twitter-to-sqlite user-timeline" command for pulling tweets by a specific user 488833698 | |
527955302 | https://github.com/dogsheep/twitter-to-sqlite/issues/2#issuecomment-527955302 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDUyNzk1NTMwMg== | simonw 9599 | 2019-09-04T15:32:39Z | 2019-09-04T15:32:39Z | MEMBER | Rate limit is 900 / 15 minutes which is 1 call per second. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"twitter-to-sqlite user-timeline" command for pulling tweets by a specific user 488833698 | |
527954898 | https://github.com/dogsheep/twitter-to-sqlite/issues/2#issuecomment-527954898 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDUyNzk1NDg5OA== | simonw 9599 | 2019-09-04T15:31:46Z | 2019-09-04T15:31:46Z | MEMBER | I'm going to call this |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"twitter-to-sqlite user-timeline" command for pulling tweets by a specific user 488833698 | |
542849963 | https://github.com/dogsheep/twitter-to-sqlite/issues/19#issuecomment-542849963 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19 | MDEyOklzc3VlQ29tbWVudDU0Mjg0OTk2Mw== | simonw 9599 | 2019-10-16T19:13:06Z | 2019-10-16T19:13:06Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
since_id support for home-timeline 506087267 | ||
542832952 | https://github.com/dogsheep/twitter-to-sqlite/issues/19#issuecomment-542832952 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19 | MDEyOklzc3VlQ29tbWVudDU0MjgzMjk1Mg== | simonw 9599 | 2019-10-16T18:30:11Z | 2019-10-16T18:30:11Z | MEMBER | The
The |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
since_id support for home-timeline 506087267 | |
541248629 | https://github.com/dogsheep/twitter-to-sqlite/issues/19#issuecomment-541248629 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19 | MDEyOklzc3VlQ29tbWVudDU0MTI0ODYyOQ== | simonw 9599 | 2019-10-11T22:48:56Z | 2019-10-11T22:48:56Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
since_id support for home-timeline 506087267 | |
541141169 | https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541141169 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDU0MTE0MTE2OQ== | simonw 9599 | 2019-10-11T16:51:29Z | 2019-10-11T16:51:29Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command to import home-timeline 505928530 | ||
541119834 | https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541119834 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDU0MTExOTgzNA== | simonw 9599 | 2019-10-11T15:51:22Z | 2019-10-11T16:51:33Z | MEMBER | In order to support multiple user timelines being saved in the same database, I'm going to import the tweets into the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command to import home-timeline 505928530 | |
541118934 | https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541118934 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDU0MTExODkzNA== | simonw 9599 | 2019-10-11T15:48:54Z | 2019-10-11T15:48:54Z | MEMBER | Rate limit is tight: 15 requests every 15 mins! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command to import home-timeline 505928530 | |
541118773 | https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541118773 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDU0MTExODc3Mw== | simonw 9599 | 2019-10-11T15:48:31Z | 2019-10-11T15:48:31Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command to import home-timeline 505928530 | ||
541112588 | https://github.com/dogsheep/twitter-to-sqlite/issues/17#issuecomment-541112588 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17 | MDEyOklzc3VlQ29tbWVudDU0MTExMjU4OA== | simonw 9599 | 2019-10-11T15:31:30Z | 2019-10-11T15:31:30Z | MEMBER | No need for an option:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
import command should empty all archive-* tables first 505674949 | |
541112108 | https://github.com/dogsheep/twitter-to-sqlite/issues/17#issuecomment-541112108 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17 | MDEyOklzc3VlQ29tbWVudDU0MTExMjEwOA== | simonw 9599 | 2019-10-11T15:30:15Z | 2019-10-11T15:30:15Z | MEMBER | It should delete the tables entirely. That way it will work even if the table schema has changed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
import command should empty all archive-* tables first 505674949 | |
538804815 | https://github.com/dogsheep/twitter-to-sqlite/issues/13#issuecomment-538804815 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDUzODgwNDgxNQ== | simonw 9599 | 2019-10-07T00:33:49Z | 2019-10-07T00:33:49Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
statuses-lookup command 503085013 | ||
542858025 | https://github.com/dogsheep/twitter-to-sqlite/issues/12#issuecomment-542858025 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDU0Mjg1ODAyNQ== | simonw 9599 | 2019-10-16T19:35:31Z | 2019-10-16T19:36:09Z | MEMBER | Maybe this means I need an |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract "source" into a separate lookup table 503053800 | |
542855427 | https://github.com/dogsheep/twitter-to-sqlite/issues/12#issuecomment-542855427 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDU0Mjg1NTQyNw== | simonw 9599 | 2019-10-16T19:27:55Z | 2019-10-16T19:27:55Z | MEMBER | I can do that by keeping |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract "source" into a separate lookup table 503053800 | |
542855081 | https://github.com/dogsheep/twitter-to-sqlite/issues/12#issuecomment-542855081 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDU0Mjg1NTA4MQ== | simonw 9599 | 2019-10-16T19:26:56Z | 2019-10-16T19:26:56Z | MEMBER | This may be the first case where I want to be able to repair existing databases rather than discarding their contents. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract "source" into a separate lookup table 503053800 | |
538711918 | https://github.com/dogsheep/twitter-to-sqlite/issues/11#issuecomment-538711918 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDUzODcxMTkxOA== | simonw 9599 | 2019-10-06T04:54:17Z | 2019-10-06T04:54:17Z | MEMBER | Shipped in 0.6. Here's the documentation: https://github.com/dogsheep/twitter-to-sqlite#capturing-tweets-in-real-time-with-track-and-follow |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commands for recording real-time tweets from the streaming API 503045221 | |
607010791 | https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-607010791 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDYwNzAxMDc5MQ== | simonw 9599 | 2020-04-01T03:45:48Z | 2020-04-01T03:45:48Z | MEMBER | I'm happy with the recent work I did on this. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rethink progress bars for various commands 492297930 | |
543271000 | https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-543271000 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU0MzI3MTAwMA== | simonw 9599 | 2019-10-17T17:05:59Z | 2019-10-17T17:05:59Z | MEMBER | Looks like tqdm already does a TTY check here: https://github.com/tqdm/tqdm/blob/89b73bdc30c099c5b53725806e7edf3a121c9b3a/tqdm/std.py#L889-L890 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rethink progress bars for various commands 492297930 | |
543270714 | https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-543270714 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU0MzI3MDcxNA== | simonw 9599 | 2019-10-17T17:05:16Z | 2019-10-17T17:05:16Z | MEMBER | https://github.com/pallets/click/blob/716a5be90f56ce6cd506bb53d5739d09374b1636/click/_termui_impl.py#L93 is how Click does this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rethink progress bars for various commands 492297930 | |
543269396 | https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-543269396 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU0MzI2OTM5Ng== | simonw 9599 | 2019-10-17T17:02:07Z | 2019-10-17T17:02:07Z | MEMBER | A neat trick that Click does is detecting if an interactive terminal is attached and NOT showing a progress bar if there isn't one. Need to figure out how to do that with tqdm. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rethink progress bars for various commands 492297930 | |
541748580 | https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-541748580 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU0MTc0ODU4MA== | simonw 9599 | 2019-10-14T15:30:44Z | 2019-10-14T15:30:44Z | MEMBER | Had several recommendations for https://github.com/tqdm/tqdm which is what goodreads-to-sqlite uses. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rethink progress bars for various commands 492297930 | |
605382373 | https://github.com/dogsheep/swarm-to-sqlite/pull/6#issuecomment-605382373 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/6 | MDEyOklzc3VlQ29tbWVudDYwNTM4MjM3Mw== | simonw 9599 | 2020-03-28T02:27:32Z | 2020-03-28T02:27:32Z | MEMBER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
don't break if source is missing 543355051 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue 183