issue_comments
525 rows where author_association = "MEMBER" sorted by issue
This data as json, CSV (advanced)
user 1
- simonw 425
id | html_url | issue_url | node_id | user | created_at | updated_at | author_association | body | reactions | issue ▼ | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
552129686 | https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552129686 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29 | MDEyOklzc3VlQ29tbWVudDU1MjEyOTY4Ng== | simonw 9599 | 2019-11-09T19:27:39Z | 2019-11-09T19:27:39Z | MEMBER | I think this is fixed by the latest version of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`import` command fails on empty files 518725064 | |
552129921 | https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552129921 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29 | MDEyOklzc3VlQ29tbWVudDU1MjEyOTkyMQ== | simonw 9599 | 2019-11-09T19:30:42Z | 2019-11-09T19:30:42Z | MEMBER | Confirmed, that seems to fix it:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`import` command fails on empty files 518725064 | |
552133449 | https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552133449 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29 | MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ0OQ== | simonw 9599 | 2019-11-09T20:15:15Z | 2019-11-09T20:15:15Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`import` command fails on empty files 518725064 | ||
552131798 | https://github.com/dogsheep/twitter-to-sqlite/issues/30#issuecomment-552131798 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/30 | MDEyOklzc3VlQ29tbWVudDU1MjEzMTc5OA== | simonw 9599 | 2019-11-09T19:54:45Z | 2019-11-09T19:54:45Z | MEMBER | Good catch - not sure how that bug crept in. Removing line 116 looks like the right fix to me. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`followers` fails because `transform_user` is called twice 518739697 | |
552133468 | https://github.com/dogsheep/twitter-to-sqlite/issues/30#issuecomment-552133468 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/30 | MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ2OA== | simonw 9599 | 2019-11-09T20:15:27Z | 2019-11-09T20:15:27Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`followers` fails because `transform_user` is called twice 518739697 | ||
550783316 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550783316 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDc4MzMxNg== | simonw 9599 | 2019-11-07T05:16:56Z | 2019-11-07T05:34:29Z | MEMBER | It looks like Apple changed the location of these in iOS 13 - they are now in separate |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | |
550806302 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550806302 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDgwNjMwMg== | simonw 9599 | 2019-11-07T05:33:31Z | 2019-11-07T05:33:31Z | MEMBER | The XML now includes references to these new files: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | |
550824838 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550824838 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDgyNDgzOA== | simonw 9599 | 2019-11-07T05:47:07Z | 2019-11-07T05:47:07Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | ||
550828084 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550828084 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDgyODA4NA== | simonw 9599 | 2019-11-07T05:49:24Z | 2019-11-07T05:49:24Z | MEMBER | So the fix there is going to be to detect the new This will be a little tricky because that function will need access to the zip file. It probably won't work at all for the mode where the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | |
552135263 | https://github.com/dogsheep/twitter-to-sqlite/issues/31#issuecomment-552135263 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31 | MDEyOklzc3VlQ29tbWVudDU1MjEzNTI2Mw== | simonw 9599 | 2019-11-09T20:38:35Z | 2019-11-09T20:38:35Z | MEMBER | Command still needs documentation and a bit more testing. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"friends" command (similar to "followers") 520508502 | |
594151327 | https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-594151327 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDU5NDE1MTMyNw== | simonw 9599 | 2020-03-03T20:26:15Z | 2020-03-03T20:32:23Z | MEMBER | Better version (since this also includes JSON array of repository topics):
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add this view for seeing new releases 520756546 | |
594155249 | https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-594155249 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDU5NDE1NTI0OQ== | simonw 9599 | 2020-03-03T20:35:17Z | 2020-03-03T20:35:17Z | MEMBER |
I think that approach can be approved by first checking if the view exists, then dropping it, then recreating it. Could even try to see if the view exists and matches what we were going to set it to and do nothing if that is the case. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add this view for seeing new releases 520756546 | |
622461223 | https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-622461223 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDYyMjQ2MTIyMw== | simonw 9599 | 2020-05-01T16:34:52Z | 2020-05-01T16:34:52Z | MEMBER | Blocked on #37 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add this view for seeing new releases 520756546 | |
622989874 | https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-622989874 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDYyMjk4OTg3NA== | simonw 9599 | 2020-05-02T17:46:14Z | 2020-05-02T17:46:14Z | MEMBER | Without the rowid column facet by topics breaks: https://github-to-sqlite.dogsheep.net/github/recent_releases?_facet=repo&_facet_array=topics&topics__arraycontains=datasette-io |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add this view for seeing new releases 520756546 | |
622998813 | https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-622998813 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDYyMjk5ODgxMw== | simonw 9599 | 2020-05-02T18:58:17Z | 2020-05-02T18:58:17Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add this view for seeing new releases 520756546 | ||
602861730 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602861730 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjg2MTczMA== | simonw 9599 | 2020-03-23T21:19:16Z | 2020-03-23T21:19:42Z | MEMBER | I'm going to do this with a scheduled GitHub Action in this repo. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602862236 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602862236 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjg2MjIzNg== | simonw 9599 | 2020-03-23T21:20:26Z | 2020-03-23T21:20:26Z | MEMBER | I'll run the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602862967 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602862967 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjg2Mjk2Nw== | simonw 9599 | 2020-03-23T21:22:04Z | 2020-03-23T21:22:04Z | MEMBER | Following these instructions: https://simonwillison.net/2020/Jan/21/github-actions-cloud-run/ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602895896 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602895896 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjg5NTg5Ng== | simonw 9599 | 2020-03-23T22:42:25Z | 2020-03-23T22:42:25Z | MEMBER | Urgh this is such a mess! I should have done this on a branch / pull request to avoid polluting my main master history, but never mind. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602916947 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602916947 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjkxNjk0Nw== | simonw 9599 | 2020-03-23T23:38:06Z | 2020-03-23T23:38:06Z | MEMBER | Woohoo! https://github-to-sqlite-j7hipcg4aq-uc.a.run.app/ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602917713 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602917713 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjkxNzcxMw== | simonw 9599 | 2020-03-23T23:40:29Z | 2020-03-23T23:40:29Z | MEMBER | Most recently updated issues across all Dogsheep repos, with faceting: https://github-to-sqlite-j7hipcg4aq-uc.a.run.app/github/issues?_facet=repo&_facet=user&_facet=state&_facet=author_association&_facet=type&_sort_desc=updated_at |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602918689 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602918689 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjkxODY4OQ== | simonw 9599 | 2020-03-23T23:43:39Z | 2020-03-23T23:47:50Z | MEMBER | I pointed https://github-to-sqlite.dogsheep.net/ at it. May take a few minutes for the new certificate to provision though. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602919058 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602919058 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjkxOTA1OA== | simonw 9599 | 2020-03-23T23:44:48Z | 2020-03-23T23:44:48Z | MEMBER | Next step: use a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
602924714 | https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602924714 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13 | MDEyOklzc3VlQ29tbWVudDYwMjkyNDcxNA== | simonw 9599 | 2020-03-24T00:03:25Z | 2020-03-24T00:03:25Z | MEMBER | This is good enough for the 1.0 release. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a live demo Datasette instance 521275281 | |
559883311 | https://github.com/dogsheep/github-to-sqlite/issues/14#issuecomment-559883311 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14 | MDEyOklzc3VlQ29tbWVudDU1OTg4MzMxMQ== | simonw 9599 | 2019-11-29T21:30:37Z | 2019-11-29T21:30:37Z | MEMBER | I should build the command to persist ETags and obey their polling guidelines:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing events 530491074 | |
559902818 | https://github.com/dogsheep/github-to-sqlite/issues/14#issuecomment-559902818 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14 | MDEyOklzc3VlQ29tbWVudDU1OTkwMjgxOA== | simonw 9599 | 2019-11-30T01:32:38Z | 2019-11-30T01:32:38Z | MEMBER | Prototype:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing events 530491074 | |
613641947 | https://github.com/dogsheep/github-to-sqlite/issues/14#issuecomment-613641947 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14 | MDEyOklzc3VlQ29tbWVudDYxMzY0MTk0Nw== | simonw 9599 | 2020-04-14T19:38:24Z | 2020-04-14T19:38:34Z | MEMBER | Since events include payloads with full object representations in them (for issues, repos and more) running this command every few minutes may be all it takes to keep a constant copy of everything updated in a very rate-limit friendly manner (thanks to the ETags). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing events 530491074 | |
605382373 | https://github.com/dogsheep/swarm-to-sqlite/pull/6#issuecomment-605382373 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/6 | MDEyOklzc3VlQ29tbWVudDYwNTM4MjM3Mw== | simonw 9599 | 2020-03-28T02:27:32Z | 2020-03-28T02:27:32Z | MEMBER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
don't break if source is missing 543355051 | |
602309825 | https://github.com/dogsheep/github-to-sqlite/issues/15#issuecomment-602309825 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15 | MDEyOklzc3VlQ29tbWVudDYwMjMwOTgyNQ== | simonw 9599 | 2020-03-23T00:52:38Z | 2020-03-23T00:52:38Z | MEMBER | Splitting assets out into a separate table totally makes sense to me. They can still be fetched as part of the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Assets table with downloads 544571092 | |
602311701 | https://github.com/dogsheep/github-to-sqlite/issues/15#issuecomment-602311701 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15 | MDEyOklzc3VlQ29tbWVudDYwMjMxMTcwMQ== | simonw 9599 | 2020-03-23T01:05:15Z | 2020-03-23T01:06:14Z | MEMBER | None of my own releases use assets (they are all pushed to PyPI instead) but I spotted that your project here uses assets, so I'll test against that: https://github.com/instrumenta/conftest/releases/tag/v0.18.0
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Assets table with downloads 544571092 | |
602312054 | https://github.com/dogsheep/github-to-sqlite/issues/15#issuecomment-602312054 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15 | MDEyOklzc3VlQ29tbWVudDYwMjMxMjA1NA== | simonw 9599 | 2020-03-23T01:07:10Z | 2020-03-23T01:07:10Z | MEMBER | Each asset looks like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Assets table with downloads 544571092 | |
601979709 | https://github.com/dogsheep/github-to-sqlite/issues/16#issuecomment-601979709 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16 | MDEyOklzc3VlQ29tbWVudDYwMTk3OTcwOQ== | simonw 9599 | 2020-03-21T02:20:08Z | 2020-03-21T02:20:08Z | MEMBER | I'm pretty sure this means I need to upgrade the underlying |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exception running first command: IndexError: list index out of range 546051181 | |
613611455 | https://github.com/dogsheep/github-to-sqlite/issues/16#issuecomment-613611455 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16 | MDEyOklzc3VlQ29tbWVudDYxMzYxMTQ1NQ== | simonw 9599 | 2020-04-14T18:37:21Z | 2020-04-14T18:37:21Z | MEMBER | This should have been fixed by #20 and #23 @jayvdb I'm definitely interested in this tool working as a library - it's purely designed as a CLI tool at the moment, but cleaning it up to work better as a dependency is totally in-scope for the project. https://sqlite-utils.readthedocs.io/ is an example of a tool I've built that works for both. Feel free to open a new issue here with some notes on what you would need for this to work as a library for your project! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exception running first command: IndexError: list index out of range 546051181 | |
601871930 | https://github.com/dogsheep/twitter-to-sqlite/pull/33#issuecomment-601871930 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/33 | MDEyOklzc3VlQ29tbWVudDYwMTg3MTkzMA== | simonw 9599 | 2020-03-20T19:21:41Z | 2020-03-20T19:21:41Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to sqlite-utils 2.2.1 561469252 | ||
597354514 | https://github.com/dogsheep/github-to-sqlite/issues/17#issuecomment-597354514 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/17 | MDEyOklzc3VlQ29tbWVudDU5NzM1NDUxNA== | simonw 9599 | 2020-03-10T22:37:45Z | 2020-03-10T22:37:45Z | MEMBER | I should add an option to stop the moment you see a commit you have fetched before. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing commits 578883725 | |
597358364 | https://github.com/dogsheep/github-to-sqlite/issues/17#issuecomment-597358364 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/17 | MDEyOklzc3VlQ29tbWVudDU5NzM1ODM2NA== | simonw 9599 | 2020-03-10T22:50:20Z | 2020-03-11T01:18:36Z | MEMBER | By default it will stop when it sees a commit that has already been stored. You will be able to over-ride that behaviour using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for importing commits 578883725 | |
601861908 | https://github.com/dogsheep/twitter-to-sqlite/issues/34#issuecomment-601861908 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34 | MDEyOklzc3VlQ29tbWVudDYwMTg2MTkwOA== | simonw 9599 | 2020-03-20T18:56:44Z | 2020-03-20T18:56:44Z | MEMBER | Could this be a bug in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
IndexError running user-timeline command 585266763 | |
601871579 | https://github.com/dogsheep/twitter-to-sqlite/issues/34#issuecomment-601871579 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34 | MDEyOklzc3VlQ29tbWVudDYwMTg3MTU3OQ== | simonw 9599 | 2020-03-20T19:20:52Z | 2020-03-20T19:20:52Z | MEMBER | Cause was the redefining of how |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
IndexError running user-timeline command 585266763 | |
601875405 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601875405 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg3NTQwNQ== | simonw 9599 | 2020-03-20T19:30:09Z | 2020-03-20T19:32:33Z | MEMBER | I'm going to redesign the command a little. Right now it defaults to fetching the authenticated user's timeline and allows I think the design of
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601875524 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601875524 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg3NTUyNA== | simonw 9599 | 2020-03-20T19:30:27Z | 2020-03-20T19:30:27Z | MEMBER | I can give it a snazzier progress bar to, as requested by #10. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601876051 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601876051 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg3NjA1MQ== | simonw 9599 | 2020-03-20T19:31:43Z | 2020-03-20T19:31:43Z | MEMBER | I'm going to keep |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601890795 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601890795 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg5MDc5NQ== | simonw 9599 | 2020-03-20T20:13:08Z | 2020-03-20T20:13:08Z | MEMBER | Needs documentation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
601892338 | https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601892338 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35 | MDEyOklzc3VlQ29tbWVudDYwMTg5MjMzOA== | simonw 9599 | 2020-03-20T20:17:00Z | 2020-03-20T20:17:00Z | MEMBER | Released as 0.18. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
twitter-to-sqlite user-timeline [screen_names] --sql / --attach 585282212 | |
615509578 | https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615509578 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37 | MDEyOklzc3VlQ29tbWVudDYxNTUwOTU3OA== | simonw 9599 | 2020-04-17T23:36:00Z | 2020-04-17T23:36:00Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle "User not found" error 585353598 | |
615509803 | https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615509803 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37 | MDEyOklzc3VlQ29tbWVudDYxNTUwOTgwMw== | simonw 9599 | 2020-04-17T23:36:40Z | 2020-04-17T23:36:40Z | MEMBER | I'm going to print a warning to stderr, skip and continue - because if you have 100 screen names and only one of them is invalid you should still execute for the other 99. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle "User not found" error 585353598 | |
615510361 | https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615510361 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37 | MDEyOklzc3VlQ29tbWVudDYxNTUxMDM2MQ== | simonw 9599 | 2020-04-17T23:38:27Z | 2020-04-17T23:38:27Z | MEMBER | That's a bit tricky since I'd have to rewrite the internals of a bunch of other commands. For the moment I'll exit the script with an error but at least it will be a decent error! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle "User not found" error 585353598 | |
601939656 | https://github.com/dogsheep/twitter-to-sqlite/issues/38#issuecomment-601939656 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/38 | MDEyOklzc3VlQ29tbWVudDYwMTkzOTY1Ng== | simonw 9599 | 2020-03-20T22:37:04Z | 2020-03-20T22:37:04Z | MEMBER | New formatting:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Screen name display for user-timeline is uneven 585359363 | |
601980724 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-601980724 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMTk4MDcyNA== | simonw 9599 | 2020-03-21T02:28:05Z | 2020-03-21T02:28:05Z | MEMBER | This suggests that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | |
601980957 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-601980957 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMTk4MDk1Nw== | simonw 9599 | 2020-03-21T02:30:03Z | 2020-03-21T02:30:03Z | MEMBER | https://github.community/t5/GitHub-API-Development-and/Request-for-commits-quot-author-quot-null-and-quot-committer/m-p/35842/highlight/true#M3372
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | |
602807178 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602807178 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMjgwNzE3OA== | simonw 9599 | 2020-03-23T19:24:43Z | 2020-03-23T19:24:43Z | MEMBER | I need to find an example before I work on this. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | |
602811468 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602811468 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMjgxMTQ2OA== | simonw 9599 | 2020-03-23T19:33:14Z | 2020-03-23T19:33:14Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | ||
602811785 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602811785 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMjgxMTc4NQ== | simonw 9599 | 2020-03-23T19:33:56Z | 2020-03-23T19:34:06Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | |
602813731 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602813731 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMjgxMzczMQ== | simonw 9599 | 2020-03-23T19:38:03Z | 2020-03-23T20:44:49Z | MEMBER | So it turns out Maybe I should be storing the nested data somewhere as well? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | |
602815120 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602815120 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMjgxNTEyMA== | simonw 9599 | 2020-03-23T19:40:55Z | 2020-03-23T19:43:19Z | MEMBER | I could pull a pk-hashed version of the name/email into separate |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | |
602846293 | https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602846293 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18 | MDEyOklzc3VlQ29tbWVudDYwMjg0NjI5Mw== | simonw 9599 | 2020-03-23T20:44:40Z | 2020-03-23T20:44:40Z | MEMBER | I implemented the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Commits in GitHub API can have null author 585411547 | |
602094386 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/1#issuecomment-602094386 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/1 | MDEyOklzc3VlQ29tbWVudDYwMjA5NDM4Ng== | simonw 9599 | 2020-03-21T19:47:46Z | 2020-03-21T19:47:46Z | MEMBER | Released as 0.3. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up full text search 585526292 | |
602304531 | https://github.com/dogsheep/github-to-sqlite/issues/19#issuecomment-602304531 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/19 | MDEyOklzc3VlQ29tbWVudDYwMjMwNDUzMQ== | simonw 9599 | 2020-03-23T00:20:28Z | 2020-03-23T00:20:28Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Enable full-text search for more stuff (like commits, issues and issue_comments) 585850715 | ||
602782437 | https://github.com/dogsheep/github-to-sqlite/issues/19#issuecomment-602782437 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/19 | MDEyOklzc3VlQ29tbWVudDYwMjc4MjQzNw== | simonw 9599 | 2020-03-23T18:36:12Z | 2020-03-23T18:51:23Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Enable full-text search for more stuff (like commits, issues and issue_comments) 585850715 | |
602896434 | https://github.com/dogsheep/github-to-sqlite/issues/21#issuecomment-602896434 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/21 | MDEyOklzc3VlQ29tbWVudDYwMjg5NjQzNA== | simonw 9599 | 2020-03-23T22:43:37Z | 2020-03-23T22:43:37Z | MEMBER | I'm going to do this now to help figure out the latest error in #13:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Turn GitHub API errors into exceptions 586561727 | |
602920163 | https://github.com/dogsheep/github-to-sqlite/issues/21#issuecomment-602920163 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/21 | MDEyOklzc3VlQ29tbWVudDYwMjkyMDE2Mw== | simonw 9599 | 2020-03-23T23:48:22Z | 2020-03-23T23:48:22Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Turn GitHub API errors into exceptions 586561727 | ||
602928533 | https://github.com/dogsheep/github-to-sqlite/issues/23#issuecomment-602928533 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/23 | MDEyOklzc3VlQ29tbWVudDYwMjkyODUzMw== | simonw 9599 | 2020-03-24T00:15:49Z | 2020-03-24T00:15:49Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Release 1.0 586595839 | ||
606304837 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606304837 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjMwNDgzNw== | simonw 9599 | 2020-03-30T23:27:50Z | 2020-03-30T23:29:31Z | MEMBER | One option would be something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606305701 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606305701 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjMwNTcwMQ== | simonw 9599 | 2020-03-30T23:30:27Z | 2020-03-30T23:30:27Z | MEMBER | A better alternative would be to maintain a separate table with the last seen since value for when we ran |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606309165 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606309165 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjMwOTE2NQ== | simonw 9599 | 2020-03-30T23:41:31Z | 2020-03-30T23:41:31Z | MEMBER | I like the separate |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606824992 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606824992 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjgyNDk5Mg== | simonw 9599 | 2020-03-31T19:24:23Z | 2020-03-31T19:24:23Z | MEMBER | The
All of them use the same Question: should I have a table for each of those four methods or a single table that is used by them all? I'm leaning towards four separate tables. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606843224 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606843224 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg0MzIyNA== | simonw 9599 | 2020-03-31T19:59:11Z | 2020-03-31T20:06:32Z | MEMBER | Or... have a single
That last example would use the hash generated here: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606844521 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606844521 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg0NDUyMQ== | simonw 9599 | 2020-03-31T20:01:39Z | 2020-03-31T20:01:39Z | MEMBER | I think |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606850008 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850008 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg1MDAwOA== | simonw 9599 | 2020-03-31T20:13:59Z | 2020-04-01T00:23:00Z | MEMBER | Table design for type | key | since_id --- | --- | --- 1 | 124324 | 2347239847293 2 | 99ff9cefff5cbfd804f7cd43e2b27ced8addbe8d | 2125947927344 Primary compound key on
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606850453 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850453 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjg1MDQ1Mw== | simonw 9599 | 2020-03-31T20:14:58Z | 2020-04-01T03:03:50Z | MEMBER | Actually I'll hard-code the population of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606998669 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606998669 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNjk5ODY2OQ== | simonw 9599 | 2020-04-01T02:57:36Z | 2020-04-01T02:57:36Z | MEMBER | The tricky thing here is thinking about the interaction between the recorded since_id and a desire to run the initial import. The first time you run We need to record the maximum ID from those as the But what happens if our initial import is cancelled after only a few tweets? We risk never pulling in the rest of the tweets. Not sure if I need to solve this at all or if I should instead trust users to run the command a second time without I had considered letting |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
607003655 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607003655 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNzAwMzY1NQ== | simonw 9599 | 2020-04-01T03:18:00Z | 2020-04-01T03:18:00Z | MEMBER | I've got this working for the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
607010634 | https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607010634 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39 | MDEyOklzc3VlQ29tbWVudDYwNzAxMDYzNA== | simonw 9599 | 2020-04-01T03:45:16Z | 2020-04-01T03:45:16Z | MEMBER | OK, fix is applied to everything now. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--since feature can be confused by retweets 590666760 | |
606307019 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-606307019 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNjMwNzAxOQ== | simonw 9599 | 2020-03-30T23:34:27Z | 2020-03-30T23:34:27Z | MEMBER | The count properties available for a user are:
May as well track history for all of them? Should be pretty cheap to store. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
606307376 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-606307376 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNjMwNzM3Ng== | simonw 9599 | 2020-03-30T23:35:40Z | 2020-03-30T23:39:15Z | MEMBER | I think five separate tables:
Each with the following structure:
I'm tempted to have a compound primary key here - user, datetime |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
607011421 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011421 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNzAxMTQyMQ== | simonw 9599 | 2020-04-01T03:47:37Z | 2020-04-01T03:55:08Z | MEMBER | Actually a single table with a In which case the compound primary key would be |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
607011972 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011972 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNzAxMTk3Mg== | simonw 9599 | 2020-04-01T03:49:02Z | 2020-04-01T03:50:01Z | MEMBER | I want the datetime value to look like
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
607019151 | https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607019151 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40 | MDEyOklzc3VlQ29tbWVudDYwNzAxOTE1MQ== | simonw 9599 | 2020-04-01T04:11:10Z | 2020-04-01T04:11:10Z | MEMBER | In testing this collects a LOT of data. I'm going to skip tracking favourites_count and statuses_count and just track followers, friends and listed instead. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature: record history of follower counts 590669793 | |
614810417 | https://github.com/dogsheep/github-to-sqlite/issues/25#issuecomment-614810417 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/25 | MDEyOklzc3VlQ29tbWVudDYxNDgxMDQxNw== | simonw 9599 | 2020-04-16T18:07:11Z | 2020-04-16T18:07:11Z | MEMBER | Turns out the main problem was #26 - now fixed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Improvements to demo instance 601265023 | |
614794739 | https://github.com/dogsheep/github-to-sqlite/issues/26#issuecomment-614794739 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26 | MDEyOklzc3VlQ29tbWVudDYxNDc5NDczOQ== | simonw 9599 | 2020-04-16T17:38:28Z | 2020-04-16T17:38:28Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Topics are missing from repositories 601271612 | ||
614795712 | https://github.com/dogsheep/github-to-sqlite/issues/26#issuecomment-614795712 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26 | MDEyOklzc3VlQ29tbWVudDYxNDc5NTcxMg== | simonw 9599 | 2020-04-16T17:40:27Z | 2020-04-16T17:40:27Z | MEMBER | Aha! it was missing from the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Topics are missing from repositories 601271612 | |
614831451 | https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831451 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDYxNDgzMTQ1MQ== | simonw 9599 | 2020-04-16T18:47:25Z | 2020-04-16T18:47:25Z | MEMBER | Is it possible for a repo to have an |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Repos have a big blob of JSON in the organization column 601330277 | |
614831842 | https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831842 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDYxNDgzMTg0Mg== | simonw 9599 | 2020-04-16T18:48:18Z | 2020-04-16T18:48:18Z | MEMBER | I'm going to make |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Repos have a big blob of JSON in the organization column 601330277 | |
614843406 | https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614843406 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDYxNDg0MzQwNg== | simonw 9599 | 2020-04-16T19:11:53Z | 2020-04-16T19:20:23Z | MEMBER | This didn't quite work: the column type is incorrect, so the foreign key relationship isn't sticking: https://github-to-sqlite.dogsheep.net/github/repos?organization=53015001
The problem is that if the first repo inserted has no organization it's set to One solution would be to create the column explicitly with a type, but this could get messy. I think I want a new sqlite-utils feature for this instead. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Repos have a big blob of JSON in the organization column 601330277 | |
615518606 | https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-615518606 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDYxNTUxODYwNg== | simonw 9599 | 2020-04-18T00:14:32Z | 2020-04-18T00:14:32Z | MEMBER | https://github.com/simonw/sqlite-utils/issues/100 is done and released in sqlite-utils 2.7. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Repos have a big blob of JSON in the organization column 601330277 | |
615519409 | https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-615519409 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27 | MDEyOklzc3VlQ29tbWVudDYxNTUxOTQwOQ== | simonw 9599 | 2020-04-18T00:19:16Z | 2020-04-18T00:19:16Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Repos have a big blob of JSON in the organization column 601330277 | |
615883040 | https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615883040 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28 | MDEyOklzc3VlQ29tbWVudDYxNTg4MzA0MA== | simonw 9599 | 2020-04-18T14:45:38Z | 2020-04-18T14:45:38Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Pull repository contributors 601333634 | |
615883687 | https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615883687 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28 | MDEyOklzc3VlQ29tbWVudDYxNTg4MzY4Nw== | simonw 9599 | 2020-04-18T14:49:58Z | 2020-04-18T14:49:58Z | MEMBER | That happened trying to pull contributors for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Pull repository contributors 601333634 | |
615886206 | https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615886206 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28 | MDEyOklzc3VlQ29tbWVudDYxNTg4NjIwNg== | simonw 9599 | 2020-04-18T15:04:59Z | 2020-04-18T15:04:59Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Pull repository contributors 601333634 | ||
615513491 | https://github.com/dogsheep/twitter-to-sqlite/issues/43#issuecomment-615513491 | https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/43 | MDEyOklzc3VlQ29tbWVudDYxNTUxMzQ5MQ== | simonw 9599 | 2020-04-17T23:48:28Z | 2020-04-17T23:48:28Z | MEMBER | Released in 0.21. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"twitter-to-sqlite lists" command for retrieving a user's owned lists 602176870 | |
623193947 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623193947 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzE5Mzk0Nw== | simonw 9599 | 2020-05-03T22:36:17Z | 2020-05-03T22:36:17Z | MEMBER | I'm going to use osxphotos for this. Since I've already got code to upload photos and insert them into a table based on their |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
623195197 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623195197 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzE5NTE5Nw== | simonw 9599 | 2020-05-03T22:44:33Z | 2020-05-03T22:44:33Z | MEMBER | Command will be this:
This will populate a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
623198653 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623198653 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzE5ODY1Mw== | simonw 9599 | 2020-05-03T23:09:57Z | 2020-05-03T23:09:57Z | MEMBER | For locations: I'll add |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
623198986 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623198986 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzE5ODk4Ng== | simonw 9599 | 2020-05-03T23:12:31Z | 2020-05-03T23:12:46Z | MEMBER | To get the taken date in UTC: ``` from datetime import timezone (Pdb) photo.date.astimezone(timezone.utc).isoformat() '2018-02-13T20:21:31.620000+00:00' (Pdb) photo.date.astimezone(timezone.utc).isoformat().split(".") ['2018-02-13T20:21:31', '620000+00:00'] (Pdb) photo.date.astimezone(timezone.utc).isoformat().split(".")[0] '2018-02-13T20:21:31' (Pdb) photo.date.astimezone(timezone.utc).isoformat().split(".")[0] + "+00:00" '2018-02-13T20:21:31+00:00' ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
623199214 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623199214 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzE5OTIxNA== | simonw 9599 | 2020-05-03T23:14:08Z | 2020-05-03T23:14:08Z | MEMBER | Albums have UUIDs:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
623199701 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623199701 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzE5OTcwMQ== | simonw 9599 | 2020-05-03T23:17:38Z | 2020-05-03T23:17:38Z | MEMBER | Record burst_uuid as a column:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
623199750 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623199750 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzE5OTc1MA== | simonw 9599 | 2020-05-03T23:17:58Z | 2020-05-03T23:17:58Z | MEMBER | Reading this source code is really useful for figuring out how to store a photo in a DB table: https://github.com/RhetTbull/osxphotos/blob/7444b6d173918a3ad2a07aefce5ecf054786c787/osxphotos/photoinfo.py |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
623232984 | https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623232984 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1 | MDEyOklzc3VlQ29tbWVudDYyMzIzMjk4NA== | simonw 9599 | 2020-05-04T02:41:32Z | 2020-05-04T02:41:32Z | MEMBER | Needs documentation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import photo metadata from Apple Photos into SQLite 602533300 | |
615931488 | https://github.com/dogsheep/dogsheep-photos/issues/2#issuecomment-615931488 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2 | MDEyOklzc3VlQ29tbWVudDYxNTkzMTQ4OA== | simonw 9599 | 2020-04-18T19:24:02Z | 2020-04-18T19:24:02Z | MEMBER | I made a start on this last week with a https://github.com/simonw/heic-to-jpeg proxy. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to convert HEIC images to JPEG 602533352 | |
615932007 | https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932007 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | MDEyOklzc3VlQ29tbWVudDYxNTkzMjAwNw== | simonw 9599 | 2020-04-18T19:27:55Z | 2020-04-18T19:27:55Z | MEMBER | Research thread: https://twitter.com/simonw/status/1249049694984011776
https://testdriven.io/blog/storing-django-static-and-media-files-on-amazon-s3/ looks useful |
{ "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upload all my photos to a secure S3 bucket 602533539 | |
615932204 | https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932204 | https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4 | MDEyOklzc3VlQ29tbWVudDYxNTkzMjIwNA== | simonw 9599 | 2020-04-18T19:29:22Z | 2020-04-18T19:34:44Z | MEMBER | I'm going to call my bucket |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upload all my photos to a secure S3 bucket 602533539 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue >30