issue_comments
525 rows where author_association = "MEMBER" sorted by issue_url descending
This data as json, CSV (advanced)
user 1
- simonw 425
id | html_url | issue_url ▲ | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
707332912 | https://github.com/dogsheep/swarm-to-sqlite/issues/8#issuecomment-707332912 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDcwNzMzMjkxMg== | simonw 9599 | 2020-10-12T20:35:06Z | 2020-10-12T20:35:06Z | MEMBER | Shipped a fix for this in swarm-to-sqlite 0.3.2. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error thrown: table photos has no column named hasSticker 648245071 | |
605382373 | https://github.com/dogsheep/swarm-to-sqlite/pull/6#issuecomment-605382373 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/6 | MDEyOklzc3VlQ29tbWVudDYwNTM4MjM3Mw== | simonw 9599 | 2020-03-28T02:27:32Z | 2020-03-28T02:27:32Z | MEMBER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
don't break if source is missing 543355051 | |
526853542 | https://github.com/dogsheep/swarm-to-sqlite/issues/4#issuecomment-526853542 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/4 | MDEyOklzc3VlQ29tbWVudDUyNjg1MzU0Mg== | simonw 9599 | 2019-08-31T18:06:32Z | 2019-08-31T18:06:32Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Online tool for getting a Foursquare OAuth token 487601121 | ||
527200332 | https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-527200332 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDUyNzIwMDMzMg== | simonw 9599 | 2019-09-02T16:32:20Z | 2019-09-02T16:32:39Z | MEMBER | Also needed: an option for "fetch all checkins created within the last X days". This should help provide support for that Swarm feature where you can retroactively checkin to places in the past. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to fetch only checkins more recent than the current max checkin 487600595 | |
542875885 | https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-542875885 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0Mjg3NTg4NQ== | simonw 9599 | 2019-10-16T20:23:08Z | 2019-10-16T20:23:08Z | MEMBER | https://developer.foursquare.com/docs/api/users/checkins documents
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to fetch only checkins more recent than the current max checkin 487600595 | |
542876047 | https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-542876047 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0Mjg3NjA0Nw== | simonw 9599 | 2019-10-16T20:23:36Z | 2019-10-16T20:23:36Z | MEMBER | I'm going to go with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to fetch only checkins more recent than the current max checkin 487600595 | |
542882604 | https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-542882604 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDU0Mjg4MjYwNA== | simonw 9599 | 2019-10-16T20:41:23Z | 2019-10-16T20:41:23Z | MEMBER | Documented here: https://github.com/dogsheep/swarm-to-sqlite/blob/0.2/README.md#usage |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option to fetch only checkins more recent than the current max checkin 487600595 | |
526701674 | https://github.com/dogsheep/swarm-to-sqlite/issues/2#issuecomment-526701674 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDUyNjcwMTY3NA== | simonw 9599 | 2019-08-30T18:24:26Z | 2019-08-30T18:24:26Z | MEMBER | I renamed |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
--save option to dump checkins to a JSON file on disk 487598468 | |
1502543165 | https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502543165 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13 | IC_kwDODD6af85Zjv09 | simonw 9599 | 2023-04-11T01:10:36Z | 2023-04-11T01:11:47Z | MEMBER | I just had that error myself on macOS while running the tests:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
fails before generating views. ERR: table sqlite_master may not be modified 1373210675 | |
1502546045 | https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502546045 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13 | IC_kwDODD6af85Zjwh9 | simonw 9599 | 2023-04-11T01:14:50Z | 2023-04-11T01:14:50Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
fails before generating views. ERR: table sqlite_master may not be modified 1373210675 | ||
1502629219 | https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629219 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13 | IC_kwDODD6af85ZkE1j | simonw 9599 | 2023-04-11T03:15:26Z | 2023-04-11T03:15:26Z | MEMBER | OK, I figured this out. Unfortunately it's an error that occurs on Python versions that have defensive mode turned on, and it doesn't look like there's a way to turn that mode off. See notes above. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
fails before generating views. ERR: table sqlite_master may not be modified 1373210675 | |
1502629404 | https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629404 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13 | IC_kwDODD6af85ZkE4c | simonw 9599 | 2023-04-11T03:15:47Z | 2023-04-11T03:46:17Z | MEMBER | I think |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
fails before generating views. ERR: table sqlite_master may not be modified 1373210675 | |
727692413 | https://github.com/dogsheep/swarm-to-sqlite/issues/11#issuecomment-727692413 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDcyNzY5MjQxMw== | simonw 9599 | 2020-11-16T02:15:22Z | 2020-11-16T02:15:22Z | MEMBER | Thanks, I'll look into this. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error thrown: sqlite3.OperationalError: table users has no column named lastName 743400216 | |
761967094 | https://github.com/dogsheep/swarm-to-sqlite/issues/11#issuecomment-761967094 | https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDc2MTk2NzA5NA== | simonw 9599 | 2021-01-18T04:11:13Z | 2021-01-18T04:11:13Z | MEMBER | I just got a similar error: ``` File "/home/dogsheep/datasette-venv/lib/python3.8/site-packages/swarm_to_sqlite/utils.py", line 79, in save_checkin checkins_table.m2m("users", user, m2m_table="with", pk="id") File "/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py", line 2048, in m2m id = other_table.insert(record, pk=pk, replace=True).last_pk File "/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py", line 1781, in insert return self.insert_all( File "/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py", line 1899, in insert_all self.insert_chunk( File "/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py", line 1709, in insert_chunk result = self.db.execute(query, params) File "/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py", line 226, in execute return self.conn.execute(sql, parameters) pysqlite3.dbapi2.OperationalError: table users has no column named countryCode ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Error thrown: sqlite3.OperationalError: table users has no column named lastName 743400216 | |
1221621700 | https://github.com/dogsheep/pocket-to-sqlite/pull/7#issuecomment-1221621700 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/7 | IC_kwDODLZ_YM5I0HfE | simonw 9599 | 2022-08-21T21:11:12Z | 2022-08-21T21:11:12Z | MEMBER | I thought this might need a documentation update but |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fixed conflicting CLI flags 750141615 | |
684425714 | https://github.com/dogsheep/pocket-to-sqlite/issues/5#issuecomment-684425714 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDY4NDQyNTcxNA== | simonw 9599 | 2020-09-01T06:18:32Z | 2020-09-01T06:18:32Z | MEMBER | Good suggestion, I'll setup a demo somewhere. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up a demo 629473827 | |
684424396 | https://github.com/dogsheep/pocket-to-sqlite/issues/3#issuecomment-684424396 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3 | MDEyOklzc3VlQ29tbWVudDY4NDQyNDM5Ng== | simonw 9599 | 2020-09-01T06:17:45Z | 2020-09-01T06:17:45Z | MEMBER | It looks like I could ignore the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract images into separate tables 503243784 | |
538847796 | https://github.com/dogsheep/pocket-to-sqlite/issues/2#issuecomment-538847796 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDUzODg0Nzc5Ng== | simonw 9599 | 2019-10-07T05:43:30Z | 2019-10-07T05:43:30Z | MEMBER | We can persist the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Track and use the 'since' value 503234169 | |
605337941 | https://github.com/dogsheep/pocket-to-sqlite/issues/2#issuecomment-605337941 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDYwNTMzNzk0MQ== | simonw 9599 | 2020-03-27T22:16:32Z | 2020-03-27T22:16:32Z | MEMBER | Need to test this. I have 7,394 items in my database right now. I'm going to save a new thing. Then I ran this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Track and use the 'since' value 503234169 | |
605338322 | https://github.com/dogsheep/pocket-to-sqlite/issues/2#issuecomment-605338322 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDYwNTMzODMyMg== | simonw 9599 | 2020-03-27T22:18:02Z | 2020-03-27T22:18:02Z | MEMBER | Just needs documentation now. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Track and use the 'since' value 503234169 | |
1627563202 | https://github.com/dogsheep/pocket-to-sqlite/issues/12#issuecomment-1627563202 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12 | IC_kwDODLZ_YM5hAqTC | simonw 9599 | 2023-07-09T01:14:27Z | 2023-07-09T01:14:27Z | MEMBER | I tested this locally with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch to pyproject.toml 1795187493 | |
1627564127 | https://github.com/dogsheep/pocket-to-sqlite/issues/12#issuecomment-1627564127 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12 | IC_kwDODLZ_YM5hAqhf | simonw 9599 | 2023-07-09T01:19:42Z | 2023-07-09T01:19:42Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch to pyproject.toml 1795187493 | ||
1221621466 | https://github.com/dogsheep/pocket-to-sqlite/issues/11#issuecomment-1221621466 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11 | IC_kwDODLZ_YM5I0Hba | simonw 9599 | 2022-08-21T21:09:47Z | 2022-08-21T21:09:47Z | MEMBER | Great catch, thanks. I'm going to use it to mean
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
-a option is used for "--auth" and for "--all" 1345452427 | |
1221621529 | https://github.com/dogsheep/pocket-to-sqlite/issues/11#issuecomment-1221621529 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11 | IC_kwDODLZ_YM5I0HcZ | simonw 9599 | 2022-08-21T21:10:15Z | 2022-08-21T21:11:26Z | MEMBER | Just saw that's what's implemented here already! - #7 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
-a option is used for "--auth" and for "--all" 1345452427 | |
1221622873 | https://github.com/dogsheep/pocket-to-sqlite/issues/10#issuecomment-1221622873 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/10 | IC_kwDODLZ_YM5I0HxZ | simonw 9599 | 2022-08-21T21:19:25Z | 2022-08-21T21:19:25Z | MEMBER | Agreed, that would be a much better implementation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
When running `auth` command, don't overwrite an existing auth.json file 1246826792 | |
1221623052 | https://github.com/dogsheep/pocket-to-sqlite/issues/10#issuecomment-1221623052 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/10 | IC_kwDODLZ_YM5I0H0M | simonw 9599 | 2022-08-21T21:20:33Z | 2022-08-21T21:20:33Z | MEMBER | That was clearly the intention from the description of this issue: - #4 |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
When running `auth` command, don't overwrite an existing auth.json file 1246826792 | |
538847446 | https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-538847446 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1 | MDEyOklzc3VlQ29tbWVudDUzODg0NzQ0Ng== | simonw 9599 | 2019-10-07T05:41:17Z | 2019-10-07T05:41:17Z | MEMBER | Prototype code:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use better pagination (and implement progress bar) 503233021 | |
605316146 | https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-605316146 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1 | MDEyOklzc3VlQ29tbWVudDYwNTMxNjE0Ng== | simonw 9599 | 2020-03-27T21:09:15Z | 2020-03-27T21:09:22Z | MEMBER | For a progress bar I need to know how many total items there are. I found an undocumented API for this! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use better pagination (and implement progress bar) 503233021 | |
605325897 | https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-605325897 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1 | MDEyOklzc3VlQ29tbWVudDYwNTMyNTg5Nw== | simonw 9599 | 2020-03-27T21:37:26Z | 2020-03-27T21:38:37Z | MEMBER | I keep getting 503 errors even though I appear to be staying within the rate limit: ``` {'Date': 'Fri, 27 Mar 2020 21:35:57 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Server': 'Apache/2.4.25 (Debian)', 'Content-Location': 'get.php', 'Vary': 'negotiate', 'TCN': 'choice', 'Set-Cookie': '...; httponly', 'X-Frame-Options': 'SAMEORIGIN', 'Status': '200 OK', 'X-Limit-Key-Limit': '10000', 'X-Limit-Key-Remaining': '9960', 'X-Limit-Key-Reset': '282', 'X-Source': 'Pocket', 'P3P': 'policyref="/w3c/p3p.xml", CP="ALL CURa ADMa DEVa OUR IND UNI COM NAV INT STA PRE"'} [##----------------------------------] 6% 06:49:27 {'Date': 'Fri, 27 Mar 2020 21:36:06 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '23', 'Connection': 'keep-alive', 'Server': 'Apache/2.4.25 (Debian)', 'Content-Location': 'get.php', 'Vary': 'negotiate', 'TCN': 'choice', 'Set-Cookie': '...', 'X-Frame-Options': 'SAMEORIGIN', 'X-Error': 'Pocket is currently under heavy load. Please wait a moment and try again.', 'X-Error-Code': '199', 'Status': '503 Service Unavailable', 'X-Source': 'Pocket', 'P3P': 'policyref="/w3c/p3p.xml", CP="ALL CURa ADMa DEVa OUR IND UNI COM NAV INT STA PRE"'} ``` I'm going to try doing a few automatic retries any time I see a 503 error. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use better pagination (and implement progress bar) 503233021 | |
605327655 | https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-605327655 | https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1 | MDEyOklzc3VlQ29tbWVudDYwNTMyNzY1NQ== | simonw 9599 | 2020-03-27T21:42:49Z | 2020-03-27T21:42:49Z | MEMBER | Or maybe it was because of the current Google Cloud outage? https://news.ycombinator.com/item?id=22706677 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use better pagination (and implement progress bar) 503233021 | |
515226724 | https://github.com/dogsheep/healthkit-to-sqlite/issues/9#issuecomment-515226724 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9 | MDEyOklzc3VlQ29tbWVudDUxNTIyNjcyNA== | simonw 9599 | 2019-07-25T21:46:01Z | 2019-07-25T21:46:01Z | MEMBER | I can work around this here (prior to the fix in sqlite-utils) by setting the batch size to something a bit lower here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Too many SQL variables 472429048 | |
515322294 | https://github.com/dogsheep/healthkit-to-sqlite/issues/9#issuecomment-515322294 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9 | MDEyOklzc3VlQ29tbWVudDUxNTMyMjI5NA== | simonw 9599 | 2019-07-26T06:07:12Z | 2019-07-26T06:07:12Z | MEMBER | @tholo this should be fixed in just-released version 0.3.2 - could you run a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Too many SQL variables 472429048 | |
514496725 | https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514496725 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7 | MDEyOklzc3VlQ29tbWVudDUxNDQ5NjcyNQ== | simonw 9599 | 2019-07-24T06:20:59Z | 2019-07-24T06:20:59Z | MEMBER | I'm using https://pypi.org/project/memory-profiler/ to explore this in more detail:
Then:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Script uses a lot of RAM 472097220 | |
514498221 | https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514498221 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7 | MDEyOklzc3VlQ29tbWVudDUxNDQ5ODIyMQ== | simonw 9599 | 2019-07-24T06:26:49Z | 2019-07-24T06:26:49Z | MEMBER | Adding |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Script uses a lot of RAM 472097220 | |
514500253 | https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514500253 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7 | MDEyOklzc3VlQ29tbWVudDUxNDUwMDI1Mw== | simonw 9599 | 2019-07-24T06:34:28Z | 2019-07-24T06:34:28Z | MEMBER | Clearing the root element each time saved even more: |
{ "total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 2, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Script uses a lot of RAM 472097220 | |
513626742 | https://github.com/dogsheep/healthkit-to-sqlite/issues/6#issuecomment-513626742 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/6 | MDEyOklzc3VlQ29tbWVudDUxMzYyNjc0Mg== | simonw 9599 | 2019-07-22T03:28:55Z | 2019-07-22T03:28:55Z | MEMBER | Here's what it looks like now as separate tables: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Break up records into different tables for each type 470856782 | |
513514978 | https://github.com/dogsheep/healthkit-to-sqlite/issues/5#issuecomment-513514978 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDUxMzUxNDk3OA== | simonw 9599 | 2019-07-21T02:55:12Z | 2019-07-21T02:55:12Z | MEMBER | I'm going to show this by default. Users can pass |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add progress bar 470691622 | |
513625406 | https://github.com/dogsheep/healthkit-to-sqlite/issues/5#issuecomment-513625406 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDUxMzYyNTQwNg== | simonw 9599 | 2019-07-22T03:20:16Z | 2019-07-22T03:20:16Z | MEMBER | It now renders like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add progress bar 470691622 | |
513440090 | https://github.com/dogsheep/healthkit-to-sqlite/issues/4#issuecomment-513440090 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/4 | MDEyOklzc3VlQ29tbWVudDUxMzQ0MDA5MA== | simonw 9599 | 2019-07-20T06:11:50Z | 2019-07-20T06:11:50Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import Records 470640505 | ||
902355471 | https://github.com/dogsheep/healthkit-to-sqlite/issues/20#issuecomment-902355471 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20 | IC_kwDOC8tyDs41yNoP | simonw 9599 | 2021-08-20T01:09:07Z | 2021-08-20T01:09:07Z | MEMBER | Workaround:
See https://sqlite-utils.datasette.io/en/stable/cli.html#creating-indexes |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add index on workout_points.date 975166271 | |
902356871 | https://github.com/dogsheep/healthkit-to-sqlite/issues/20#issuecomment-902356871 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20 | IC_kwDOC8tyDs41yN-H | simonw 9599 | 2021-08-20T01:12:48Z | 2021-08-20T01:12:48Z | MEMBER | Also on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add index on workout_points.date 975166271 | |
513439411 | https://github.com/dogsheep/healthkit-to-sqlite/issues/2#issuecomment-513439411 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDUxMzQzOTQxMQ== | simonw 9599 | 2019-07-20T05:58:57Z | 2019-07-20T05:58:57Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Import workouts 470637152 | |
879477586 | https://github.com/dogsheep/healthkit-to-sqlite/issues/12#issuecomment-879477586 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12 | MDEyOklzc3VlQ29tbWVudDg3OTQ3NzU4Ng== | simonw 9599 | 2021-07-13T23:50:06Z | 2021-07-13T23:50:06Z | MEMBER | Unfortunately I don't think updating the database is practical, because the export doesn't include unique identifiers which can be used to update existing records and create new ones. Recreating from scratch works around that limitation. I've not explored workouts with SpatiaLite but that's a really good idea. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Some workout columns should be float, not text 727848625 | |
711074031 | https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711074031 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDcxMTA3NDAzMQ== | simonw 9599 | 2020-10-17T20:14:01Z | 2020-10-17T20:14:01Z | MEMBER | I'd be happy to teach the tool to look for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
export.xml file name varies with different language settings 723838331 | |
711074306 | https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711074306 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDcxMTA3NDMwNg== | simonw 9599 | 2020-10-17T20:16:22Z | 2020-10-17T20:16:22Z | MEMBER | The "first XML file in the root" solution is probably easier though! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
export.xml file name varies with different language settings 723838331 | |
711078917 | https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711078917 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDcxMTA3ODkxNw== | simonw 9599 | 2020-10-17T20:51:55Z | 2020-10-17T20:52:03Z | MEMBER | I switched my phone to Spanish and ran an export - I got a file called |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
export.xml file name varies with different language settings 723838331 | |
711079056 | https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711079056 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDcxMTA3OTA1Ng== | simonw 9599 | 2020-10-17T20:53:00Z | 2020-10-17T20:53:00Z | MEMBER | I think the safest thing is to sniff the first few lines of the file. Those should be the same no matter the language that was used: ```xml <!DOCTYPE HealthData [ ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
export.xml file name varies with different language settings 723838331 | |
711079760 | https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711079760 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDcxMTA3OTc2MA== | simonw 9599 | 2020-10-17T21:00:05Z | 2020-10-17T21:00:05Z | MEMBER | Checking for either |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
export.xml file name varies with different language settings 723838331 | |
711081703 | https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711081703 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11 | MDEyOklzc3VlQ29tbWVudDcxMTA4MTcwMw== | simonw 9599 | 2020-10-17T21:18:35Z | 2020-10-17T21:18:35Z | MEMBER | OK, if you upgrade to the just-released 1.0 this should work (it worked against my Spanish export). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
export.xml file name varies with different language settings 723838331 | |
550783316 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550783316 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDc4MzMxNg== | simonw 9599 | 2019-11-07T05:16:56Z | 2019-11-07T05:34:29Z | MEMBER | It looks like Apple changed the location of these in iOS 13 - they are now in separate |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | |
550806302 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550806302 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDgwNjMwMg== | simonw 9599 | 2019-11-07T05:33:31Z | 2019-11-07T05:33:31Z | MEMBER | The XML now includes references to these new files: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | |
550824838 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550824838 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDgyNDgzOA== | simonw 9599 | 2019-11-07T05:47:07Z | 2019-11-07T05:47:07Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | ||
550828084 | https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550828084 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10 | MDEyOklzc3VlQ29tbWVudDU1MDgyODA4NA== | simonw 9599 | 2019-11-07T05:49:24Z | 2019-11-07T05:49:24Z | MEMBER | So the fix there is going to be to detect the new This will be a little tricky because that function will need access to the zip file. It probably won't work at all for the mode where the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Failed to import workout points 519038979 | |
513437463 | https://github.com/dogsheep/healthkit-to-sqlite/issues/1#issuecomment-513437463 | https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/1 | MDEyOklzc3VlQ29tbWVudDUxMzQzNzQ2Mw== | simonw 9599 | 2019-07-20T05:19:59Z | 2019-07-20T05:19:59Z | MEMBER | I ran xml_analyser against the XML HealthKit
The most interesting bit is this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use XML Analyser to figure out the structure of the export XML 470637068 | |
886142671 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/3#issuecomment-886142671 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3 | IC_kwDODtX3eM400XbP | simonw 9599 | 2021-07-25T03:51:05Z | 2021-07-25T03:51:05Z | MEMBER | Prototype:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use HN algolia endpoint to retrieve trees 952189173 | |
886237834 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/3#issuecomment-886237834 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3 | IC_kwDODtX3eM400uqK | simonw 9599 | 2021-07-25T18:05:32Z | 2021-07-25T18:05:32Z | MEMBER | If you hit the endpoint for a comment that's part of a thread you get that comment and its recursive children: https://hn.algolia.com/api/v1/items/27941552 You can tell that it's not the top-level because the
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use HN algolia endpoint to retrieve trees 952189173 | |
886241674 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/3#issuecomment-886241674 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3 | IC_kwDODtX3eM400vmK | simonw 9599 | 2021-07-25T18:41:17Z | 2021-07-25T18:41:17Z | MEMBER | Got a TIL out of this: https://til.simonwillison.net/jq/extracting-objects-recursively |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use HN algolia endpoint to retrieve trees 952189173 | |
886135562 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886135562 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2 | IC_kwDODtX3eM400VsK | simonw 9599 | 2021-07-25T02:01:11Z | 2021-07-25T02:01:11Z | MEMBER | That page doesn't have an API but does look easy to scrape. The other option here is the HN Search API powered by Algolia, documented at https://hn.algolia.com/api |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for fetching Hacker News threads from the search API 952179830 | |
886135922 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886135922 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2 | IC_kwDODtX3eM400Vxy | simonw 9599 | 2021-07-25T02:06:20Z | 2021-07-25T02:06:20Z | MEMBER | https://hn.algolia.com/api/v1/search_by_date?query=simonwillison.net&restrictSearchableAttributes=url looks like it does what I want. https://hn.algolia.com/api/v1/search_by_date?query=simonwillison.net&restrictSearchableAttributes=url&hitsPerPage=1000 - returns 1000 at once. Otherwise you have to paginate using https://www.algolia.com/doc/api-reference/api-parameters/hitsPerPage/ says 1000 is the maximum. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for fetching Hacker News threads from the search API 952179830 | |
886136224 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886136224 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2 | IC_kwDODtX3eM400V2g | simonw 9599 | 2021-07-25T02:08:29Z | 2021-07-25T02:08:29Z | MEMBER | Prototype:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for fetching Hacker News threads from the search API 952179830 | |
886140431 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886140431 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2 | IC_kwDODtX3eM400W4P | simonw 9599 | 2021-07-25T03:12:57Z | 2021-07-25T03:12:57Z | MEMBER | I'm going to build a general-purpose |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Command for fetching Hacker News threads from the search API 952179830 | |
602094386 | https://github.com/dogsheep/hacker-news-to-sqlite/issues/1#issuecomment-602094386 | https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/1 | MDEyOklzc3VlQ29tbWVudDYwMjA5NDM4Ng== | simonw 9599 | 2020-03-21T19:47:46Z | 2020-03-21T19:47:46Z | MEMBER | Released as 0.3. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Set up full text search 585526292 | |
790384087 | https://github.com/dogsheep/google-takeout-to-sqlite/issues/6#issuecomment-790384087 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/6 | MDEyOklzc3VlQ29tbWVudDc5MDM4NDA4Nw== | simonw 9599 | 2021-03-04T07:22:51Z | 2021-03-04T07:22:51Z | MEMBER | 3 also mentions the conflicting version with other tools. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Upgrade to latest sqlite-utils 821841046 | |
786925280 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-786925280 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc4NjkyNTI4MA== | simonw 9599 | 2021-02-26T22:23:10Z | 2021-02-26T22:23:10Z | MEMBER | Thanks! I requested my Gmail export from takeout - once that arrives I'll test it against this and then merge the PR. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790312268 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790312268 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDMxMjI2OA== | simonw 9599 | 2021-03-04T05:48:16Z | 2021-03-04T05:48:16Z | MEMBER | Wow, my mbox is a 10.35 GB download! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790369076 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790369076 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDM2OTA3Ng== | simonw 9599 | 2021-03-04T06:54:46Z | 2021-03-04T06:54:46Z | MEMBER | The Rich-powered progress bar is pretty: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790370485 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790370485 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDM3MDQ4NQ== | simonw 9599 | 2021-03-04T06:57:25Z | 2021-03-04T06:57:48Z | MEMBER | The command takes quite a while to start running, presumably because this line causes it to have to scan the WHOLE file in order to generate a count: I'm fine with waiting though. It's not like this is a command people run every day - and without that count we can't show a progress bar, which seems pretty important for a process that takes this long. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790372621 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790372621 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDM3MjYyMQ== | simonw 9599 | 2021-03-04T07:01:18Z | 2021-03-04T07:01:18Z | MEMBER | I'm not sure if it would work, but there is an alternative pattern for showing a progress bar against a really large file that I've used in https://github.com/dogsheep/healthkit-to-sqlite/blob/3eb2b06bfe3b4faaf10e9cf9dfcb28e3d16c14ff/healthkit_to_sqlite/cli.py#L24-L57 and https://github.com/dogsheep/healthkit-to-sqlite/blob/3eb2b06bfe3b4faaf10e9cf9dfcb28e3d16c14ff/healthkit_to_sqlite/utils.py#L4-L19 (the It can be a bit of a convoluted pattern, and I'm not at all sure it would work for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790373024 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790373024 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDM3MzAyNA== | simonw 9599 | 2021-03-04T07:01:58Z | 2021-03-04T07:04:06Z | MEMBER | I got 9 warnings that look like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790378658 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790378658 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDM3ODY1OA== | simonw 9599 | 2021-03-04T07:12:48Z | 2021-03-04T07:12:48Z | MEMBER | It looks like the If I It would be great if we could store the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790379629 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790379629 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDM3OTYyOQ== | simonw 9599 | 2021-03-04T07:14:41Z | 2021-03-04T07:14:41Z | MEMBER | Confirmed: removing the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790380839 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790380839 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDM4MDgzOQ== | simonw 9599 | 2021-03-04T07:17:05Z | 2021-03-04T07:17:05Z | MEMBER | Looks like you're doing this:
I imagine the reason the column is a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790668263 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790668263 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDY2ODI2Mw== | simonw 9599 | 2021-03-04T14:43:58Z | 2021-03-04T14:43:58Z | MEMBER | I added this code to output a message ID on errors:
This was for the following error:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790669767 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790669767 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDY2OTc2Nw== | simonw 9599 | 2021-03-04T14:46:06Z | 2021-03-04T14:46:06Z | MEMBER | Solution could be to pre-process that string by splitting on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790693674 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790693674 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDY5MzY3NA== | simonw 9599 | 2021-03-04T15:18:36Z | 2021-03-04T15:18:36Z | MEMBER | I imported my 10GB mbox with 750,000 emails in it, ran this tool (with a hacked fix for the blob column problem) - and now a search that returns 92 results takes 25.37ms! This is fantastic. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
790695126 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790695126 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5 | MDEyOklzc3VlQ29tbWVudDc5MDY5NTEyNg== | simonw 9599 | 2021-03-04T15:20:42Z | 2021-03-04T15:20:42Z | MEMBER | I'm not sure why but my most recent import, when displayed in Datasette, looks like this: Sorting by |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
WIP: Add Gmail takeout mbox import 813880401 | |
747126777 | https://github.com/dogsheep/google-takeout-to-sqlite/issues/2#issuecomment-747126777 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/2 | MDEyOklzc3VlQ29tbWVudDc0NzEyNjc3Nw== | simonw 9599 | 2020-12-17T00:36:52Z | 2020-12-17T00:36:52Z | MEMBER | The memory profiler tricks I used in https://github.com/dogsheep/healthkit-to-sqlite/issues/7 could help figure out what's going on here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
killed by oomkiller on large location-history 769376447 | |
549094195 | https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549094195 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDU0OTA5NDE5NQ== | simonw 9599 | 2019-11-03T00:43:16Z | 2019-11-03T00:43:28Z | MEMBER | Also need to take #5 into account - if this command creates incomplete user records, how do we repair them? And make sure that if we run this command first any future commands that populate users don't break (probably just a case of using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
stargazers command, refs #4 516763727 | |
549230583 | https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549230583 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDU0OTIzMDU4Mw== | simonw 9599 | 2019-11-04T05:49:26Z | 2019-11-04T05:49:26Z | MEMBER | Adding the view from #10 would be useful here too. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
stargazers command, refs #4 516763727 | |
549233778 | https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549233778 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDU0OTIzMzc3OA== | simonw 9599 | 2019-11-04T06:14:40Z | 2019-11-04T06:14:40Z | MEMBER | Spotted a tricky problem: running But then... when it gets to the I need to find a way of NOT over-writing a good record with a thinner one. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
stargazers command, refs #4 516763727 | |
594154644 | https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-594154644 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDU5NDE1NDY0NA== | simonw 9599 | 2020-03-03T20:33:57Z | 2020-03-03T20:33:57Z | MEMBER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
stargazers command, refs #4 516763727 | |
623006004 | https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-623006004 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8 | MDEyOklzc3VlQ29tbWVudDYyMzAwNjAwNA== | simonw 9599 | 2020-05-02T20:00:26Z | 2020-05-02T20:00:26Z | MEMBER | I'm abandoning this in favour of a new implementation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
stargazers command, refs #4 516763727 | |
1498167714 | https://github.com/dogsheep/github-to-sqlite/issues/79#issuecomment-1498167714 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/79 | IC_kwDODFdgUs5ZTDmi | simonw 9599 | 2023-04-05T21:12:55Z | 2023-04-05T21:12:55Z | MEMBER | It's a rate limiting problem: https://github.com/dogsheep/github-to-sqlite/actions/runs/4622674009/jobs/8175633155
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Deploy demo job is failing due to rate limit 1570375808 | |
1188223933 | https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188223933 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74 | IC_kwDODFdgUs5G0tu9 | simonw 9599 | 2022-07-18T19:40:50Z | 2022-07-18T19:42:41Z | MEMBER | Here's how the demo is deployed: https://github.com/dogsheep/github-to-sqlite/blob/dbac2e5dd8a562b45d8255a265859cf8020ca22a/.github/workflows/deploy-demo.yml#L103-L119 I'm suspicious of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error in github-to-sqlite demo 1308461063 | |
1188225625 | https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188225625 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74 | IC_kwDODFdgUs5G0uJZ | simonw 9599 | 2022-07-18T19:41:52Z | 2022-07-18T19:41:52Z | MEMBER | https://github.com/Zopieux/py-gfm says that library is no longer maintained, and suggests https://github.com/Zopieux/pycmarkgfm as an alternative. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error in github-to-sqlite demo 1308461063 | |
1188228964 | https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188228964 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74 | IC_kwDODFdgUs5G0u9k | simonw 9599 | 2022-07-18T19:45:30Z | 2022-07-18T19:47:35Z | MEMBER |
I'm going to drop the GFM rendering from the demo, and just treat it as regular markdown. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error in github-to-sqlite demo 1308461063 | |
1188233729 | https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188233729 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74 | IC_kwDODFdgUs5G0wIB | simonw 9599 | 2022-07-18T19:51:02Z | 2022-07-18T19:51:02Z | MEMBER | Takes 30m to deploy the demo! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error in github-to-sqlite demo 1308461063 | |
1188317682 | https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188317682 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74 | IC_kwDODFdgUs5G1Eny | simonw 9599 | 2022-07-18T21:14:22Z | 2022-07-18T21:14:22Z | MEMBER | That fixed it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error in github-to-sqlite demo 1308461063 | |
1188223299 | https://github.com/dogsheep/github-to-sqlite/pull/73#issuecomment-1188223299 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/73 | IC_kwDODFdgUs5G0tlD | simonw 9599 | 2022-07-18T19:40:06Z | 2022-07-18T19:40:06Z | MEMBER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Fixing 'NoneType' object has no attribute 'items' 1261884917 | |
1105474232 | https://github.com/dogsheep/github-to-sqlite/issues/72#issuecomment-1105474232 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/72 | IC_kwDODFdgUs5B5DK4 | simonw 9599 | 2022-04-21T17:02:15Z | 2022-04-21T17:02:15Z | MEMBER | That's interesting - yeah it looks like the number of pages can be derived from the https://docs.github.com/en/rest/guides/traversing-with-pagination |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
feature: display progress bar when downloading multi-page responses 1211283427 | |
541721437 | https://github.com/dogsheep/github-to-sqlite/issues/7#issuecomment-541721437 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/7 | MDEyOklzc3VlQ29tbWVudDU0MTcyMTQzNw== | simonw 9599 | 2019-10-14T14:44:12Z | 2019-10-14T14:44:12Z | MEMBER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
issue-comments command for importing issue comments 506276893 | ||
985928838 | https://github.com/dogsheep/github-to-sqlite/issues/69#issuecomment-985928838 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69 | IC_kwDODFdgUs46xBSG | simonw 9599 | 2021-12-04T00:34:52Z | 2021-12-04T00:34:52Z | MEMBER | First attempt at this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
View that combines issues and issue comments 1071071397 | |
861041597 | https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861041597 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64 | MDEyOklzc3VlQ29tbWVudDg2MTA0MTU5Nw== | simonw 9599 | 2021-06-14T22:44:54Z | 2021-06-14T22:44:54Z | MEMBER | Have you found a way to access events in GraphQL? I can only see way to access a timeline of events for a single issue or a single pull request. See also https://github.community/t/get-event-equivalent-for-v4/13600/2 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
feature: support "events" 920636216 | |
861042050 | https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861042050 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64 | MDEyOklzc3VlQ29tbWVudDg2MTA0MjA1MA== | simonw 9599 | 2021-06-14T22:45:42Z | 2021-06-14T22:45:42Z | MEMBER | I'm definitely interested in supporting events in this tool - see #14. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
feature: support "events" 920636216 | |
844249385 | https://github.com/dogsheep/github-to-sqlite/pull/61#issuecomment-844249385 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/61 | MDEyOklzc3VlQ29tbWVudDg0NDI0OTM4NQ== | simonw 9599 | 2021-05-19T16:07:06Z | 2021-05-19T16:07:06Z | MEMBER | Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
fixing typo in get cli help text 797108702 | |
770071568 | https://github.com/dogsheep/github-to-sqlite/issues/60#issuecomment-770071568 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60 | MDEyOklzc3VlQ29tbWVudDc3MDA3MTU2OA== | simonw 9599 | 2021-01-29T21:56:15Z | 2021-01-29T21:56:15Z | MEMBER | I really like the way you're using pipes here - really smart. It's similar to how I build the demo database in this GitHub Actions workflow:
It lets you do things like:
|
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use Data from SQLite in other commands 797097140 | |
541387822 | https://github.com/dogsheep/github-to-sqlite/issues/6#issuecomment-541387822 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/6 | MDEyOklzc3VlQ29tbWVudDU0MTM4NzgyMg== | simonw 9599 | 2019-10-13T05:27:39Z | 2019-10-13T05:27:39Z | MEMBER | This should be fixed by https://github.com/dogsheep/github-to-sqlite/commit/552543a74970f8a3a3f87f887be23a0c6eb1cb5b |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
sqlite3.OperationalError: table users has no column named bio 504238461 | |
541387941 | https://github.com/dogsheep/github-to-sqlite/issues/6#issuecomment-541387941 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/6 | MDEyOklzc3VlQ29tbWVudDU0MTM4Nzk0MQ== | simonw 9599 | 2019-10-13T05:30:19Z | 2019-10-13T05:30:19Z | MEMBER | Fix released in 0.5: https://github.com/dogsheep/github-to-sqlite/releases/tag/0.5 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
sqlite3.OperationalError: table users has no column named bio 504238461 | |
844250232 | https://github.com/dogsheep/github-to-sqlite/pull/59#issuecomment-844250232 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59 | MDEyOklzc3VlQ29tbWVudDg0NDI1MDIzMg== | simonw 9599 | 2021-05-19T16:08:10Z | 2021-05-19T16:08:10Z | MEMBER | Thanks for catching this. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Remove unneeded exists=True for -a/--auth flag. 771872303 | |
746734412 | https://github.com/dogsheep/github-to-sqlite/issues/58#issuecomment-746734412 | https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58 | MDEyOklzc3VlQ29tbWVudDc0NjczNDQxMg== | simonw 9599 | 2020-12-16T17:58:56Z | 2020-12-16T17:58:56Z | MEMBER | I'm going to rewrite those |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Readme HTML has broken internal links 769150394 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue >30