html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/dogsheep/twitter-to-sqlite/issues/45#issuecomment-616029262,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/45,616029262,MDEyOklzc3VlQ29tbWVudDYxNjAyOTI2Mg==,9599,simonw,2020-04-19T04:39:21Z,2020-04-19T04:39:21Z,MEMBER,"![44714E00-8CC5-46CD-9E48-1F4DD148FCC8](https://user-images.githubusercontent.com/9599/79679696-09b6d300-81bd-11ea-80e4-0653d92e4f58.jpeg)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602619330,Use raise_for_status() everywhere,
https://github.com/dogsheep/google-takeout-to-sqlite/issues/6#issuecomment-790384087,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/6,790384087,MDEyOklzc3VlQ29tbWVudDc5MDM4NDA4Nw==,9599,simonw,2021-03-04T07:22:51Z,2021-03-04T07:22:51Z,MEMBER,#3 also mentions the conflicting version with other tools.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",821841046,Upgrade to latest sqlite-utils,
https://github.com/dogsheep/github-to-sqlite/issues/19#issuecomment-602782437,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/19,602782437,MDEyOklzc3VlQ29tbWVudDYwMjc4MjQzNw==,9599,simonw,2020-03-23T18:36:12Z,2020-03-23T18:51:23Z,MEMBER,"* `commits`: `message`
* `issue_comments`: `body`
* `issues`: `title`, `body`
* `labels`: `name`, `description`
* `licenses`: `name`
* `milestones`: `title`, `description`
* `releases`: `name`, `body`
* `repos`: `name`, `description`
* `users`: `login`, `name` (maybe `company`, `bio`)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585850715,"Enable full-text search for more stuff (like commits, issues and issue_comments)",
https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-530417631,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8,530417631,MDEyOklzc3VlQ29tbWVudDUzMDQxNzYzMQ==,9599,simonw,2019-09-11T14:52:44Z,2019-09-14T19:09:22Z,MEMBER,- [x] This needs documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",490803176,--sql and --attach options for feeding commands from SQL queries,
https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776808,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776808,MDEyOklzc3VlQ29tbWVudDcwNjc3NjgwOA==,9599,simonw,2020-10-11T22:23:14Z,2020-10-11T22:23:14Z,MEMBER,... but it's still important to be able to get to the rendered note directly from the browse notes `/evernote/notes` page. Maybe use a simple `render_cell()` hook that just knows how to generate the link to the rendered note page?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from <en-media> tags inline in Datasette,
https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-544648863,https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1,544648863,MDEyOklzc3VlQ29tbWVudDU0NDY0ODg2Mw==,9599,simonw,2019-10-21T18:36:03Z,2019-10-21T18:36:03Z,MEMBER,"<img width=""1418"" alt=""natalie__select_rsid__genotype__case_genotype_when__AA__then__brown_eye_color__80__of_the_time__when__AG__then__brown_eye_color__when__GG__then__blue_eye_color__99__of_the_time__end_as_interpretation_from_genome_where_rsid____rs12913832__an"" src=""https://user-images.githubusercontent.com/9599/67232810-f4701000-f3f6-11e9-90e2-8fe2cca1d98d.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",496415321,Figure out some interesting example SQL queries,
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395103,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626395103,MDEyOklzc3VlQ29tbWVudDYyNjM5NTEwMw==,9599,simonw,2020-05-10T21:51:36Z,2020-05-10T21:51:36Z,MEMBER,"@RhetTbull I tried that workaround and it turns out I'm getting this error on ALL of my photos now!

It's weird: a few day ago this wasn't happening. Now it's happening to everything. I'm not sure what I might have changed. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,bpylist.archiver.CircularReference: archive has a cycle with uid(13),
https://github.com/dogsheep/healthkit-to-sqlite/issues/9#issuecomment-515322294,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9,515322294,MDEyOklzc3VlQ29tbWVudDUxNTMyMjI5NA==,9599,simonw,2019-07-26T06:07:12Z,2019-07-26T06:07:12Z,MEMBER,@tholo this should be fixed in just-released version 0.3.2 - could you run a `pip install -U healthkit-to-sqlite` and let me know if it works for you now?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",472429048,Too many SQL variables,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606305701,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606305701,MDEyOklzc3VlQ29tbWVudDYwNjMwNTcwMQ==,9599,simonw,2020-03-30T23:30:27Z,2020-03-30T23:30:27Z,MEMBER,A better alternative would be to maintain a separate table with the last seen since value for when we ran `user-timeline` for any specific user.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/dogsheep-photos/issues/9#issuecomment-618725155,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/9,618725155,MDEyOklzc3VlQ29tbWVudDYxODcyNTE1NQ==,9599,simonw,2020-04-23T23:39:14Z,2020-04-23T23:39:14Z,MEMBER,"A few minutes later...
```
Fetching existing keys from S3...
Got 22,446 existing keys
Calculating hashes  [####################################]  100%          
22,441 hashed files, 610 are not yet in S3
Uploading 0.99 GB
Uploading 610 photos  [------------------------------------]  1/610  03:10:35
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",605938063,"upload command should be resumable, should only upload photos not already uploaded",
https://github.com/dogsheep/evernote-to-sqlite/issues/1#issuecomment-706775706,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/1,706775706,MDEyOklzc3VlQ29tbWVudDcwNjc3NTcwNg==,9599,simonw,2020-10-11T22:14:00Z,2020-10-11T22:14:00Z,MEMBER,A live demo would be good too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718934942,Documentation on how to use this with Datasette,
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-543269396,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10,543269396,MDEyOklzc3VlQ29tbWVudDU0MzI2OTM5Ng==,9599,simonw,2019-10-17T17:02:07Z,2019-10-17T17:02:07Z,MEMBER,A neat trick that Click does is detecting if an interactive terminal is attached and NOT showing a progress bar if there isn't one. Need to figure out how to do that with tqdm.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",492297930,Rethink progress bars for various commands,
https://github.com/dogsheep/dogsheep-beta/issues/19#issuecomment-688626037,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/19,688626037,MDEyOklzc3VlQ29tbWVudDY4ODYyNjAzNw==,9599,simonw,2020-09-08T05:27:07Z,2020-09-08T05:27:07Z,MEMBER,"A really clever way to do this would be with triggers. The indexer script would add triggers to each of the database tables that it is indexing - each in their own database.

Those triggers would then maintain a `_index_queue_` table. This table would record the primary key of rows that are added, modified or deleted. The indexer could then work by reading through the `_index_queue_` table, re-indexing (or deleting) just the primary keys listed there, and then emptying the queue once it has finished.

This would add a small amount of overhead to insert/update/delete queries run against the table. My hunch is that the overhead would be miniscule, but I could still allow people to opt-out for tables that are so high traffic that this would matter.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695556681,Figure out incremental re-indexing,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850453,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606850453,MDEyOklzc3VlQ29tbWVudDYwNjg1MDQ1Mw==,9599,simonw,2020-03-31T20:14:58Z,2020-04-01T03:03:50Z,MEMBER,Actually I'll hard-code the population of `since_id_types` to get known ID constants.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011421,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,607011421,MDEyOklzc3VlQ29tbWVudDYwNzAxMTQyMQ==,9599,simonw,2020-04-01T03:47:37Z,2020-04-01T03:55:08Z,MEMBER,"Actually a single table with a `type` integer ID referencing a `count_history_types` table would better match the way I implemented the `since_ids` table:

https://github.com/dogsheep/twitter-to-sqlite/blob/4b6c8d8c1cc6fefdb566ec8506157133f47c569a/twitter_to_sqlite/utils.py#L331-L341

In which case the compound primary key would be `type`, `user`, `datetime`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695879531,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695879531,MDEyOklzc3VlQ29tbWVudDY5NTg3OTUzMQ==,9599,simonw,2020-09-21T02:55:28Z,2020-09-21T02:55:54Z,MEMBER,"Actually for the tie-breaker it should be something like https://latest.datasette.io/fixtures?sql=select+pk%2C+created%2C+planet_int%2C+on_earth%2C+state%2C+city_id%2C+neighborhood%2C+tags%2C+complex_array%2C+distinct_some_null+from+facetable+where+%28created+%3E+%3Ap1+or+%28created+%3D+%3Ap1+and+%28%28pk+%3E+%3Ap0%29%29%29%29+order+by+created%2C+pk+limit+11&p0=10&p1=2019-01-16+08%3A00%3A00
```sql
where
  (
    created > :p1
    or (
      created = :p1
      and ((pk > :p0))
    )
  )
```
But with `rowid` and `timestamp` in place of `pk` and `created`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination,
https://github.com/dogsheep/dogsheep-beta/issues/2#issuecomment-685115519,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/2,685115519,MDEyOklzc3VlQ29tbWVudDY4NTExNTUxOQ==,9599,simonw,2020-09-01T20:31:57Z,2020-09-01T20:31:57Z,MEMBER,"Actually this doesn't work: you can't turn on stemming for specific tables, because all of the content goes into a single `search_index` table which is configured the same way.

So stemming needs to be a global option.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",689809225,Apply porter stemming,
https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514498221,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7,514498221,MDEyOklzc3VlQ29tbWVudDUxNDQ5ODIyMQ==,9599,simonw,2019-07-24T06:26:49Z,2019-07-24T06:26:49Z,MEMBER,"Adding `el.clear()` got me a huge improvement:

<img width=""1128"" alt=""Screen Shot 2019-07-24 at 8 23 26 AM"" src=""https://user-images.githubusercontent.com/9599/61770130-c3c70d00-adec-11e9-886a-93a2cea5bf4b.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",472097220,Script uses a lot of RAM,
https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549230583,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,549230583,MDEyOklzc3VlQ29tbWVudDU0OTIzMDU4Mw==,9599,simonw,2019-11-04T05:49:26Z,2019-11-04T05:49:26Z,MEMBER,Adding the view from #10 would be useful here too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727,"stargazers command, refs #4",
https://github.com/dogsheep/apple-notes-to-sqlite/issues/1#issuecomment-1461230197,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/1,1461230197,IC_kwDOJHON9s5XGJp1,9599,simonw,2023-03-09T03:51:36Z,2023-03-09T03:51:36Z,MEMBER,"After a few more rounds I got to this script, which outputs them to a `/tmp/notes.txt` file:

```zsh
#!/bin/zsh

osascript -e '
set notesFile to ""/tmp/notes.txt""
set fileRef to open for access notesFile with write permission
tell application ""Notes""
   repeat with eachNote in every note
      set noteId to the id of eachNote
      set noteTitle to the name of eachNote
      set noteBody to the body of eachNote
      write ""------------------------"" & ""\n"" to fileRef
      write noteId & ""\n"" to fileRef
      write noteTitle & ""\n\n"" to fileRef
      write noteBody & ""\n"" to fileRef
   end repeat
end tell
close access fileRef'
```
Then I wrote this little Python script to load them into a database:
```python
import sqlite_utils
split = b""------------------------\n""
s = open(""/tmp/notes.txt"", ""rb"").read()
notes = [n.decode(""mac_roman"") for n in s.split(split) if n]

cleaned_notes = [{
    ""id"": n.split(""\n"")[0],
    ""title"": n.split(""\n"")[1],
    ""body"": ""\n"".join(n.split(""\n"")[2:]).strip()
} for n in notes]

db = sqlite_utils.Database(""/tmp/notes.db"")
db[""notes""].insert_all(cleaned_notes)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616347574,Initial proof of concept with ChatGPT,
https://github.com/dogsheep/pocket-to-sqlite/issues/10#issuecomment-1221622873,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/10,1221622873,IC_kwDODLZ_YM5I0HxZ,9599,simonw,2022-08-21T21:19:25Z,2022-08-21T21:19:25Z,MEMBER,"Agreed, that would be a much better implementation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1246826792,"When running `auth` command, don't overwrite an existing auth.json file",
https://github.com/dogsheep/evernote-to-sqlite/issues/9#issuecomment-777821383,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/9,777821383,MDEyOklzc3VlQ29tbWVudDc3NzgyMTM4Mw==,9599,simonw,2021-02-11T22:01:28Z,2021-02-11T22:01:28Z,MEMBER,"Aha! I think I've figured out what's going on here.

The CData blocks containing the notes look like this:

`<![CDATA[<!DOCTYPE en-note SYSTEM ""http://xml.evernote.com/pub/enml2.dtd""><en-note><div>This note includes two images.</div><div><br /></div>...`

The DTD at http://xml.evernote.com/pub/enml2.dtd includes some entities:

```
<!--=========== External character mnemonic entities ===================-->

<!ENTITY % HTMLlat1 PUBLIC
   ""-//W3C//ENTITIES Latin 1 for XHTML//EN""
   ""http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent"">
%HTMLlat1;

<!ENTITY % HTMLsymbol PUBLIC
   ""-//W3C//ENTITIES Symbols for XHTML//EN""
   ""http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent"">
%HTMLsymbol;

<!ENTITY % HTMLspecial PUBLIC
   ""-//W3C//ENTITIES Special for XHTML//EN""
   ""http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent"">
%HTMLspecial;
```
So I need to be able to handle all of those different entities. I think I can do that using `html.entities.entitydefs` from the Python standard library, which looks a bit like this:

```python
{'Aacute': 'Á',
 'aacute': 'á',
 'Aacute;': 'Á',
 'aacute;': 'á',
 'Abreve;': 'Ă',
 'abreve;': 'ă',
 'ac;': '∾',
 'acd;': '∿',
# ...
}
```

","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",748372469,ParseError: undefined entity &scaron;,
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395209,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626395209,MDEyOklzc3VlQ29tbWVudDYyNjM5NTIwOQ==,9599,simonw,2020-05-10T21:52:42Z,2020-05-10T21:52:42Z,MEMBER,"Aha! It looks like I accidentally installed the old bplist into the same environment:
```
$ pip freeze | grep bpylist
bpylist==0.1.4
bpylist2==3.0.0
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,bpylist.archiver.CircularReference: archive has a cycle with uid(13),
https://github.com/dogsheep/github-to-sqlite/issues/26#issuecomment-614795712,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26,614795712,MDEyOklzc3VlQ29tbWVudDYxNDc5NTcxMg==,9599,simonw,2020-04-16T17:40:27Z,2020-04-16T17:40:27Z,MEMBER,Aha! it was missing from the `fetch_repo()` function.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601271612,Topics are missing from repositories,
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623199214,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623199214,MDEyOklzc3VlQ29tbWVudDYyMzE5OTIxNA==,9599,simonw,2020-05-03T23:14:08Z,2020-05-03T23:14:08Z,MEMBER,"Albums have UUIDs:
```
(Pdb) photo.album_info[0].__dict__
{'_uuid': '17816791-ABF3-447B-942C-9FA8065EEBBA', '_db': osxphotos.PhotosDB(dbfile='/Users/simon/Pictures/Photos Library.photoslibrary/database/photos.db'), '_title': 'Geotaggable Photos geotagged'}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/dogsheep-photos/issues/8#issuecomment-618100658,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/8,618100658,MDEyOklzc3VlQ29tbWVudDYxODEwMDY1OA==,9599,simonw,2020-04-23T00:03:35Z,2020-04-23T00:03:35Z,MEMBER,"Also MD5 isn't guaranteed for the ETag:
> If an object is created by either the Multipart Upload or Part Copy operation, the ETag is not an MD5 digest, regardless of the method of encryption.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",605147638,Should I have used MD5 instead of SHA256?,
https://github.com/dogsheep/twitter-to-sqlite/issues/4#issuecomment-540879620,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/4,540879620,MDEyOklzc3VlQ29tbWVudDU0MDg3OTYyMA==,9599,simonw,2019-10-11T02:59:16Z,2019-10-11T02:59:16Z,MEMBER,Also import ad preferences and all that other junk.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488835586,Command for importing data from a Twitter Export file,
https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549094195,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,549094195,MDEyOklzc3VlQ29tbWVudDU0OTA5NDE5NQ==,9599,simonw,2019-11-03T00:43:16Z,2019-11-03T00:43:28Z,MEMBER,"Also need to take #5 into account - if this command creates incomplete user records, how do we repair them?

And make sure that if we run this command first any future commands that populate users don't break (probably just a case of using `alter=True` in a few places).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727,"stargazers command, refs #4",
https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-527200332,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3,527200332,MDEyOklzc3VlQ29tbWVudDUyNzIwMDMzMg==,9599,simonw,2019-09-02T16:32:20Z,2019-09-02T16:32:39Z,MEMBER,"Also needed: an option for ""fetch all checkins created within the last X days"".

This should help provide support for that Swarm feature where you can retroactively checkin to places in the past.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",487600595,Option to fetch only checkins more recent than the current max checkin,
https://github.com/dogsheep/healthkit-to-sqlite/issues/20#issuecomment-902356871,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20,902356871,IC_kwDOC8tyDs41yN-H,9599,simonw,2021-08-20T01:12:48Z,2021-08-20T01:12:48Z,MEMBER,Also on `workout_points.workout_id` to speed up queries to show all points in a specific workout.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",975166271,Add index on workout_points.date,
https://github.com/dogsheep/dogsheep-beta/issues/17#issuecomment-689226390,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/17,689226390,MDEyOklzc3VlQ29tbWVudDY4OTIyNjM5MA==,9599,simonw,2020-09-09T00:36:07Z,2020-09-09T00:36:07Z,MEMBER,"Alternative names:

- type
- record_type
- doctype

I think `type` is right. It matches what Elasticsearch used to call their equivalent of this (before they removed the feature!). https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694500679,"Rename ""table"" to ""type""",
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623004836,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623004836,MDEyOklzc3VlQ29tbWVudDYyMzAwNDgzNg==,9599,simonw,2020-05-02T19:49:51Z,2020-05-02T19:49:51Z,MEMBER,"Alternative pattern:
```
sqlite-utils releases.db 'select full_name from repos' --csv --no-headers \
  | tr -d '\r' \
  | xargs github-to-sqlite stargazers stars.db
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/dogsheep-beta/issues/18#issuecomment-688622995,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18,688622995,MDEyOklzc3VlQ29tbWVudDY4ODYyMjk5NQ==,9599,simonw,2020-09-08T05:15:21Z,2020-09-08T05:15:21Z,MEMBER,"Alternatively it could run as it does now but add a `DELETE FROM index1.search_index WHERE key not in (select key from ...)`.

I'm not sure which would be more efficient.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695553522,Deleted records stay in the search index,
https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776242,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776242,MDEyOklzc3VlQ29tbWVudDcwNjc3NjI0Mg==,9599,simonw,2020-10-11T22:18:30Z,2020-10-11T22:19:48Z,MEMBER,"Alternatively, rather than relying on `datasette-media` this could base64-embed the images. `evernote-to-sqlite` could register itself as a Datasette plugin that knows how to do this.

Maybe rename the column to `evernote_content` and register a render cell hook that knows how to rewrite those note bodies so that they are visible?

Might need to feed them through Bleach too, just in case any nasty code can get into them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from <en-media> tags inline in Datasette,
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622136585,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622136585,MDEyOklzc3VlQ29tbWVudDYyMjEzNjU4NQ==,9599,simonw,2020-04-30T21:55:51Z,2020-04-30T21:55:51Z,MEMBER,"And to find the ""Next"" pagination link:
```python
soup.select("".paginate-container"")[0].find(""a"", text=""Next"")
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,Command for retrieving dependents for a repo,
https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735482546,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735482546,MDEyOklzc3VlQ29tbWVudDczNTQ4MjU0Ng==,9599,simonw,2020-11-30T00:22:02Z,2020-11-30T00:22:02Z,MEMBER,"As for reviews... here's the output of `github-to-sqlite get https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48/reviews --accept 'application/vnd.github.v3+json'`

```json
[
    {
        ""id"": 503368921,
        ""node_id"": ""MDE3OlB1bGxSZXF1ZXN0UmV2aWV3NTAzMzY4OTIx"",
        ""user"": {
            ""login"": ""simonw"",
            ""id"": 9599,
            ""node_id"": ""MDQ6VXNlcjk1OTk="",
            ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"",
            ""gravatar_id"": """",
            ""url"": ""https://api.github.com/users/simonw"",
            ""html_url"": ""https://github.com/simonw"",
            ""followers_url"": ""https://api.github.com/users/simonw/followers"",
            ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"",
            ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"",
            ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"",
            ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"",
            ""organizations_url"": ""https://api.github.com/users/simonw/orgs"",
            ""repos_url"": ""https://api.github.com/users/simonw/repos"",
            ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"",
            ""received_events_url"": ""https://api.github.com/users/simonw/received_events"",
            ""type"": ""User"",
            ""site_admin"": false
        },
        ""body"": """",
        ""state"": ""CHANGES_REQUESTED"",
        ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-503368921"",
        ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"",
        ""author_association"": ""MEMBER"",
        ""_links"": {
            ""html"": {
                ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-503368921""
            },
            ""pull_request"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48""
            }
        },
        ""submitted_at"": ""2020-10-06T21:28:40Z"",
        ""commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c""
    },
    {
        ""id"": 513118561,
        ""node_id"": ""MDE3OlB1bGxSZXF1ZXN0UmV2aWV3NTEzMTE4NTYx"",
        ""user"": {
            ""login"": ""adamjonas"",
            ""id"": 755825,
            ""node_id"": ""MDQ6VXNlcjc1NTgyNQ=="",
            ""avatar_url"": ""https://avatars1.githubusercontent.com/u/755825?v=4"",
            ""gravatar_id"": """",
            ""url"": ""https://api.github.com/users/adamjonas"",
            ""html_url"": ""https://github.com/adamjonas"",
            ""followers_url"": ""https://api.github.com/users/adamjonas/followers"",
            ""following_url"": ""https://api.github.com/users/adamjonas/following{/other_user}"",
            ""gists_url"": ""https://api.github.com/users/adamjonas/gists{/gist_id}"",
            ""starred_url"": ""https://api.github.com/users/adamjonas/starred{/owner}{/repo}"",
            ""subscriptions_url"": ""https://api.github.com/users/adamjonas/subscriptions"",
            ""organizations_url"": ""https://api.github.com/users/adamjonas/orgs"",
            ""repos_url"": ""https://api.github.com/users/adamjonas/repos"",
            ""events_url"": ""https://api.github.com/users/adamjonas/events{/privacy}"",
            ""received_events_url"": ""https://api.github.com/users/adamjonas/received_events"",
            ""type"": ""User"",
            ""site_admin"": false
        },
        ""body"": """",
        ""state"": ""COMMENTED"",
        ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513118561"",
        ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"",
        ""author_association"": ""CONTRIBUTOR"",
        ""_links"": {
            ""html"": {
                ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513118561""
            },
            ""pull_request"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48""
            }
        },
        ""submitted_at"": ""2020-10-20T20:45:05Z"",
        ""commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c""
    },
    {
        ""id"": 513127529,
        ""node_id"": ""MDE3OlB1bGxSZXF1ZXN0UmV2aWV3NTEzMTI3NTI5"",
        ""user"": {
            ""login"": ""adamjonas"",
            ""id"": 755825,
            ""node_id"": ""MDQ6VXNlcjc1NTgyNQ=="",
            ""avatar_url"": ""https://avatars1.githubusercontent.com/u/755825?v=4"",
            ""gravatar_id"": """",
            ""url"": ""https://api.github.com/users/adamjonas"",
            ""html_url"": ""https://github.com/adamjonas"",
            ""followers_url"": ""https://api.github.com/users/adamjonas/followers"",
            ""following_url"": ""https://api.github.com/users/adamjonas/following{/other_user}"",
            ""gists_url"": ""https://api.github.com/users/adamjonas/gists{/gist_id}"",
            ""starred_url"": ""https://api.github.com/users/adamjonas/starred{/owner}{/repo}"",
            ""subscriptions_url"": ""https://api.github.com/users/adamjonas/subscriptions"",
            ""organizations_url"": ""https://api.github.com/users/adamjonas/orgs"",
            ""repos_url"": ""https://api.github.com/users/adamjonas/repos"",
            ""events_url"": ""https://api.github.com/users/adamjonas/events{/privacy}"",
            ""received_events_url"": ""https://api.github.com/users/adamjonas/received_events"",
            ""type"": ""User"",
            ""site_admin"": false
        },
        ""body"": """",
        ""state"": ""COMMENTED"",
        ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513127529"",
        ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"",
        ""author_association"": ""CONTRIBUTOR"",
        ""_links"": {
            ""html"": {
                ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#pullrequestreview-513127529""
            },
            ""pull_request"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48""
            }
        },
        ""submitted_at"": ""2020-10-20T20:57:33Z"",
        ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3""
    }
]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022,Feature: pull request reviews and comments,
https://github.com/dogsheep/dogsheep-beta/issues/17#issuecomment-687880459,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/17,687880459,MDEyOklzc3VlQ29tbWVudDY4Nzg4MDQ1OQ==,9599,simonw,2020-09-06T19:36:32Z,2020-09-06T19:36:32Z,MEMBER,At some point I may even want to support search types which are indexed from (and inflated from) more than one database file. I'm going to ignore that for the moment though.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694500679,"Rename ""table"" to ""type""",
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461260978,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2,1461260978,IC_kwDOJHON9s5XGRKy,9599,simonw,2023-03-09T04:27:18Z,2023-03-09T04:27:18Z,MEMBER,"Before that conversion:

    Monday, March 6, 2023 at 11:55:15 AM

After:

    2023-03-06T11:55:15","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616354999,First working version,
https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-594151327,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12,594151327,MDEyOklzc3VlQ29tbWVudDU5NDE1MTMyNw==,9599,simonw,2020-03-03T20:26:15Z,2020-03-03T20:32:23Z,MEMBER,"Better version (since this also includes JSON array of repository topics):
```sql
CREATE VIEW recent_releases AS select
  repos.rowid as rowid,
  json_object(""label"", repos.full_name, ""href"", repos.html_url) as repo,
  json_object(
    ""href"",
    releases.html_url,
    ""label"",
    releases.name
  ) as release,
  substr(releases.published_at, 0, 11) as date,
  releases.body as body_markdown,
  releases.published_at,
  coalesce(repos.topics, '[]') as topics
from
  releases
  join repos on repos.id = releases.repo
order by
  releases.published_at desc
```
That `repos.rowid as rowid` bit is necessary because otherwise clicking on a link in facet-by-topic doesn't return any results.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520756546,Add this view for seeing new releases,
https://github.com/dogsheep/github-to-sqlite/issues/36#issuecomment-622461025,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/36,622461025,MDEyOklzc3VlQ29tbWVudDYyMjQ2MTAyNQ==,9599,simonw,2020-05-01T16:34:24Z,2020-05-01T16:34:24Z,MEMBER,Blocked on #37,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610842926,Add view for better display of dependent repos,
https://github.com/dogsheep/github-to-sqlite/issues/10#issuecomment-622461122,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10,622461122,MDEyOklzc3VlQ29tbWVudDYyMjQ2MTEyMg==,9599,simonw,2020-05-01T16:34:39Z,2020-05-01T16:34:39Z,MEMBER,Blocked on #37,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516967682,Add this repos_starred view,
https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-622461223,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12,622461223,MDEyOklzc3VlQ29tbWVudDYyMjQ2MTIyMw==,9599,simonw,2020-05-01T16:34:52Z,2020-05-01T16:34:52Z,MEMBER,Blocked on #37,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520756546,Add this view for seeing new releases,
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693789129,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693789129,MDEyOklzc3VlQ29tbWVudDY5Mzc4OTEyOQ==,9599,simonw,2020-09-17T03:40:01Z,2020-09-17T03:40:01Z,MEMBER,"Bug with endpoints that return dictionaries rather than arrays:
```
github-to-sqlite get /users/simonw
[
    ""login"",
    ""id"",
    ""node_id"",
    ""avatar_url"",
    ""gravatar_id"",
    ""url"",
    ""html_url"",
    ""followers_url"",
    ""following_url"",
    ""gists_url"",
    ""starred_url"",
    ""subscriptions_url"",
    ""organizations_url"",
    ""repos_url"",
    ""events_url"",
    ""received_events_url"",
    ""type"",
    ""site_admin"",
    ""name"",
    ""company"",
    ""blog"",
    ""location"",
    ""email"",
    ""hireable"",
    ""bio"",
    ""twitter_username"",
    ""public_repos"",
    ""public_gists"",
    ""followers"",
    ""following"",
    ""created_at"",
    ""updated_at""
]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615945056,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615945056,MDEyOklzc3VlQ29tbWVudDYxNTk0NTA1Ng==,9599,simonw,2020-04-18T20:42:41Z,2020-04-18T20:42:41Z,MEMBER,"But... `list_objects` failed for both of my keys (read and write):

![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670798-75c41780-817a-11ea-9907-2cbc4a2e497c.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/twitter-to-sqlite/issues/9#issuecomment-530028567,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/9,530028567,MDEyOklzc3VlQ29tbWVudDUzMDAyODU2Nw==,9599,simonw,2019-09-10T16:59:25Z,2019-09-10T16:59:25Z,MEMBER,"By default in SQLite foreign key constraints are not enforced (you need to run `PRAGMA foreign_keys = ON;` to enforce them).

We will take advantage of this - even though the `following` table has foreign keys against user we will allow IDs to populate that table without a corresponding user record.

In the future we may add a command that can backfill missing user records.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",491791152,followers-ids and friends-ids subcommands,
https://github.com/dogsheep/github-to-sqlite/issues/17#issuecomment-597358364,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/17,597358364,MDEyOklzc3VlQ29tbWVudDU5NzM1ODM2NA==,9599,simonw,2020-03-10T22:50:20Z,2020-03-11T01:18:36Z,MEMBER,By default it will stop when it sees a commit that has already been stored. You will be able to over-ride that behaviour using `--all`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",578883725,Command for importing commits,
https://github.com/dogsheep/twitter-to-sqlite/issues/34#issuecomment-601871579,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34,601871579,MDEyOklzc3VlQ29tbWVudDYwMTg3MTU3OQ==,9599,simonw,2020-03-20T19:20:52Z,2020-03-20T19:20:52Z,MEMBER,Cause was the redefining of how `upsert()` works in sqlite-utils 2.x.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585266763,IndexError running user-timeline command,
https://github.com/dogsheep/github-to-sqlite/issues/32#issuecomment-617369247,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/32,617369247,MDEyOklzc3VlQ29tbWVudDYxNzM2OTI0Nw==,9599,simonw,2020-04-21T19:33:03Z,2020-04-21T19:33:03Z,MEMBER,Caused by #31.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",604222295,Issue comments don't appear to populate issues foreign key,
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686163754,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686163754,MDEyOklzc3VlQ29tbWVudDY4NjE2Mzc1NA==,9599,simonw,2020-09-03T00:46:21Z,2020-09-03T00:46:21Z,MEMBER,"Challenge: the `dogsheep-beta.yml` configuration file that is passed to the `dogsheep-beta index` command needs to also be made available to Datasette itself, so that it can read the configuration.

Let's say it can either be duplicated in the `plugins` configuration block of the `metadata.yml` OR you can do this in `metadata.yml`:

```yaml
plugins:
  dogsheep-beta:
    config_file: dogsheep-beta.yml
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711079760,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711079760,MDEyOklzc3VlQ29tbWVudDcxMTA3OTc2MA==,9599,simonw,2020-10-17T21:00:05Z,2020-10-17T21:00:05Z,MEMBER,Checking for either `<!DOCTYPE HealthData` or `<HealthData` in the first 1000 bytes should do it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723838331,export.xml file name varies with different language settings,
https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514500253,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7,514500253,MDEyOklzc3VlQ29tbWVudDUxNDUwMDI1Mw==,9599,simonw,2019-07-24T06:34:28Z,2019-07-24T06:34:28Z,MEMBER,"Clearing the root element each time saved even more:

<img width=""1128"" alt=""Screen Shot 2019-07-24 at 8 30 38 AM"" src=""https://user-images.githubusercontent.com/9599/61770555-d3932100-aded-11e9-8ffe-bebd682f94ed.png"">
","{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",472097220,Script uses a lot of RAM,
https://github.com/dogsheep/dogsheep-photos/issues/12#issuecomment-618796564,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/12,618796564,MDEyOklzc3VlQ29tbWVudDYxODc5NjU2NA==,9599,simonw,2020-04-24T04:35:25Z,2020-04-24T04:35:25Z,MEMBER,Code: https://github.com/dogsheep/photos-to-sqlite/blob/a388cf1f1b6b67752d669466cda8b171b6582171/photos_to_sqlite/cli.py#L109-L114,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",606033104,"If less than 500MB, show size in MB not GB",
https://github.com/dogsheep/twitter-to-sqlite/issues/31#issuecomment-552135263,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31,552135263,MDEyOklzc3VlQ29tbWVudDU1MjEzNTI2Mw==,9599,simonw,2019-11-09T20:38:35Z,2019-11-09T20:38:35Z,MEMBER,Command still needs documentation and a bit more testing.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520508502,"""friends"" command (similar to ""followers"")",
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623195197,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623195197,MDEyOklzc3VlQ29tbWVudDYyMzE5NTE5Nw==,9599,simonw,2020-05-03T22:44:33Z,2020-05-03T22:44:33Z,MEMBER,"Command will be this:

    $ photos-to-sqlite apple-photos photos.db

This will populate a `apple_photos` table with the data imported by the `osxphotos` library, plus the calculated sha256.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552129921,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29,552129921,MDEyOklzc3VlQ29tbWVudDU1MjEyOTkyMQ==,9599,simonw,2019-11-09T19:30:42Z,2019-11-09T19:30:42Z,MEMBER,"Confirmed, that seems to fix it:
```
(twitter-to-sqlite) ~/Dropbox/Development/twitter-to-sqlite $ twitter-to-sqlite import blah.db ~/Dropbox/dogsheep/twitter-2019-06-25-b31f246100821b551f2f9a23f21ac6fb565dab49dd23a35630cabbf2b94a1f03/account-suspension.js 
Traceback (most recent call last):
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/bin/twitter-to-sqlite"", line 11, in <module>
    load_entry_point('twitter-to-sqlite', 'console_scripts', 'twitter-to-sqlite')()
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 764, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 717, in main
    rv = self.invoke(ctx)
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/click/core.py"", line 555, in invoke
    return callback(*args, **kwargs)
  File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/cli.py"", line 633, in import_
    archive.import_from_file(db, path.name, open(path, ""rb"").read())
  File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/archive.py"", line 224, in import_from_file
    db[table_name].upsert_all(rows, hash_id=""pk"")
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py"", line 1094, in upsert_all
    extracts=extracts,
  File ""/Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages/sqlite_utils/db.py"", line 961, in insert_all
    first_record = next(records)
StopIteration
(twitter-to-sqlite) ~/Dropbox/Development/twitter-to-sqlite $ pip install -U sqlite-utils
Collecting sqlite-utils
  Using cached https://files.pythonhosted.org/packages/ee/a2/1b135010c7ac8e2d7545f659e9e6c6ede0f406f20b52e08d5817e1e31a9a/sqlite_utils-1.12.1-py3-none-any.whl
Requirement already satisfied, skipping upgrade: click in /Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages (from sqlite-utils) (7.0)
Requirement already satisfied, skipping upgrade: tabulate in /Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages (from sqlite-utils) (0.8.5)
Requirement already satisfied, skipping upgrade: click-default-group in /Users/simonw/.local/share/virtualenvs/twitter-to-sqlite-4ech4lJi/lib/python3.7/site-packages (from sqlite-utils) (1.2.2)
Installing collected packages: sqlite-utils
  Found existing installation: sqlite-utils 1.11
    Uninstalling sqlite-utils-1.11:
      Successfully uninstalled sqlite-utils-1.11
Successfully installed sqlite-utils-1.12.1
(twitter-to-sqlite) ~/Dropbox/Development/twitter-to-sqlite $ twitter-to-sqlite import blah.db ~/Dropbox/dogsheep/twitter-2019-06-25-b31f246100821b551f2f9a23f21ac6fb565dab49dd23a35630cabbf2b94a1f03/account-suspension.js 
(twitter-to-sqlite) ~/Dropbox/Development/twitter-to-sqlite $ 
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",518725064,`import` command fails on empty files,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790379629,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790379629,MDEyOklzc3VlQ29tbWVudDc5MDM3OTYyOQ==,9599,simonw,2021-03-04T07:14:41Z,2021-03-04T07:14:41Z,MEMBER,"Confirmed: removing the `len()` call does not speed things up, so it's reading through the entire file for some other purpose too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461259490,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2,1461259490,IC_kwDOJHON9s5XGQzi,9599,simonw,2023-03-09T04:24:27Z,2023-03-09T04:24:27Z,MEMBER,"Converting AppleScript date strings to ISO format is hard!

https://forum.latenightsw.com/t/formatting-dates/841 has a recipe I'll try:

    set todayISO to (todayDate as «class isot» as string)

Not clear to me how timezones work here. I'm going to ignore them for the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616354999,First working version,
https://github.com/dogsheep/twitter-to-sqlite/issues/34#issuecomment-601861908,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/34,601861908,MDEyOklzc3VlQ29tbWVudDYwMTg2MTkwOA==,9599,simonw,2020-03-20T18:56:44Z,2020-03-20T18:56:44Z,MEMBER,"Could this be a bug in `sqlite-utils`? This table has a primary key, so why is it running a query on `rowid = ?`?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585266763,IndexError running user-timeline command,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/7#issuecomment-1462682795,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7,1462682795,IC_kwDOJHON9s5XLsSr,9599,simonw,2023-03-09T19:52:20Z,2023-03-09T19:52:44Z,MEMBER,"Created through several rounds with ChatGPT (including hints like ""rewrite that using setdefault()""):
```python
def topological_sort(nodes):
    children = {}
    for node in nodes:
        parent_id = node[""parent""]
        if parent_id is not None:
            children.setdefault(parent_id, []).append(node)

    def traverse(node, result):
        result.append(node)
        if node[""id""] in children:
            for child in children[node[""id""]]:
                traverse(child, result)

    sorted_data = []

    for node in nodes:
        if node[""parent""] is None:
            traverse(node, sorted_data)

    return sorted_data
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1617769847,Folder support,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615935577,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615935577,MDEyOklzc3VlQ29tbWVudDYxNTkzNTU3Nw==,9599,simonw,2020-04-18T19:54:59Z,2020-04-18T19:55:30Z,MEMBER,"Creating IAM groups called `dogsheep-photos-simon-read-write` and `dogsheep-photos-simon-read`: https://console.aws.amazon.com/iam/home#/groups - I created them with no attached policies.

Now I can attach an ""inline policy"" to each one. For the read-write group I go here:

https://console.aws.amazon.com/iam/home#/groups/dogsheep-photos-simon-read-write

![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669703-2d086080-8172-11ea-9597-83e0b155193e.png)

Example policies are here: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html

For the read-write one I went with:
```json
{
    ""Version"": ""2012-10-17"",
    ""Statement"": [
        {
            ""Effect"": ""Allow"",
            ""Action"": ""s3:*"",
            ""Resource"": [
                ""arn:aws:s3:::dogsheep-photos-simon/*""
            ]
        }
    ]
}
```
For the read-only policy I'm going to guess that this is appropriate:

```json
{
    ""Version"": ""2012-10-17"",
    ""Statement"": [
        {
            ""Effect"": ""Allow"",
            ""Action"": [
                ""s3:GetObject*"",
                ""s3:ListBucket""
            ],
            ""Resource"": [
                ""arn:aws:s3:::dogsheep-photos-simon/*""
            ]
        }
    ]
}
```
I tried the policy simulator to test this out: https://policysim.aws.amazon.com/home/index.jsp?#groups/dogsheep-photos-simon-read - this worked:

![IAM_Policy_Simulator](https://user-images.githubusercontent.com/9599/79669893-cd12b980-8173-11ea-8dfb-5660ce3652da.png)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/github-to-sqlite/issues/19#issuecomment-602304531,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/19,602304531,MDEyOklzc3VlQ29tbWVudDYwMjMwNDUzMQ==,9599,simonw,2020-03-23T00:20:28Z,2020-03-23T00:20:28Z,MEMBER,"Current FTS code:
https://github.com/dogsheep/github-to-sqlite/blob/e35eec4343aa560c58c1634cc228d0d46c442304/github_to_sqlite/utils.py#L136-L143","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585850715,"Enable full-text search for more stuff (like commits, issues and issue_comments)",
https://github.com/dogsheep/dogsheep-photos/issues/14#issuecomment-620772190,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14,620772190,MDEyOklzc3VlQ29tbWVudDYyMDc3MjE5MA==,9599,simonw,2020-04-28T18:14:43Z,2020-04-28T18:14:43Z,MEMBER,"Database schema for this will require some thought. Just dumping the output into a JSON column isn't going to be flexible enough - I want to be able to FTS against labels and OCR text, and potentially query against other characteristics too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",608512747,Annotate photos using the Google Cloud Vision API,
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623007441,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623007441,MDEyOklzc3VlQ29tbWVudDYyMzAwNzQ0MQ==,9599,simonw,2020-05-02T20:13:37Z,2020-05-02T20:13:37Z,MEMBER,Datasette cumulative stars over time: https://github-to-sqlite.dogsheep.net/github?sql=select%0D%0A++yyyymmdd%2C%0D%0A++sum%28n%29+over+%28%0D%0A++++order+by%0D%0A++++++yyyymmdd+rows+unbounded+preceding%0D%0A++%29+as+cumulative_count%0D%0Afrom%0D%0A++%28%0D%0A++++select%0D%0A++++++substr%28starred_at%2C+0%2C+11%29+as+yyyymmdd%2C%0D%0A++++++count%28*%29+as+n%0D%0A++++from%0D%0A++++++stars%0D%0A++++where+repo+%3D+107914493%0D%0A++++group+by%0D%0A++++++yyyymmdd%0D%0A++%29,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695875274,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695875274,MDEyOklzc3VlQ29tbWVudDY5NTg3NTI3NA==,9599,simonw,2020-09-21T02:28:58Z,2020-09-21T02:28:58Z,MEMBER,Datasette's implementation is complex because it has to support compound primary keys: https://github.com/simonw/datasette/blob/a258339a935d8d29a95940ef1db01e98bb85ae63/datasette/utils/__init__.py#L88-L114 - but that's not something that's needed for dogsheep-beta.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination,
https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685970384,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7,685970384,MDEyOklzc3VlQ29tbWVudDY4NTk3MDM4NA==,9599,simonw,2020-09-02T20:11:41Z,2020-09-02T20:11:59Z,MEMBER,"Default categories:

- 1 = created
- 2 = saved","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691265198,"Mechanism for differentiating between ""by me"" and ""liked by me""",
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-711569063,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,711569063,MDEyOklzc3VlQ29tbWVudDcxMTU2OTA2Mw==,9599,simonw,2020-10-19T05:01:29Z,2020-10-19T05:01:29Z,MEMBER,"Demo of `--accept`:

    github-to-sqlite get /repos/simonw/datasette/readme --accept 'application/vnd.github.VERSION.html'
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls,
https://github.com/dogsheep/github-to-sqlite/issues/30#issuecomment-616883726,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/30,616883726,MDEyOklzc3VlQ29tbWVudDYxNjg4MzcyNg==,9599,simonw,2020-04-21T00:45:23Z,2020-04-21T00:45:23Z,MEMBER,"Demo of fix: https://github-to-sqlite.dogsheep.net/github/issues?assignee__notblank=1&milestone__notblank=1

![github__issues__4_rows_where_where_assignee_is_not_blank_and_milestone_is_not_blank_sorted_by_updated_at_descending](https://user-images.githubusercontent.com/9599/79812758-b49ace80-832e-11ea-81db-bdf993b872cc.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",603618244,Issues milestone column is the wrong type,
https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735465708,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54,735465708,MDEyOklzc3VlQ29tbWVudDczNTQ2NTcwOA==,9599,simonw,2020-11-29T22:08:46Z,2020-11-29T22:08:46Z,MEMBER,"Demo:
- https://github-to-sqlite.dogsheep.net/github/steps?_facet=repo
- https://github-to-sqlite.dogsheep.net/github/workflows
- https://github-to-sqlite.dogsheep.net/github/jobs","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753026003,github-to-sqlite workflows command,
https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615886206,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28,615886206,MDEyOklzc3VlQ29tbWVudDYxNTg4NjIwNg==,9599,simonw,2020-04-18T15:04:59Z,2020-04-18T15:04:59Z,MEMBER,"Demo: https://github-to-sqlite.dogsheep.net/github/contributors

Documentation: https://github.com/dogsheep/github-to-sqlite/blob/13f8868fb5efa01c263b24f6dd91c617e6e938e1/README.md#fetching-contributors-to-a-repository ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601333634,Pull repository contributors,
https://github.com/dogsheep/github-to-sqlite/issues/36#issuecomment-622982346,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/36,622982346,MDEyOklzc3VlQ29tbWVudDYyMjk4MjM0Ng==,9599,simonw,2020-05-02T16:50:31Z,2020-05-02T16:50:31Z,MEMBER,Demo: https://github-to-sqlite.dogsheep.net/github/dependent_repos,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610842926,Add view for better display of dependent repos,
https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554299,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660554299,MDEyOklzc3VlQ29tbWVudDY2MDU1NDI5OQ==,9599,simonw,2020-07-18T22:58:24Z,2020-07-18T23:02:52Z,MEMBER,"Deploying the fixed version like this:
```
$ gcloud config set run/region us-central1
$ gcloud config set project datasette-222320
$ datasette publish cloudrun /tmp/github.db \
            -m demo-metadata.json \
            --service github-to-sqlite \
            --install=py-gfm \
            --install='datasette-search-all>=0.3' \
            --install='datasette-render-markdown>=1.1.2' \
            --install=datasette-pretty-json \
            --install=datasette-json-html \
            --install=datasette-vega
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601,Fix the demo - it breaks because of the tags table change,
https://github.com/dogsheep/github-to-sqlite/issues/7#issuecomment-541721437,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/7,541721437,MDEyOklzc3VlQ29tbWVudDU0MTcyMTQzNw==,9599,simonw,2019-10-14T14:44:12Z,2019-10-14T14:44:12Z,MEMBER,Docs: https://github.com/dogsheep/github-to-sqlite/blob/0.5/README.md#retrieving-issue-comments-for-a-repository,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506276893,issue-comments command for importing issue comments,
https://github.com/dogsheep/dogsheep-beta/issues/2#issuecomment-685121074,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/2,685121074,MDEyOklzc3VlQ29tbWVudDY4NTEyMTA3NA==,9599,simonw,2020-09-01T20:42:00Z,2020-09-01T20:42:00Z,MEMBER,Documentation at the bottom of the Usage section here: https://github.com/dogsheep/dogsheep-beta/blob/0.2/README.md#usage,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",689809225,Apply porter stemming,
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686767208,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686767208,MDEyOklzc3VlQ29tbWVudDY4Njc2NzIwOA==,9599,simonw,2020-09-03T21:12:14Z,2020-09-03T21:12:14Z,MEMBER,Documentation: https://github.com/dogsheep/dogsheep-beta/blob/0.4/README.md#custom-results-display,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693788032,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693788032,MDEyOklzc3VlQ29tbWVudDY5Mzc4ODAzMg==,9599,simonw,2020-09-17T03:35:22Z,2020-09-17T03:35:22Z,MEMBER,Documentation: https://github.com/dogsheep/github-to-sqlite/blob/b02bf135485c0a7a3768868967f45a6b5e515289/README.md#making-authenticated-api-calls,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls,
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622162835,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622162835,MDEyOklzc3VlQ29tbWVudDYyMjE2MjgzNQ==,9599,simonw,2020-04-30T22:59:26Z,2020-04-30T22:59:26Z,MEMBER,Documentation: https://github.com/dogsheep/github-to-sqlite/blob/c9f48404481882e8b3af06f35e4801a80ac79ed6/README.md#scraping-dependents-for-a-repository,"{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,Command for retrieving dependents for a repo,
https://github.com/dogsheep/dogsheep-photos/issues/23#issuecomment-631120771,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/23,631120771,MDEyOklzc3VlQ29tbWVudDYzMTEyMDc3MQ==,9599,simonw,2020-05-19T22:32:48Z,2020-05-19T22:32:48Z,MEMBER,Documentation: https://github.com/dogsheep/photos-to-sqlite/blob/e2fab012551eed05278040b5d57e7373a1b9a0bf/README.md#creating-a-subset-database,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621280529,create-subset command for creating a publishable subset of a photos database,
https://github.com/dogsheep/twitter-to-sqlite/issues/13#issuecomment-538804815,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/13,538804815,MDEyOklzc3VlQ29tbWVudDUzODgwNDgxNQ==,9599,simonw,2019-10-07T00:33:49Z,2019-10-07T00:33:49Z,MEMBER,Documentation: https://github.com/dogsheep/twitter-to-sqlite#retrieve-tweets-in-bulk,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503085013,statuses-lookup command,
https://github.com/dogsheep/dogsheep-beta/issues/25#issuecomment-695109140,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25,695109140,MDEyOklzc3VlQ29tbWVudDY5NTEwOTE0MA==,9599,simonw,2020-09-18T22:12:20Z,2020-09-18T22:12:20Z,MEMBER,Documented here: https://github.com/dogsheep/dogsheep-beta/blob/534fc9689227eba70e69a45da0cee5820bbda9e1/README.md#datasette-plugin,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",704685890,template_debug mechanism,
https://github.com/dogsheep/github-to-sqlite/issues/33#issuecomment-622997410,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/33,622997410,MDEyOklzc3VlQ29tbWVudDYyMjk5NzQxMA==,9599,simonw,2020-05-02T18:46:10Z,2020-05-02T18:46:10Z,MEMBER,Documented here: https://github.com/dogsheep/github-to-sqlite/blob/10fb34de41aaa35681f08b5991540d65bfcf2e2e/README.md#authentication,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",609950090,Fall back to authentication via ENV,
https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-542882604,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3,542882604,MDEyOklzc3VlQ29tbWVudDU0Mjg4MjYwNA==,9599,simonw,2019-10-16T20:41:23Z,2019-10-16T20:41:23Z,MEMBER,Documented here: https://github.com/dogsheep/swarm-to-sqlite/blob/0.2/README.md#usage,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",487600595,Option to fetch only checkins more recent than the current max checkin,
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549228535,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3,549228535,MDEyOklzc3VlQ29tbWVudDU0OTIyODUzNQ==,9599,simonw,2019-11-04T05:31:55Z,2019-11-04T05:31:55Z,MEMBER,Documented here: https://github.com/dogsheep/twitter-to-sqlite/blob/801c0c2daf17d8abce9dcb5d8d610410e7e25dbe/README.md#running-searches,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833975,Command for running a search and saving tweets for that search,
https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541141169,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18,541141169,MDEyOklzc3VlQ29tbWVudDU0MTE0MTE2OQ==,9599,simonw,2019-10-11T16:51:29Z,2019-10-11T16:51:29Z,MEMBER,Documented here: https://github.com/dogsheep/twitter-to-sqlite/blob/master/README.md#retrieving-tweets-from-your-home-timeline,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",505928530,Command to import home-timeline,
https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-662630868,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48,662630868,MDEyOklzc3VlQ29tbWVudDY2MjYzMDg2OA==,9599,simonw,2020-07-22T19:03:02Z,2020-07-22T19:03:02Z,MEMBER,Done!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",663976976,Add a table of contents to the README,
https://github.com/dogsheep/github-to-sqlite/issues/15#issuecomment-602312054,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15,602312054,MDEyOklzc3VlQ29tbWVudDYwMjMxMjA1NA==,9599,simonw,2020-03-23T01:07:10Z,2020-03-23T01:07:10Z,MEMBER,"Each asset looks like this:
```json
    {
        ""url"": ""https://api.github.com/repos/instrumenta/conftest/releases/assets/11811946"",
        ""id"": 11811946,
        ""node_id"": ""MDEyOlJlbGVhc2VBc3NldDExODExOTQ2"",
        ""name"": ""checksums.txt"",
        ""label"": """",
        ""uploader"": {
            ""login"": ""garethr"",
            ""id"": 2029,
            ""node_id"": ""MDQ6VXNlcjIwMjk="",
            ""avatar_url"": ""https://avatars2.githubusercontent.com/u/2029?v=4"",
            ""gravatar_id"": """",
            ""url"": ""https://api.github.com/users/garethr"",
            ""html_url"": ""https://github.com/garethr"",
            ""followers_url"": ""https://api.github.com/users/garethr/followers"",
            ""following_url"": ""https://api.github.com/users/garethr/following{/other_user}"",
            ""gists_url"": ""https://api.github.com/users/garethr/gists{/gist_id}"",
            ""starred_url"": ""https://api.github.com/users/garethr/starred{/owner}{/repo}"",
            ""subscriptions_url"": ""https://api.github.com/users/garethr/subscriptions"",
            ""organizations_url"": ""https://api.github.com/users/garethr/orgs"",
            ""repos_url"": ""https://api.github.com/users/garethr/repos"",
            ""events_url"": ""https://api.github.com/users/garethr/events{/privacy}"",
            ""received_events_url"": ""https://api.github.com/users/garethr/received_events"",
            ""type"": ""User"",
            ""site_admin"": false
        },
        ""content_type"": ""text/plain; charset=utf-8"",
        ""state"": ""uploaded"",
        ""size"": 600,
        ""download_count"": 2,
        ""created_at"": ""2019-03-30T16:56:44Z"",
        ""updated_at"": ""2019-03-30T16:56:44Z"",
        ""browser_download_url"": ""https://github.com/instrumenta/conftest/releases/download/v0.1.0/checksums.txt""
    }
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",544571092,Assets table with downloads,
https://github.com/dogsheep/github-to-sqlite/issues/35#issuecomment-622982667,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35,622982667,MDEyOklzc3VlQ29tbWVudDYyMjk4MjY2Nw==,9599,simonw,2020-05-02T16:52:53Z,2020-05-02T16:52:53Z,MEMBER,Easiest option: use `db.index_foreign_keys()`: https://sqlite-utils.readthedocs.io/en/stable/python-api.html#adding-indexes-for-all-foreign-keys,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610511450,Create index on issue_comments(user) and other foreign keys,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623857417,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623857417,MDEyOklzc3VlQ29tbWVudDYyMzg1NzQxNw==,9599,simonw,2020-05-05T05:01:47Z,2020-05-05T05:01:47Z,MEMBER,"Even that didn't work - it didn't copy across the rowid values. I'm pretty sure that's what's wrong here:
```
sqlite3 /Users/simon/Pictures/Photos\ Library.photoslibrary/database/search/psi.sqlite 'select rowid, uuid_0, uuid_1 from assets limit 10'                                              
1619605|-9205353363298198838|4814875488794983828
1641378|-9205348195631362269|390804289838822030
1634974|-9205331524553603243|-3834026796261633148
1619083|-9205326176986145401|7563404215614709654
22131|-9205315724827218763|8370531509591906734
1645633|-9205247376092758131|-1311540150497601346
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-622998813,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12,622998813,MDEyOklzc3VlQ29tbWVudDYyMjk5ODgxMw==,9599,simonw,2020-05-02T18:58:17Z,2020-05-02T18:58:17Z,MEMBER,Faceting works now: https://github-to-sqlite.dogsheep.net/github/recent_releases?_facet_array=topics&topics__arraycontains=datasette-io&topics__arraycontains=sqlite&_facet=repo#facet-repo,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520756546,Add this view for seeing new releases,
https://github.com/dogsheep/dogsheep-beta/issues/3#issuecomment-686689612,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3,686689612,MDEyOklzc3VlQ29tbWVudDY4NjY4OTYxMg==,9599,simonw,2020-09-03T18:44:20Z,2020-09-03T18:44:20Z,MEMBER,Facets are now displayed but selecting them doesn't work yet.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",689810340,"Datasette plugin to provide custom page for running faceted, ranked searches",
https://github.com/dogsheep/github-to-sqlite/issues/69#issuecomment-985928838,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69,985928838,IC_kwDODFdgUs46xBSG,9599,simonw,2021-12-04T00:34:52Z,2021-12-04T00:34:52Z,MEMBER,"First attempt at this:
```sql
select 'issues' as ""table"", id, node_id, title, user, created_at, body, repo
from issues
union all
select 'issue_comments' as ""table"", issue_comments.id, issue_comments.node_id, '' as title, issue_comments.user, issue_comments.created_at, issue_comments.body, issues.repo
from issue_comments join issues on issues.id = issue_comments.issue
order by created_at desc
```
https://github-to-sqlite.dogsheep.net/github?sql=select+%27issues%27+as+%22table%22%2C+id%2C+node_id%2C+title%2C+user%2C+created_at%2C+body%2C+repo%0D%0Afrom+issues%0D%0Aunion+all%0D%0Aselect+%27issue_comments%27+as+%22table%22%2C+issue_comments.id%2C+issue_comments.node_id%2C+%27%27+as+title%2C+issue_comments.user%2C+issue_comments.created_at%2C+issue_comments.body%2C+issues.repo%0D%0Afrom+issue_comments+join+issues+on+issues.id+%3D+issue_comments.issue%0D%0Aorder+by+created_at+desc","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1071071397,View that combines issues and issue comments,
https://github.com/dogsheep/github-to-sqlite/issues/6#issuecomment-541387941,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/6,541387941,MDEyOklzc3VlQ29tbWVudDU0MTM4Nzk0MQ==,9599,simonw,2019-10-13T05:30:19Z,2019-10-13T05:30:19Z,MEMBER,Fix released in 0.5: https://github.com/dogsheep/github-to-sqlite/releases/tag/0.5,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",504238461,sqlite3.OperationalError: table users has no column named bio,
https://github.com/dogsheep/github-to-sqlite/issues/10#issuecomment-622980203,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10,622980203,MDEyOklzc3VlQ29tbWVudDYyMjk4MDIwMw==,9599,simonw,2020-05-02T16:34:29Z,2020-05-02T16:34:29Z,MEMBER,"Fixed definition:
```sql
select
  stars.starred_at,
  starring_user.login as starred_by,
  repos.*
from
  repos
  join stars on repos.id = stars.repo
  join users as starring_user on stars.user = starring_user.id
  join users on repos.owner = users.id
order by
  starred_at desc;
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516967682,Add this repos_starred view,
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602862967,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602862967,MDEyOklzc3VlQ29tbWVudDYwMjg2Mjk2Nw==,9599,simonw,2020-03-23T21:22:04Z,2020-03-23T21:22:04Z,MEMBER,Following these instructions: https://simonwillison.net/2020/Jan/21/github-actions-cloud-run/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-605316146,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1,605316146,MDEyOklzc3VlQ29tbWVudDYwNTMxNjE0Ng==,9599,simonw,2020-03-27T21:09:15Z,2020-03-27T21:09:22Z,MEMBER,"For a progress bar I need to know how many total items there are.

I found an undocumented API for this! `/v3/stats` which returns:
```json
{
    ""count_list"": 7394,
    ""count_read"": 1016,
    ""count_unread"": 6378,
    ""status"": 1
}
```
I guessed this based on the documented v2 API: https://getpocket.com/api/v2_docs/#stats","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503233021,Use better pagination (and implement progress bar),
https://github.com/dogsheep/dogsheep-photos/issues/14#issuecomment-620771698,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14,620771698,MDEyOklzc3VlQ29tbWVudDYyMDc3MTY5OA==,9599,simonw,2020-04-28T18:13:48Z,2020-04-28T18:13:48Z,MEMBER,"For face detection:
```
    {""type"": vision.enums.Feature.Type.Type.FACE_DETECTION}
```
For OCR:
```
    {""type"": vision.enums.Feature.Type.DOCUMENT_TEXT_DETECTION}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",608512747,Annotate photos using the Google Cloud Vision API,
https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-541388038,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20,541388038,MDEyOklzc3VlQ29tbWVudDU0MTM4ODAzOA==,9599,simonw,2019-10-13T05:31:58Z,2019-10-13T05:31:58Z,MEMBER,For favourites a `--stop_after=200` option is probably good enough.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506268945,--since support for various commands for refresh-by-cron,
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623198653,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623198653,MDEyOklzc3VlQ29tbWVudDYyMzE5ODY1Mw==,9599,simonw,2020-05-03T23:09:57Z,2020-05-03T23:09:57Z,MEMBER,"For locations: I'll add `place_x` columns for all of these:
```
(Pdb) photo.place.address._asdict()
{'street': None, 'sub_locality': None, 'city': 'Loreto', 'sub_administrative_area': 'Loreto', 'state_province': 'BCS', 'postal_code': None, 'country': 'Mexico', 'iso_country_code': 'MX'}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602811468,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,602811468,MDEyOklzc3VlQ29tbWVudDYwMjgxMTQ2OA==,9599,simonw,2020-03-23T19:33:14Z,2020-03-23T19:33:14Z,MEMBER,Found one: https://api.github.com/repos/simonw/simonw.github.com/commits,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/dogsheep-photos/issues/17#issuecomment-624278714,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17,624278714,MDEyOklzc3VlQ29tbWVudDYyNDI3ODcxNA==,9599,simonw,2020-05-05T20:07:19Z,2020-05-05T20:07:19Z,MEMBER,"From https://hynek.me/articles/conditional-python-dependencies/ I think this will look like:
```python
setup(
    # ...
    install_requires=[
        # ...
        ""osxphotos>=0.28.13 ; sys_platform=='darwin'"",
    ]
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612860531,Only install osxphotos if running on macOS,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/7#issuecomment-1462562735,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7,1462562735,IC_kwDOJHON9s5XLO-v,9599,simonw,2023-03-09T18:23:56Z,2023-03-09T18:25:22Z,MEMBER,"From the Script Editor library docs:

A note has a:

> - `container` (folder), r/o) : the folder of the note

Here's what a folder looks like:

> folder n : a folder containing notes
> elements:
>
> - contains folders, notes; contained by application, accounts, folders.
>
> properties:
>
> - `name` (text) : the name of the folder
> - `id` (text, r/o) : the unique identifier of the folder
> - `shared` (boolean, r/o) : Is the folder shared?
> - `container` (account or folder, r/o) : the container of the folder
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1617769847,Folder support,
https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-663143160,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48,663143160,MDEyOklzc3VlQ29tbWVudDY2MzE0MzE2MA==,9599,simonw,2020-07-23T17:46:07Z,2020-07-23T17:46:07Z,MEMBER,"Frustratingly, these links don't work on PyPI: https://pypi.org/project/twitter-to-sqlite/

There's an issue about that here: https://github.com/pypa/readme_renderer/issues/169","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",663976976,Add a table of contents to the README,
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693788387,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693788387,MDEyOklzc3VlQ29tbWVudDY5Mzc4ODM4Nw==,9599,simonw,2020-09-17T03:36:47Z,2020-09-17T03:36:58Z,MEMBER,"Fun demo of the `--nl` option:

    github-to-sqlite get /users/simonw/repos --paginate --nl | sqlite-utils insert simonw.db repos - --nl
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls,
https://github.com/dogsheep/dogsheep-beta/issues/16#issuecomment-695877627,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16,695877627,MDEyOklzc3VlQ29tbWVudDY5NTg3NzYyNw==,9599,simonw,2020-09-21T02:42:29Z,2020-09-21T02:42:29Z,MEMBER,"Fun twist: assuming `timestamp` is always stored as UTC, I need the interface to be timezone aware so I can see e.g. everything from 4th July 2020 in the San Francisco timezone definition of 4th July 2020.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694493566,Timeline view,
https://github.com/dogsheep/github-to-sqlite/issues/53#issuecomment-735485677,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/53,735485677,MDEyOklzc3VlQ29tbWVudDczNTQ4NTY3Nw==,9599,simonw,2020-11-30T00:36:09Z,2020-11-30T00:36:09Z,MEMBER,Given rate limits (see #51) this command might be better implemented by running a `git clone` into a temporary directory - doing so would retrieve all of the files in one go.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753000405,Command for fetching file contents,
https://github.com/dogsheep/twitter-to-sqlite/issues/30#issuecomment-552131798,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/30,552131798,MDEyOklzc3VlQ29tbWVudDU1MjEzMTc5OA==,9599,simonw,2019-11-09T19:54:45Z,2019-11-09T19:54:45Z,MEMBER,Good catch - not sure how that bug crept in. Removing line 116 looks like the right fix to me.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",518739697,`followers` fails because `transform_user` is called twice,
https://github.com/dogsheep/pocket-to-sqlite/issues/5#issuecomment-684425714,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/5,684425714,MDEyOklzc3VlQ29tbWVudDY4NDQyNTcxNA==,9599,simonw,2020-09-01T06:18:32Z,2020-09-01T06:18:32Z,MEMBER,"Good suggestion, I'll setup a demo somewhere.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",629473827,Set up a demo,
https://github.com/dogsheep/hacker-news-to-sqlite/issues/3#issuecomment-886241674,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3,886241674,IC_kwDODtX3eM400vmK,9599,simonw,2021-07-25T18:41:17Z,2021-07-25T18:41:17Z,MEMBER,Got a TIL out of this: https://til.simonwillison.net/jq/extracting-objects-recursively,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",952189173,Use HN algolia endpoint to retrieve trees,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615957385,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615957385,MDEyOklzc3VlQ29tbWVudDYxNTk1NzM4NQ==,9599,simonw,2020-04-18T21:56:16Z,2020-04-18T21:58:11Z,MEMBER,Got this working! I'll do EXIF in a separate ticket #3.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623010272,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623010272,MDEyOklzc3VlQ29tbWVudDYyMzAxMDI3Mg==,9599,simonw,2020-05-02T20:39:14Z,2020-05-02T20:39:14Z,MEMBER,"Graph of cumulative stars for Datasette over time: https://github-to-sqlite.dogsheep.net/github?sql=select%0D%0A++yyyymmdd%2C%0D%0A++sum%28n%29+over+%28%0D%0A++++order+by%0D%0A++++++yyyymmdd+rows+unbounded+preceding%0D%0A++%29+as+cumulative_count%0D%0Afrom%0D%0A++%28%0D%0A++++select%0D%0A++++++substr%28starred_at%2C+0%2C+11%29+as+yyyymmdd%2C%0D%0A++++++count%28*%29+as+n%0D%0A++++from%0D%0A++++++stars%0D%0A++++where+repo+%3D+107914493%0D%0A++++group+by%0D%0A++++++yyyymmdd%0D%0A++%29#g.mark=line&g.x_column=yyyymmdd&g.x_type=temporal&g.y_column=cumulative_count&g.y_type=quantitative

<img width=""291"" alt=""github__select_yyyymmdd__sum_n__over___order_by_yyyymmdd_rows_unbounded_preceding___as_cumulative_count_from___select_substr_starred_at__0__11__as_yyyymmdd__count____as_n_from_stars_where_repo___107914493_group_by_yyyymmdd__"" src=""https://user-images.githubusercontent.com/9599/80891642-31587180-8c7a-11ea-9169-60eb3cb6d3e0.png"">

Stars per day (as a label bar chart, so very wide):

https://github-to-sqlite.dogsheep.net/github?sql=%0D%0A++++select%0D%0A++++++substr%28starred_at%2C+0%2C+11%29+as+yyyymmdd%2C%0D%0A++++++count%28*%29+as+n%0D%0A++++from%0D%0A++++++stars%0D%0A++++where+repo+%3D+107914493%0D%0A++++group+by%0D%0A++++++yyyymmdd%0D%0A++#g.mark=bar&g.x_column=yyyymmdd&g.x_type=ordinal&g.y_column=n&g.y_type=quantitative

<img width=""1424"" alt=""github__select_substr_starred_at__0__11__as_yyyymmdd__count____as_n_from_stars_where_repo___107914493_group_by_yyyymmdd"" src=""https://user-images.githubusercontent.com/9599/80891656-4e8d4000-8c7a-11ea-9ff2-02c9cf4cb5d5.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/pocket-to-sqlite/issues/11#issuecomment-1221621466,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11,1221621466,IC_kwDODLZ_YM5I0Hba,9599,simonw,2022-08-21T21:09:47Z,2022-08-21T21:09:47Z,MEMBER,"Great catch, thanks.

I'm going to use it to mean `--auth` - since other tools in the Dogsheep family have the same convention.

 `--all` will be the only way to specify all.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1345452427,"-a option is used for ""--auth"" and for ""--all""",
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-541748580,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10,541748580,MDEyOklzc3VlQ29tbWVudDU0MTc0ODU4MA==,9599,simonw,2019-10-14T15:30:44Z,2019-10-14T15:30:44Z,MEMBER,Had several recommendations for https://github.com/tqdm/tqdm which is what goodreads-to-sqlite uses.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",492297930,Rethink progress bars for various commands,
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861041597,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861041597,MDEyOklzc3VlQ29tbWVudDg2MTA0MTU5Nw==,9599,simonw,2021-06-14T22:44:54Z,2021-06-14T22:44:54Z,MEMBER,Have you found a way to access events in GraphQL? I can only see way to access a timeline of events for a single issue or a single pull request. See also https://github.community/t/get-event-equivalent-for-v4/13600/2,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216,"feature: support ""events""",
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-550388354,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,550388354,MDEyOklzc3VlQ29tbWVudDU1MDM4ODM1NA==,9599,simonw,2019-11-06T16:26:55Z,2019-11-06T16:26:55Z,MEMBER,"Here's a query I figured out using a window function that shows cumulative stargazers over time:
```sql
select
  yyyymmdd,
  sum(n) over (
    order by
      yyyymmdd rows unbounded preceding
  ) as cumulative_count
from
  (
    select
      substr(starred_at, 0, 11) as yyyymmdd,
      count(*) as n
    from
      stars
    group by
      yyyymmdd
  )
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/11#issuecomment-1462962682,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/11,1462962682,IC_kwDOJHON9s5XMwn6,9599,simonw,2023-03-09T23:20:35Z,2023-03-09T23:22:41Z,MEMBER,"Here's a query that returns all notes in folder 1, including notes in descendant folders:
```sql
with recursive nested_folders(folder_id, descendant_folder_id) as (
  -- base case: select all immediate children of the root folder
  select id, id from folders where parent is null
  union all
  -- recursive case: select all children of the previous level of nested folders
  select nf.folder_id, f.id from nested_folders nf
  join folders f on nf.descendant_folder_id = f.parent
)
-- Find notes within all descendants of folder 1
select *
from notes
where folder in (
  select descendant_folder_id from nested_folders where folder_id = 1
);
```
With assistance from ChatGPT. Prompts were:

```
SQLite schema:

CREATE TABLE [folders] (
   [id] INTEGER PRIMARY KEY,
   [long_id] TEXT,
   [name] TEXT,
   [parent] INTEGER,
   FOREIGN KEY([parent]) REFERENCES [folders]([id])
);

Write a recursive CTE that returns the following:

folder_id | descendant_folder_id

With a row for every nested child of every folder - so the top level folder has lots of rows
```
Then I tweaked it a bit, then ran this:
```
WITH RECURSIVE nested_folders(folder_id, descendant_folder_id) AS (
  -- base case: select all immediate children of the root folder
  SELECT id, id FROM folders WHERE parent IS NULL
  UNION ALL
  -- recursive case: select all children of the previous level of nested folders
  SELECT nf.folder_id, f.id FROM nested_folders nf
  JOIN folders f ON nf.descendant_folder_id = f.parent
)
-- select all rows from the recursive CTE
SELECT * from notes where folder in (select descendant_folder_id FROM nested_folders where folder_id = 1)

Convert all SQL keywords to lower case, and re-indent
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1618130434,Implement a SQL view to make it easier to query files in a nested folder,
https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188223933,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188223933,IC_kwDODFdgUs5G0tu9,9599,simonw,2022-07-18T19:40:50Z,2022-07-18T19:42:41Z,MEMBER,"Here's how the demo is deployed: https://github.com/dogsheep/github-to-sqlite/blob/dbac2e5dd8a562b45d8255a265859cf8020ca22a/.github/workflows/deploy-demo.yml#L103-L119

I'm suspicious of `py-gfm`, which is used like this:

https://github.com/dogsheep/github-to-sqlite/blob/dbac2e5dd8a562b45d8255a265859cf8020ca22a/demo-metadata.json#L49-L51","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063,500 error in github-to-sqlite demo,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623811131,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623811131,MDEyOklzc3VlQ29tbWVudDYyMzgxMTEzMQ==,9599,simonw,2020-05-05T03:16:18Z,2020-05-05T03:16:18Z,MEMBER,"Here's how to convert two integers unto a UUID using Java. Not sure if it's the solution I need though (or how to do the same thing in Python):

https://repl.it/repls/EuphoricSomberClasslibrary

<img width=""1494"" alt=""Repl_it_-_EuphoricSomberClasslibrary"" src=""https://user-images.githubusercontent.com/9599/81032267-0d488c00-8e44-11ea-9be7-680eaccd1611.png"">

```java
import java.util.UUID;

class Main {
  public static void main(String[] args) {
    java.util.UUID uuid = new java.util.UUID(
      2544182952487526660L,
      -3640314103732024685L
    );
    System.out.println(
      uuid
    );
  }
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706834800,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706834800,MDEyOklzc3VlQ29tbWVudDcwNjgzNDgwMA==,9599,simonw,2020-10-12T03:24:57Z,2020-10-16T20:16:28Z,MEMBER,"Here's my first attempt at a plugin for this:
```python
from datasette import hookimpl
import jinja2

START = ""<en-note""
END = ""</en-note>""
TEMPLATE = """"""
<div style=""max-width: 500px; white-space: normal; overflow-wrap: break-word;"">{}</div>
"""""".strip()

EN_MEDIA_SCRIPT = """"""
Array.from(document.querySelectorAll('en-media')).forEach(el => {
    let hash = el.getAttribute('hash');
    let type = el.getAttribute('type');
    let path = `/evernote/resources_data/${hash}.json?_shape=array`;
    fetch(path).then(r => r.json()).then(rows => {
        let b64 = rows[0].data.encoded;
        let data = `data:${type};base64,${b64}`;
        el.innerHTML = `<img style=""max-width: 300px"" src=""${data}"">`;
    });
});
""""""


@hookimpl
def render_cell(value, table):
    if not table:
        # Don't render content from arbitrary SQL queries, could be XSS hole
        return
    if not value or not isinstance(value, str):
        return
    value = value.strip()
    if value.startswith(START) and value.endswith(END):
        trimmed = value[len(START) : -len(END)]
        trimmed = trimmed.split("">"", 1)[1]
        # Replace those horrible double newlines
        trimmed = trimmed.replace(""<div><br /></div>"", ""<br>"")
        return jinja2.Markup(TEMPLATE.format(trimmed))


@hookimpl
def extra_body_script():
    return EN_MEDIA_SCRIPT
```
It works!

It does however demonstrate that Evernote's ""clip this webpage"" feature means there is a LOT of weird HTML that can get into a note. It looks like they've filtered out the scripts but I wouldn't bet on it - they certainly don't filter out many of the inline styles. So running Bleach is almost certainly a good idea.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from <en-media> tags inline in Datasette,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408738,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,624408738,MDEyOklzc3VlQ29tbWVudDYyNDQwODczOA==,9599,simonw,2020-05-06T02:21:05Z,2020-05-06T02:21:32Z,MEMBER,"Here's rendering code from my hacked-together not-yet-released S3 image proxy:
```python
from starlette.responses import Response
from PIL import Image, ExifTags
import pyheif

for ORIENTATION_TAG in ExifTags.TAGS.keys():
    if ExifTags.TAGS[ORIENTATION_TAG] == ""Orientation"":
        break
    ...
    # Load it into Pillow
    if ext == ""heic"":
        heic = pyheif.read_heif(image_response.content)
        image = Image.frombytes(mode=heic.mode, size=heic.size, data=heic.data)
    else:
        image = Image.open(io.BytesIO(image_response.content))

    # Does EXIF tell us to rotate it?
    try:
        exif = dict(image._getexif().items())
        if exif[ORIENTATION_TAG] == 3:
            image = image.rotate(180, expand=True)
        elif exif[ORIENTATION_TAG] == 6:
            image = image.rotate(270, expand=True)
        elif exif[ORIENTATION_TAG] == 8:
            image = image.rotate(90, expand=True)
    except (AttributeError, KeyError, IndexError):
        pass

    # Resize based on ?w= and ?h=, if set
    width, height = image.size
    w = request.query_params.get(""w"")
    h = request.query_params.get(""h"")
    if w is not None or h is not None:
        if h is None:
            # Set h based on w
            w = int(w)
            h = int((float(height) / width) * w)
        elif w is None:
            h = int(h)
            # Set w based on h
            w = int((float(width) / height) * h)
        w = int(w)
        h = int(h)
        image.thumbnail((w, h))

    # ?bw= converts to black and white
    if request.query_params.get(""bw""):
        image = image.convert(""L"")

    # ?q= sets the quality - defaults to 75
    quality = 75
    q = request.query_params.get(""q"")
    if q and q.isdigit() and 1 <= int(q) <= 100:
        quality = int(q)

    # Output as JPEG or PNG
    output_image = io.BytesIO()
    image_type = ""JPEG""
    kwargs = {""quality"": quality}
    if image.format == ""PNG"":
        image_type = ""PNG""
        kwargs = {}

    image.save(output_image, image_type, **kwargs)
    return Response(
        output_image.getvalue(),
        media_type=""image/jpeg"",
        headers={""cache-control"": ""s-maxage={}, public"".format(365 * 24 * 60 * 60)},
    )
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-623730934,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15,623730934,MDEyOklzc3VlQ29tbWVudDYyMzczMDkzNA==,9599,simonw,2020-05-04T22:00:38Z,2020-05-04T22:00:48Z,MEMBER,"Here's the query to create the new table:
```sql
create table apple_photos_scores as select
    ZGENERICASSET.ZUUID,
    ZGENERICASSET.ZOVERALLAESTHETICSCORE,
    ZGENERICASSET.ZCURATIONSCORE,
    ZGENERICASSET.ZPROMOTIONSCORE,
    ZGENERICASSET.ZHIGHLIGHTVISIBILITYSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZBEHAVIORALSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZFAILURESCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZHARMONIOUSCOLORSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZIMMERSIVENESSSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZINTERACTIONSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZINTERESTINGSUBJECTSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZINTRUSIVEOBJECTPRESENCESCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZLIVELYCOLORSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZLOWLIGHT,
    ZCOMPUTEDASSETATTRIBUTES.ZNOISESCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTCAMERATILTSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTCOMPOSITIONSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTLIGHTINGSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPATTERNSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPERSPECTIVESCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTPOSTPROCESSINGSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTREFLECTIONSSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZPLEASANTSYMMETRYSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZSHARPLYFOCUSEDSUBJECTSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZTASTEFULLYBLURREDSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZWELLCHOSENSUBJECTSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZWELLFRAMEDSUBJECTSCORE,
    ZCOMPUTEDASSETATTRIBUTES.ZWELLTIMEDSHOTSCORE
from
    attached.ZGENERICASSET
      join attached.ZCOMPUTEDASSETATTRIBUTES on
          attached.ZGENERICASSET.Z_PK = attached.ZCOMPUTEDASSETATTRIBUTES.Z_PK;
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612151767,Expose scores from ZCOMPUTEDASSETATTRIBUTES,
https://github.com/dogsheep/healthkit-to-sqlite/issues/6#issuecomment-513626742,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/6,513626742,MDEyOklzc3VlQ29tbWVudDUxMzYyNjc0Mg==,9599,simonw,2019-07-22T03:28:55Z,2019-07-22T03:28:55Z,MEMBER,"Here's what it looks like now as separate tables:

<img width=""1050"" alt=""hello9_and_Populate__endpoint__key_in_ASGI_scope_·_Issue__537_·_simonw_datasette"" src=""https://user-images.githubusercontent.com/9599/61604322-eba05000-abf5-11e9-8f8b-d0204e9a705a.png"">


","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",470856782,Break up records into different tables for each type,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/11#issuecomment-1462965256,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/11,1462965256,IC_kwDOJHON9s5XMxQI,9599,simonw,2023-03-09T23:22:12Z,2023-03-09T23:22:12Z,MEMBER,"Here's what the CTE from that looks like:

<img width=""469"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/224182888-50f315eb-417f-4fb1-b624-151e1740ffed.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1618130434,Implement a SQL view to make it easier to query files in a nested folder,
https://github.com/dogsheep/dogsheep-beta/issues/31#issuecomment-748426877,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/31,748426877,MDEyOklzc3VlQ29tbWVudDc0ODQyNjg3Nw==,9599,simonw,2020-12-19T06:16:11Z,2020-12-19T06:16:11Z,MEMBER,"Here's why:

    if ""fts5"" in str(e):

But the error being raised here is:

    sqlite3.OperationalError: no such column: to

I'm going to attempt the escaped on on every error.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771316301,"Searching for ""github-to-sqlite"" throws an error",
https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095317,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27,549095317,MDEyOklzc3VlQ29tbWVudDU0OTA5NTMxNw==,9599,simonw,2019-11-03T01:08:10Z,2019-11-03T01:08:10Z,MEMBER,"Hmm... one thing that could be useful is that `retweets_of_me` can support a `--since` parameter - so if run frequently it should hopefully let us know which tweets we would need to run `statuses/retweets/:id.json` against.

I'm not sure if the `--since` parameter would show me a tweet that was previously retweeted but has now been retweeted again. I'll have a bit of a test and see.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",514459062,retweets-of-me command,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615946537,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615946537,MDEyOklzc3VlQ29tbWVudDYxNTk0NjUzNw==,9599,simonw,2020-04-18T20:48:13Z,2020-04-18T20:48:13Z,MEMBER,"How about generating a signed URL?
```python
read_client.generate_presigned_url(
    ""get_object"",
    Params={
        ""Bucket"": ""dogsheep-photos-simon"",
        ""Key"": ""this_is_fine.jpg"",
    },
    ExpiresIn=600
)
```
Gave me https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398

Which does this:

```
~ $ curl -i 'https://dogsheep-photos-simon.s3.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'
HTTP/1.1 307 Temporary Redirect
x-amz-bucket-region: us-west-1
x-amz-request-id: E78CD859AEE21D33
x-amz-id-2: 648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=
Location: https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Sat, 18 Apr 2020 20:47:21 GMT
Server: AmazonS3

<?xml version=""1.0"" encoding=""UTF-8""?>
<Error><Code>TemporaryRedirect</Code><Message>Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.</Message><Endpoint>dogsheep-photos-simon.s3-us-west-1.amazonaws.com</Endpoint><Bucket>dogsheep-photos-simon</Bucket><RequestId>E78CD859AEE21D33</RequestId><HostId>648mx+1+YSGga7NDOU7Q6isfsKnEPWOLC+DI4+x2o9FCc6pSCdIaoHJUbFMI8Vsuh1ADtx46ymU=</HostId></Error>~ $ 
```
So it redirects to another URL... which returns this:
```
~ $ curl -i 'https://dogsheep-photos-simon.s3-us-west-1.amazonaws.com/this_is_fine.jpg?AWSAccessKeyId=AKIAWXFXAIOZNZ3JFO7I&Signature=x1zrS4w4OTGAACd7yHp9mYqXvN8%3D&Expires=1587243398'
HTTP/1.1 200 OK
x-amz-id-2: XafOl6mswj3yz0GJC9+Ptot1ll5sROVwqsMc10CUUfgpaUANTdIx2GhnONb5d1GVFJ6wlS2j3UY=
x-amz-request-id: 258387C180411AFE
Date: Sat, 18 Apr 2020 20:47:52 GMT
Last-Modified: Sat, 18 Apr 2020 20:37:35 GMT
ETag: ""ee04081c3182a44a1c6944e94012e977""
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 53072
Server: AmazonS3

????JFIF??C
```
So that worked! It did come back with `Content-Type: binary/octet-stream` though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631226953,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631226953,MDEyOklzc3VlQ29tbWVudDYzMTIyNjk1Mw==,9599,simonw,2020-05-20T04:20:34Z,2020-05-20T04:20:34Z,MEMBER,"Huh, it looks like Circle CI picked up the name change automatically. https://app.circleci.com/pipelines/github/dogsheep/dogsheep-photos","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790668263,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790668263,MDEyOklzc3VlQ29tbWVudDc5MDY2ODI2Mw==,9599,simonw,2021-03-04T14:43:58Z,2021-03-04T14:43:58Z,MEMBER,"I added this code to output a message ID on errors:
```diff
             print(""Errors: {}"".format(num_errors))
             print(traceback.format_exc())
+            print(""Message-Id: {}"".format(email.get(""Message-Id"", ""None"")))
             continue
```
Having found a message ID that had an error, I ran this command to see the context:

    rg --text --context 20 '44F289B0.000001.02100@SCHWARZE-DWFXMI' ~/gmail.mbox

This was for the following error:
```
  File ""/Users/simon/Dropbox/Development/google-takeout-to-sqlite/google_takeout_to_sqlite/utils.py"", line 102, in get_mbox
    message[""date""] = get_message_date(email.get(""Date""), email.get_from())
  File ""/Users/simon/Dropbox/Development/google-takeout-to-sqlite/google_takeout_to_sqlite/utils.py"", line 178, in get_message_date
    datetime_tuple = email.utils.parsedate_tz(mail_date)
  File ""/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/email/_parseaddr.py"", line 50, in parsedate_tz
    res = _parsedate_tz(data)
  File ""/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/email/_parseaddr.py"", line 69, in _parsedate_tz
    data = data.split()
AttributeError: 'Header' object has no attribute 'split'
```
Here's what I spotted in the `ripgrep` output:
```
177133570:Message-Id: <44F289B0.000001.02100@SCHWARZE-DWFXMI>
177133571-Date: Mon, 28 Aug 2006 08:14:08 +0200 (Westeurop�ische Sommerzeit)
177133572-X-Mailer: IncrediMail (5002253)
```
So it could it be that `_parsedate_tz` is having trouble with that `Mon, 28 Aug 2006 08:14:08 +0200 (Westeurop�ische Sommerzeit)` string.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-531404891,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8,531404891,MDEyOklzc3VlQ29tbWVudDUzMTQwNDg5MQ==,9599,simonw,2019-09-13T22:01:57Z,2019-09-13T22:01:57Z,MEMBER,I also wrote about this in https://simonwillison.net/2019/Sep/13/weeknotestwitter-sqlite-datasette-rure/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",490803176,--sql and --attach options for feeding commands from SQL queries,
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962708,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962708,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjcwOA==,9599,simonw,2020-07-06T00:43:10Z,2020-07-06T00:43:10Z,MEMBER,I bet it's datasette-search-all.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727,Demo is failing to deploy,
https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631127454,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25,631127454,MDEyOklzc3VlQ29tbWVudDYzMTEyNzQ1NA==,9599,simonw,2020-05-19T22:48:00Z,2020-05-21T15:58:32Z,MEMBER,"I built #23 to help with this.

    $ dogsheep-photos create-subset photos.db public.db \
        ""select sha256 from apple_photos where albums like '%Public%'""

And publish with Vercel:

    $ datasette publish now public.db --project dogsheep-photos \
        --about=dogsheep/dogsheep-photos \
        --about_url=""https://github.com/dogsheep/dogsheep-photos"" \
        --install=datasette-json-html \
        --install=datasette-cluster-map","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621332242,Create a public demo,
https://github.com/dogsheep/twitter-to-sqlite/issues/12#issuecomment-542855427,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12,542855427,MDEyOklzc3VlQ29tbWVudDU0Mjg1NTQyNw==,9599,simonw,2019-10-16T19:27:55Z,2019-10-16T19:27:55Z,MEMBER,I can do that by keeping `source` as a `TEXT` column but turning it into a non-enforced foreign key against a new `sources` table. Then I can run code that scans that column for any values beginning with a `<` and converts them.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503053800,"Extract ""source"" into a separate lookup table",
https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601875524,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35,601875524,MDEyOklzc3VlQ29tbWVudDYwMTg3NTUyNA==,9599,simonw,2020-03-20T19:30:27Z,2020-03-20T19:30:27Z,MEMBER,"I can give it a snazzier progress bar to, as requested by #10.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585282212,twitter-to-sqlite user-timeline [screen_names] --sql / --attach,
https://github.com/dogsheep/github-to-sqlite/issues/37#issuecomment-622978173,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37,622978173,MDEyOklzc3VlQ29tbWVudDYyMjk3ODE3Mw==,9599,simonw,2020-05-02T16:19:31Z,2020-05-02T16:19:47Z,MEMBER,"I can use the new `.create_view(..., replace=True)` parameter in `sqlite-utils` 2.7.2 for this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610843136,Mechanism for creating views if they don't yet exist,
https://github.com/dogsheep/healthkit-to-sqlite/issues/9#issuecomment-515226724,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/9,515226724,MDEyOklzc3VlQ29tbWVudDUxNTIyNjcyNA==,9599,simonw,2019-07-25T21:46:01Z,2019-07-25T21:46:01Z,MEMBER,I can work around this here (prior to the fix in sqlite-utils) by setting the batch size to something a bit lower here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",472429048,Too many SQL variables,
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-622990947,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,622990947,MDEyOklzc3VlQ29tbWVudDYyMjk5MDk0Nw==,9599,simonw,2020-05-02T17:54:16Z,2020-05-02T17:54:16Z,MEMBER,"I could add that window function query as a view, but only if the detected version of SQLite supports window functions.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660553646,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660553646,MDEyOklzc3VlQ29tbWVudDY2MDU1MzY0Ng==,9599,simonw,2020-07-18T22:51:41Z,2020-07-18T22:51:41Z,MEMBER,"I could fix this by putting `REFRESH_DB` in a commit message:

https://github.com/dogsheep/github-to-sqlite/blob/4ae4aa6f172344b19ff3513707195ee6d2654bd4/.github/workflows/deploy-demo.yml#L41-L46

But... doing so would lose the data I've collected in https://github-to-sqlite.dogsheep.net/github/dependents?_sort_desc=first_seen_utc concerning the first time each dependent repo was spotted.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601,Fix the demo - it breaks because of the tags table change,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602815120,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,602815120,MDEyOklzc3VlQ29tbWVudDYwMjgxNTEyMA==,9599,simonw,2020-03-23T19:40:55Z,2020-03-23T19:43:19Z,MEMBER,I could pull a pk-hashed version of the name/email into separate `raw_author` and `raw_committer` columns perhaps - against a `commit_authors` table. Could be interesting.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/dogsheep-beta/issues/31#issuecomment-748426501,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/31,748426501,MDEyOklzc3VlQ29tbWVudDc0ODQyNjUwMQ==,9599,simonw,2020-12-19T06:12:22Z,2020-12-19T06:12:22Z,MEMBER,I deliberately added support for advanced FTS in https://github.com/dogsheep/dogsheep-beta/commit/cbb2491b85d7ff416d6d429b60109e6c2d6d50b9 for #13 but that's the cause of this bug.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771316301,"Searching for ""github-to-sqlite"" throws an error",
https://github.com/dogsheep/dogsheep-photos/issues/8#issuecomment-618100434,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/8,618100434,MDEyOklzc3VlQ29tbWVudDYxODEwMDQzNA==,9599,simonw,2020-04-23T00:02:53Z,2020-04-23T00:02:53Z,MEMBER,"I don't think it matters one way or the other - I'm storing the sha256 in the filename, so the fact that I could read the MD5 back from the list bucket operation doesn't give me any benefits.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",605147638,Should I have used MD5 instead of SHA256?,
https://github.com/dogsheep/github-to-sqlite/issues/58#issuecomment-746735889,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58,746735889,MDEyOklzc3VlQ29tbWVudDc0NjczNTg4OQ==,9599,simonw,2020-12-16T17:59:50Z,2020-12-16T17:59:50Z,MEMBER,"I don't want to add a full HTML parser (like BeautifulSoup) as a dependency for this feature. Since the HTML comes from a single, trusted source (GitHub) I could probably handle this using [regular expressions](https://stackoverflow.com/a/1732454).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",769150394,Readme HTML has broken internal links,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623806533,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623806533,MDEyOklzc3VlQ29tbWVudDYyMzgwNjUzMw==,9599,simonw,2020-05-05T02:50:16Z,2020-05-05T02:50:16Z,MEMBER,"I figured there must be a separate database that Photos uses to store the text of the identified labels.

I used ""Open Files and Ports"" in Activity Monitor against the Photos app to try and spot candidates... and found `/Users/simon/Pictures/Photos Library.photoslibrary/database/search/psi.sqlite` - a 53MB SQLite database file.

<img width=""1365"" alt=""Item-0_and_Item-0_and_Item-0_and_Item-0"" src=""https://user-images.githubusercontent.com/9599/81031213-61ea0800-8e40-11ea-8237-cfce4a5128e0.png"">

Here's the schema of that file:
```
$ sqlite3 psi.sqlite .schema
CREATE TABLE word_embedding(word TEXT, extended_word TEXT, score DOUBLE);
CREATE INDEX word_embedding_index ON word_embedding(word);
CREATE VIRTUAL TABLE word_embedding_prefix USING fts5(extended_word)
/* word_embedding_prefix(extended_word) */;
CREATE TABLE IF NOT EXISTS 'word_embedding_prefix_data'(id INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE IF NOT EXISTS 'word_embedding_prefix_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS 'word_embedding_prefix_content'(id INTEGER PRIMARY KEY, c0);
CREATE TABLE IF NOT EXISTS 'word_embedding_prefix_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
CREATE TABLE IF NOT EXISTS 'word_embedding_prefix_config'(k PRIMARY KEY, v) WITHOUT ROWID;
CREATE TABLE groups(category INT2, owning_groupid INT, content_string TEXT, normalized_string TEXT, lookup_identifier TEXT, token_ranges_0 INT8, token_ranges_1 INT8, UNIQUE(category, owning_groupid, content_string, lookup_identifier, token_ranges_0, token_ranges_1));
CREATE TABLE assets(uuid_0 INT, uuid_1 INT, creationDate INT, UNIQUE(uuid_0, uuid_1));
CREATE TABLE ga(groupid INT, assetid INT, PRIMARY KEY(groupid, assetid));
CREATE TABLE collections(uuid_0 INT, uuid_1 INT, startDate INT, endDate INT, title TEXT, subtitle TEXT, keyAssetUUID_0 INT, keyAssetUUID_1 INT, typeAndNumberOfAssets INT32, sortDate DOUBLE, UNIQUE(uuid_0, uuid_1));
CREATE TABLE gc(groupid INT, collectionid INT, PRIMARY KEY(groupid, collectionid));
CREATE VIRTUAL TABLE prefix USING fts5(content='groups', normalized_string, category UNINDEXED, tokenize = 'PSITokenizer');
CREATE TABLE IF NOT EXISTS 'prefix_data'(id INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE IF NOT EXISTS 'prefix_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS 'prefix_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
CREATE TABLE IF NOT EXISTS 'prefix_config'(k PRIMARY KEY, v) WITHOUT ROWID;
CREATE TABLE lookup(identifier TEXT PRIMARY KEY, category INT2);
CREATE TRIGGER trigger_groups_insert AFTER INSERT ON groups BEGIN INSERT INTO prefix(rowid, normalized_string, category) VALUES (new.rowid, new.normalized_string, new.category); END;
CREATE TRIGGER trigger_groups_delete AFTER DELETE ON groups BEGIN INSERT INTO prefix(prefix, rowid, normalized_string, category) VALUES('delete', old.rowid, old.normalized_string, old.category); END;
CREATE INDEX group_pk ON groups(category, content_string, normalized_string, lookup_identifier);
CREATE INDEX asset_pk ON assets(uuid_0, uuid_1);
CREATE INDEX ga_assetid ON ga(assetid, groupid);
CREATE INDEX collection_pk ON collections(uuid_0, uuid_1);
CREATE INDEX gc_collectionid ON gc(collectionid);
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623805823,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623805823,MDEyOklzc3VlQ29tbWVudDYyMzgwNTgyMw==,9599,simonw,2020-05-05T02:45:56Z,2020-05-05T02:45:56Z,MEMBER,I filed an issue with `osxphotos` about this here: https://github.com/RhetTbull/osxphotos/issues/121,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790373024,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790373024,MDEyOklzc3VlQ29tbWVudDc5MDM3MzAyNA==,9599,simonw,2021-03-04T07:01:58Z,2021-03-04T07:04:06Z,MEMBER,"I got 9 warnings that look like this:
```
Errors: 1
Traceback (most recent call last):
  File ""/Users/simon/Dropbox/Development/google-takeout-to-sqlite/google_takeout_to_sqlite/utils.py"", line 103, in get_mbox
    message[""date""] = get_message_date(email.get(""Date""), email.get_from())
  File ""/Users/simon/Dropbox/Development/google-takeout-to-sqlite/google_takeout_to_sqlite/utils.py"", line 167, in get_message_date
    datetime_tuple = email.utils.parsedate_tz(mail_date)
  File ""/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/email/_parseaddr.py"", line 50, in parsedate_tz
    res = _parsedate_tz(data)
  File ""/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/email/_parseaddr.py"", line 69, in _parsedate_tz
    data = data.split()
AttributeError: 'Header' object has no attribute 'split'
```
It would be useful if those warnings told me the message ID (or similar) of the affected message so I could grep for it in the `mbox` and see what was going on.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631253852,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25,631253852,MDEyOklzc3VlQ29tbWVudDYzMTI1Mzg1Mg==,9599,simonw,2020-05-20T05:56:17Z,2020-05-21T22:26:16Z,MEMBER,"I have a `deploy-demo.sh` script now:
```bash
#!/bin/bash
if [ -f public.db ]; then
  rm public.db
fi
pipenv run dogsheep-photos create-subset photos.db public.db \
    ""select sha256 from apple_photos where albums like '%Public%'""
pipenv run sqlite-utils create-view public.db photos_on_a_map \
    ""select
      date,
      latitude,
      longitude,
      apple_photos.sha256,
      uploads.ext,
      json_object(
        'title',
        'Taken on ' || date,
        'image',
        'https://photos.simonwillison.net/i/' || uploads.sha256 || '.' || uploads.ext || '?w=400',
        'link',
        'https://photos.simonwillison.net/i/' || uploads.sha256 || '.' || uploads.ext || '?w=1200'
      ) as popup
    from
      apple_photos
      join uploads on apple_photos.sha256 = uploads.sha256
    where
      latitude is not null
    order by
      date desc"" \
    --replace
pipenv run datasette publish now public.db --project dogsheep-photos \
    --about=dogsheep/dogsheep-photos \
    --about_url=""https://github.com/dogsheep/dogsheep-photos"" \
    --install=datasette-json-html \
    --install=datasette-pretty-json \
    --install=datasette-cluster-map>=0.10 \
    --title ""Dogsheep Photos demo""
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621332242,Create a public demo,
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-543290744,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3,543290744,MDEyOklzc3VlQ29tbWVudDU0MzI5MDc0NA==,9599,simonw,2019-10-17T17:57:14Z,2019-10-17T17:57:14Z,MEMBER,I have a working command now. I'm going to ship it early because it could do with some other people trying it out.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833975,Command for running a search and saving tweets for that search,
https://github.com/dogsheep/dogsheep-beta/issues/16#issuecomment-694548909,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16,694548909,MDEyOklzc3VlQ29tbWVudDY5NDU0ODkwOQ==,9599,simonw,2020-09-17T23:15:09Z,2020-09-17T23:15:09Z,MEMBER,"I have sort by date now, #21.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694493566,Timeline view,
https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706784028,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4,706784028,MDEyOklzc3VlQ29tbWVudDcwNjc4NDAyOA==,9599,simonw,2020-10-11T23:20:32Z,2020-10-11T23:20:32Z,MEMBER,I haven't done the FTS on OCR yet. I'm going to move that to another ticket because it requires more thought.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938508,Configure FTS + add an index on the date columns,
https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095463,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27,549095463,MDEyOklzc3VlQ29tbWVudDU0OTA5NTQ2Mw==,9599,simonw,2019-11-03T01:10:52Z,2019-11-03T01:10:52Z,MEMBER,"I imagine it won't, since the data I would be recording and then passing to `since_id` would be the highest ID of my own tweets that have been retweeted at least once. So it won't be able to spot if I should check for fresh retweets of a given tweet.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",514459062,retweets-of-me command,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602846293,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,602846293,MDEyOklzc3VlQ29tbWVudDYwMjg0NjI5Mw==,9599,simonw,2020-03-23T20:44:40Z,2020-03-23T20:44:40Z,MEMBER,I implemented the `raw_authors` idea.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790693674,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790693674,MDEyOklzc3VlQ29tbWVudDc5MDY5MzY3NA==,9599,simonw,2021-03-04T15:18:36Z,2021-03-04T15:18:36Z,MEMBER,"I imported my 10GB mbox with 750,000 emails in it, ran this tool (with a hacked fix for the blob column problem) - and now a search that returns 92 results takes 25.37ms! This is fantastic.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/11#issuecomment-1462968053,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/11,1462968053,IC_kwDOJHON9s5XMx71,9599,simonw,2023-03-09T23:24:01Z,2023-03-09T23:24:01Z,MEMBER,"I improved the readability by removing some unnecessary table aliases:
```sql
with recursive nested_folders(folder_id, descendant_folder_id) as (
  -- base case: select all immediate children of the root folder
  select id, id from folders where parent is null
  union all
  -- recursive case: select all children of the previous level of nested folders
  select nested_folders.folder_id, folders.id from nested_folders
  join folders on nested_folders.descendant_folder_id = folders.parent
)
-- Find notes within all descendants of folder 1
select *
from notes
where folder in (
  select descendant_folder_id from nested_folders where folder_id = 1
);
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1618130434,Implement a SQL view to make it easier to query files in a nested folder,
https://github.com/dogsheep/swarm-to-sqlite/issues/11#issuecomment-761967094,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11,761967094,MDEyOklzc3VlQ29tbWVudDc2MTk2NzA5NA==,9599,simonw,2021-01-18T04:11:13Z,2021-01-18T04:11:13Z,MEMBER,"I just got a similar error:

```
  File ""/home/dogsheep/datasette-venv/lib/python3.8/site-packages/swarm_to_sqlite/utils.py"", line 79, in save_checkin
    checkins_table.m2m(""users"", user, m2m_table=""with"", pk=""id"")
  File ""/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py"", line 2048, in m2m
    id = other_table.insert(record, pk=pk, replace=True).last_pk
  File ""/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py"", line 1781, in insert
    return self.insert_all(
  File ""/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py"", line 1899, in insert_all
    self.insert_chunk(
  File ""/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py"", line 1709, in insert_chunk
    result = self.db.execute(query, params)
  File ""/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py"", line 226, in execute
    return self.conn.execute(sql, parameters)
pysqlite3.dbapi2.OperationalError: table users has no column named countryCode

```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",743400216,Error thrown: sqlite3.OperationalError: table users has no column named lastName,
https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502543165,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502543165,IC_kwDODD6af85Zjv09,9599,simonw,2023-04-11T01:10:36Z,2023-04-11T01:11:47Z,MEMBER,"I just had that error myself on macOS while running the tests:
```
ERROR tests/test_save_checkin.py::test_tables - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_venue - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_event - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_sticker - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_likes - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_with_ - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_users - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_photos - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_posts - sqlite3.OperationalError: table sqlite_master may not be modified
ERROR tests/test_save_checkin.py::test_view - sqlite3.OperationalError: table sqlite_master may not be modified
```
`pytest --pdb` shows it happening in the bit that adds foreign keys:
```
> /Users/simon/.local/share/virtualenvs/swarm-to-sqlite-daPW7yIJ/lib/python3.9/site-packages/sqlite_utils/db.py(1096)add_foreign_keys()
-> cursor.execute(
(Pdb) list
1096 >>	                cursor.execute(
1097 	                    ""UPDATE sqlite_master SET sql = ? WHERE name = ?"",
1098 	                    (new_sql, table_name),
1099 	                )
1100 	            cursor.execute(""PRAGMA schema_version = %d"" % (schema_version + 1))
1101 ->	            cursor.execute(""PRAGMA writable_schema = 0"")
1102 	        # Have to VACUUM outside the transaction to ensure .foreign_keys property
1103 	        # can see the newly created foreign key.
1104 	        self.vacuum()
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR:  table sqlite_master may not be modified,
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631226481,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631226481,MDEyOklzc3VlQ29tbWVudDYzMTIyNjQ4MQ==,9599,simonw,2020-05-20T04:18:29Z,2020-05-20T04:18:29Z,MEMBER,I just renamed the repository.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/twitter-to-sqlite/issues/2#issuecomment-527990908,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2,527990908,MDEyOklzc3VlQ29tbWVudDUyNzk5MDkwOA==,9599,simonw,2019-09-04T16:57:24Z,2019-09-04T16:57:24Z,MEMBER,"I just tried this using `max_id=` pagination as described in [Working with timelines](https://developer.twitter.com/en/docs/tweets/timelines/guides/working-with-timelines) and I got back all 17,759 of my tweets.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833698,"""twitter-to-sqlite user-timeline"" command for pulling tweets by a specific user",
https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-605325897,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1,605325897,MDEyOklzc3VlQ29tbWVudDYwNTMyNTg5Nw==,9599,simonw,2020-03-27T21:37:26Z,2020-03-27T21:38:37Z,MEMBER,"I keep getting 503 errors even though I appear to be staying within the rate limit:
```
{'Date': 'Fri, 27 Mar 2020 21:35:57 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding':
'chunked', 'Connection': 'keep-alive', 'Server': 'Apache/2.4.25 (Debian)', 'Content-Location':
'get.php', 'Vary': 'negotiate', 'TCN': 'choice', 'Set-Cookie': '...;  httponly',
'X-Frame-Options': 'SAMEORIGIN', 'Status': '200 OK', 'X-Limit-Key-Limit': '10000',
'X-Limit-Key-Remaining': '9960', 'X-Limit-Key-Reset': '282', 'X-Source': 'Pocket',
'P3P': 'policyref=""/w3c/p3p.xml"", CP=""ALL CURa ADMa DEVa OUR IND UNI COM NAV INT STA PRE""'}

  [##----------------------------------]    6%  06:49:27

{'Date': 'Fri, 27 Mar 2020 21:36:06 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length':
'23', 'Connection': 'keep-alive', 'Server': 'Apache/2.4.25 (Debian)', 'Content-Location': 'get.php',
'Vary': 'negotiate', 'TCN': 'choice', 'Set-Cookie': '...', 'X-Frame-Options': 'SAMEORIGIN',
'X-Error': 'Pocket is currently under heavy load. Please wait a moment and try again.',
'X-Error-Code': '199', 'Status': '503 Service Unavailable', 'X-Source': 'Pocket',
'P3P': 'policyref=""/w3c/p3p.xml"", CP=""ALL CURa ADMa DEVa OUR IND UNI COM NAV INT STA PRE""'}
```
I'm going to try doing a few automatic retries any time I see a 503 error.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503233021,Use better pagination (and implement progress bar),
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606309165,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606309165,MDEyOklzc3VlQ29tbWVudDYwNjMwOTE2NQ==,9599,simonw,2020-03-30T23:41:31Z,2020-03-30T23:41:31Z,MEMBER,I like the separate `user_timeline_since` table solution.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/dogsheep-photos/issues/2#issuecomment-615931488,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2,615931488,MDEyOklzc3VlQ29tbWVudDYxNTkzMTQ4OA==,9599,simonw,2020-04-18T19:24:02Z,2020-04-18T19:24:02Z,MEMBER,I made a start on this last week with a https://github.com/simonw/heic-to-jpeg proxy.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533352,Ability to convert HEIC images to JPEG,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602807178,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,602807178,MDEyOklzc3VlQ29tbWVudDYwMjgwNzE3OA==,9599,simonw,2020-03-23T19:24:43Z,2020-03-23T19:24:43Z,MEMBER,I need to find an example before I work on this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643393506,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40,643393506,MDEyOklzc3VlQ29tbWVudDY0MzM5MzUwNg==,9599,simonw,2020-06-12T17:21:14Z,2020-06-12T17:21:14Z,MEMBER,"I only install SQLite for this:

https://github.com/dogsheep/github-to-sqlite/blob/c0d54e0260468be38152293df5abd775c068495d/.github/workflows/deploy-demo.yml#L77-L78

I'm going to remove the need to install sqlite3 by making this possible with sqlite-utils: https://github.com/simonw/sqlite-utils/issues/115","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637899539,Demo deploy is broken,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461234311,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2,1461234311,IC_kwDOJHON9s5XGKqH,9599,simonw,2023-03-09T03:56:24Z,2023-03-09T03:56:24Z,MEMBER,"I opened the ""Script Editor"" app on my computer, used Window -> Library to open the Library panel, then clicked on the Notes app there. I got this:

<img width=""1240"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/223912876-09e8660e-f59a-4f04-8992-668e09c8d7d4.png"">

So the notes object has these properties:

- name (text) : the name of the note (normally the first line of the body)
- id (text, r/o) : the unique identifier of the note
- container ([folder](applewebdata://621FA8D9-C995-4081-B3B3-149B0EA04C7F#Notes-Suite.folder), r/o) : the folder of the note
- body (text) : the HTML content of the note
- plaintext (text, r/o) : the plaintext content of the note
- creation date (date, r/o) : the creation date of the note
- modification date (date, r/o) : the modification date of the note
- password protected (boolean, r/o) : Is the note password protected?
- shared (boolean, r/o) : Is the note shared?

I'm going to ignore the concept of attachments for the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616354999,First working version,
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602918689,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602918689,MDEyOklzc3VlQ29tbWVudDYwMjkxODY4OQ==,9599,simonw,2020-03-23T23:43:39Z,2020-03-23T23:47:50Z,MEMBER,I pointed https://github-to-sqlite.dogsheep.net/ at it. May take a few minutes for the new certificate to provision though.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/twitter-to-sqlite/issues/5#issuecomment-527684202,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/5,527684202,MDEyOklzc3VlQ29tbWVudDUyNzY4NDIwMg==,9599,simonw,2019-09-03T23:56:28Z,2019-09-03T23:56:28Z,MEMBER,I previously used betamax here: https://github.com/simonw/github-contents/blob/master/test_github_contents.py,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488874815,Write tests that simulate the Twitter API,
https://github.com/dogsheep/dogsheep-beta/issues/4#issuecomment-684395444,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/4,684395444,MDEyOklzc3VlQ29tbWVudDY4NDM5NTQ0NA==,9599,simonw,2020-09-01T06:00:03Z,2020-09-01T06:00:03Z,MEMBER,I ran `sqlite-utils optimize beta.db` against my test DB and the size reduced from 183M to 176M - and a 450ms search ran in 359ms. So not a huge improvement but still worthwhile.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",689839399,Optimize the FTS table,
https://github.com/dogsheep/healthkit-to-sqlite/issues/1#issuecomment-513437463,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/1,513437463,MDEyOklzc3VlQ29tbWVudDUxMzQzNzQ2Mw==,9599,simonw,2019-07-20T05:19:59Z,2019-07-20T05:19:59Z,MEMBER,"I ran xml_analyser against the XML HealthKit `export.xml` file and got the following results:

```python
{
  'ActivitySummary': {'attr_counts': {'activeEnergyBurned': 980,
                                     'activeEnergyBurnedGoal': 980,
                                     'activeEnergyBurnedUnit': 980,
                                     'appleExerciseTime': 980,
                                     'appleExerciseTimeGoal': 980,
                                     'appleStandHours': 980,
                                     'appleStandHoursGoal': 980,
                                     'dateComponents': 980},
                     'child_counts': {},
                     'count': 980,
                     'parent_counts': {'HealthData': 980}},
 'Correlation': {'attr_counts': {'creationDate': 1,
                                 'endDate': 1,
                                 'sourceName': 1,
                                 'sourceVersion': 1,
                                 'startDate': 1,
                                 'type': 1},
                 'child_counts': {'MetadataEntry': 1, 'Record': 2},
                 'count': 1,
                 'parent_counts': {'HealthData': 1}},
 'ExportDate': {'attr_counts': {'value': 1},
                'child_counts': {},
                'count': 1,
                'parent_counts': {'HealthData': 1}},
 'HealthData': {'attr_counts': {'locale': 1},
                'child_counts': {'ActivitySummary': 980,
                                 'Correlation': 1,
                                 'ExportDate': 1,
                                 'Me': 1,
                                 'Record': 2672231,
                                 'Workout': 663},
                'count': 1,
                'parent_counts': {}},
 'HeartRateVariabilityMetadataList': {'attr_counts': {},
                                      'child_counts': {'InstantaneousBeatsPerMinute': 93653},
                                      'count': 2318,
                                      'parent_counts': {'Record': 2318}},
 'InstantaneousBeatsPerMinute': {'attr_counts': {'bpm': 93653, 'time': 93653},
                                 'child_counts': {},
                                 'count': 93653,
                                 'parent_counts': {'HeartRateVariabilityMetadataList': 93653}},
 'Location': {'attr_counts': {'altitude': 398683,
                              'course': 398683,
                              'date': 398683,
                              'horizontalAccuracy': 398683,
                              'latitude': 398683,
                              'longitude': 398683,
                              'speed': 398683,
                              'verticalAccuracy': 398683},
              'child_counts': {},
              'count': 398683,
              'parent_counts': {'WorkoutRoute': 398683}},
 'Me': {'attr_counts': {'HKCharacteristicTypeIdentifierBiologicalSex': 1,
                        'HKCharacteristicTypeIdentifierBloodType': 1,
                        'HKCharacteristicTypeIdentifierDateOfBirth': 1,
                        'HKCharacteristicTypeIdentifierFitzpatrickSkinType': 1},
        'child_counts': {},
        'count': 1,
        'parent_counts': {'HealthData': 1}},
 'MetadataEntry': {'attr_counts': {'key': 290449, 'value': 290449},
                   'child_counts': {},
                   'count': 290449,
                   'parent_counts': {'Correlation': 1,
                                     'Record': 287974,
                                     'Workout': 1928,
                                     'WorkoutRoute': 546}},
 'Record': {'attr_counts': {'creationDate': 2672233,
                            'device': 2665111,
                            'endDate': 2672233,
                            'sourceName': 2672233,
                            'sourceVersion': 2671779,
                            'startDate': 2672233,
                            'type': 2672233,
                            'unit': 2650012,
                            'value': 2672232},
            'child_counts': {'HeartRateVariabilityMetadataList': 2318,
                             'MetadataEntry': 287974},
            'count': 2672233,
            'parent_counts': {'Correlation': 2, 'HealthData': 2672231}},
 'Workout': {'attr_counts': {'creationDate': 663,
                             'device': 230,
                             'duration': 663,
                             'durationUnit': 663,
                             'endDate': 663,
                             'sourceName': 663,
                             'sourceVersion': 663,
                             'startDate': 663,
                             'totalDistance': 663,
                             'totalDistanceUnit': 663,
                             'totalEnergyBurned': 663,
                             'totalEnergyBurnedUnit': 663,
                             'workoutActivityType': 663},
             'child_counts': {'MetadataEntry': 1928,
                              'WorkoutEvent': 2094,
                              'WorkoutRoute': 340},
             'count': 663,
             'parent_counts': {'HealthData': 663}},
 'WorkoutEvent': {'attr_counts': {'date': 2094,
                                  'duration': 837,
                                  'durationUnit': 837,
                                  'type': 2094},
                  'child_counts': {},
                  'count': 2094,
                  'parent_counts': {'Workout': 2094}},
 'WorkoutRoute': {'attr_counts': {'creationDate': 340,
                                  'endDate': 340,
                                  'sourceName': 340,
                                  'sourceVersion': 340,
                                  'startDate': 340},
                  'child_counts': {'Location': 398683, 'MetadataEntry': 546},
                  'count': 340,
                  'parent_counts': {'Workout': 340}}}
```

The most interesting bit is this:

```python
 'HealthData': {'attr_counts': {'locale': 1},
                'child_counts': {'ActivitySummary': 980,
                                 'Correlation': 1,
                                 'ExportDate': 1,
                                 'Me': 1,
                                 'Record': 2672231,
                                 'Workout': 663},
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",470637068,Use XML Analyser to figure out the structure of the export XML,
https://github.com/dogsheep/github-to-sqlite/issues/60#issuecomment-770071568,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/60,770071568,MDEyOklzc3VlQ29tbWVudDc3MDA3MTU2OA==,9599,simonw,2021-01-29T21:56:15Z,2021-01-29T21:56:15Z,MEMBER,"I really like the way you're using pipes here - really smart. It's similar to how I build the demo database in this GitHub Actions workflow:

https://github.com/dogsheep/github-to-sqlite/blob/62dfd3bc4014b108200001ef4bc746feb6f33b45/.github/workflows/deploy-demo.yml#L52-L82

`twitter-to-sqlite` actually has a mechanism for doing this kind of thing, documented at https://github.com/dogsheep/twitter-to-sqlite#providing-input-from-a-sql-query-with---sql-and---attach

It lets you do things like:

```
$ twitter-to-sqlite users-lookup my.db --sql=""select follower_id from following"" --ids
```
Maybe I should add something similar to `github-to-sqlite`? Feels like it could be really useful.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",797097140,Use Data from SQLite in other commands,
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694557425,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694557425,MDEyOklzc3VlQ29tbWVudDY5NDU1NzQyNQ==,9599,simonw,2020-09-17T23:41:01Z,2020-09-17T23:41:01Z,MEMBER,I removed all of the `json.loads()` calls and I'm still getting that `Undefined` error.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/swarm-to-sqlite/issues/2#issuecomment-526701674,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/2,526701674,MDEyOklzc3VlQ29tbWVudDUyNjcwMTY3NA==,9599,simonw,2019-08-30T18:24:26Z,2019-08-30T18:24:26Z,MEMBER,I renamed `--file` to `--load` in 0e5b6025c6f9823ff81aa8aae1cbff5c45e57baf,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",487598468,--save option to dump checkins to a JSON file on disk,
https://github.com/dogsheep/github-to-sqlite/issues/17#issuecomment-597354514,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/17,597354514,MDEyOklzc3VlQ29tbWVudDU5NzM1NDUxNA==,9599,simonw,2020-03-10T22:37:45Z,2020-03-10T22:37:45Z,MEMBER,I should add an option to stop the moment you see a commit you have fetched before.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",578883725,Command for importing commits,
https://github.com/dogsheep/github-to-sqlite/issues/14#issuecomment-559883311,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14,559883311,MDEyOklzc3VlQ29tbWVudDU1OTg4MzMxMQ==,9599,simonw,2019-11-29T21:30:37Z,2019-11-29T21:30:37Z,MEMBER,"I should build the command to persist ETags and obey their polling guidelines:

> Events are optimized for polling with the ""ETag"" header. If no new events have been triggered, you will see a ""304 Not Modified"" response, and your current rate limit will be untouched. There is also an ""X-Poll-Interval"" header that specifies how often (in seconds) you are allowed to poll. In times of high server load, the time may increase. Please obey the header.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",530491074,Command for importing events,
https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711078917,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711078917,MDEyOklzc3VlQ29tbWVudDcxMTA3ODkxNw==,9599,simonw,2020-10-17T20:51:55Z,2020-10-17T20:52:03Z,MEMBER,"I switched my phone to Spanish and ran an export - I got a file called `exportar.zip`. Unzipped I still got a `apple_
health_export` folder but the root contained:
```
electrocardiograms/
export_cda.xml
exportar.xml
workout-routes/
```
It looks like `export_cda.xml` does not have a translated name, so maybe I can ignore it and look for the _other_ `.xml` file in that directory.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723838331,export.xml file name varies with different language settings,
https://github.com/dogsheep/pocket-to-sqlite/issues/12#issuecomment-1627563202,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12,1627563202,IC_kwDODLZ_YM5hAqTC,9599,simonw,2023-07-09T01:14:27Z,2023-07-09T01:14:27Z,MEMBER,I tested this locally with `python -m build` and then `pip install ...whl` in a fresh virtual environment.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1795187493,Switch to pyproject.toml,
https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-747029636,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29,747029636,MDEyOklzc3VlQ29tbWVudDc0NzAyOTYzNg==,9599,simonw,2020-12-16T21:14:03Z,2020-12-16T21:14:03Z,MEMBER,"I think I can do this as a cunning trick in `display_sql`. Consider this example query: https://til.simonwillison.net/tils?sql=select%0D%0A++path%2C%0D%0A++snippet%28til_fts%2C+-1%2C+%27b4de2a49c8%27%2C+%278c94a2ed4b%27%2C+%27...%27%2C+60%29+as+snippet%0D%0Afrom%0D%0A++til%0D%0A++join+til_fts+on+til.rowid+%3D+til_fts.rowid%0D%0Awhere%0D%0A++til_fts+match+escape_fts%28%3Aq%29%0D%0A++and+path+%3D+%27asgi_lifespan-test-httpx.md%27%0D%0A&q=pytest

```sql
select
  path,
  snippet(til_fts, -1, 'b4de2a49c8', '8c94a2ed4b', '...', 60) as snippet
from
  til
  join til_fts on til.rowid = til_fts.rowid
where
  til_fts match escape_fts(:q)
  and path = 'asgi_lifespan-test-httpx.md'
```
The `and path = 'asgi_lifespan-test-httpx.md'` bit means we only get back a specific document - but the snippet highlighting is applied to it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",724759588,Add search highlighting snippets,
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-695113871,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,695113871,MDEyOklzc3VlQ29tbWVudDY5NTExMzg3MQ==,9599,simonw,2020-09-18T22:30:17Z,2020-09-18T22:30:17Z,MEMBER,"I think I know what's going on here:

https://github.com/dogsheep/dogsheep-beta/blob/0f1b951c5131d16f3c8559a8e4d79ed5c559e3cb/dogsheep_beta/__init__.py#L166-L171

This is a logic bug - the `compiled` variable could be the template from the previous loop!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629404,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502629404,IC_kwDODD6af85ZkE4c,9599,simonw,2023-04-11T03:15:47Z,2023-04-11T03:46:17Z,MEMBER,"I think `swarm-to-sqlite` needs to avoid this error, maybe by setting up foreign keys in another way - or even by skipping foreign keys entirely on databases that don't support this kind of operation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR:  table sqlite_master may not be modified,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606844521,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606844521,MDEyOklzc3VlQ29tbWVudDYwNjg0NDUyMQ==,9599,simonw,2020-03-31T20:01:39Z,2020-03-31T20:01:39Z,MEMBER,"I think `utils.fetch_timeline()` grows a new argument, `since_key`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/dogsheep-beta/issues/11#issuecomment-686618669,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/11,686618669,MDEyOklzc3VlQ29tbWVudDY4NjYxODY2OQ==,9599,simonw,2020-09-03T16:47:34Z,2020-09-03T16:53:25Z,MEMBER,I think a `is_public` integer column which defaults to 0 would be good here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",692125110,Public / Private mechanism,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-606307376,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,606307376,MDEyOklzc3VlQ29tbWVudDYwNjMwNzM3Ng==,9599,simonw,2020-03-30T23:35:40Z,2020-03-30T23:39:15Z,MEMBER,"I think five separate tables:

* followers_count_history
* friends_count_history
* listed_count_history
* favourites_count_history
* statuses_count_history

Each with the following structure:

* datetime (ISO UTC)
* user (ID, foreign key to users)
* count (integer)

I'm tempted to have a compound primary key here - user, datetime
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461232709,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2,1461232709,IC_kwDOJHON9s5XGKRF,9599,simonw,2023-03-09T03:54:28Z,2023-03-09T03:54:28Z,MEMBER,"I think the AppleScript I want to pass to `osascript` looks like this:
```applescript
tell application ""Notes""
   repeat with eachNote in every note
      set noteId to the id of eachNote
      set noteTitle to the name of eachNote
      set noteBody to the body of eachNote
      log ""------------------------"" & ""\n""
      log noteId & ""\n""
      log noteTitle & ""\n\n""
      log noteBody & ""\n""
   end repeat
end tell
```
But there are a few more properties I'd like to get - created and updated date for example.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616354999,First working version,
https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660553711,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660553711,MDEyOklzc3VlQ29tbWVudDY2MDU1MzcxMQ==,9599,simonw,2020-07-18T22:52:16Z,2020-07-18T22:52:16Z,MEMBER,"I think the best fix is to download the `github.db` database, manually fix it and then manually deploy it to Cloud Run from my laptop.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601,Fix the demo - it breaks because of the tags table change,
https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711079056,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711079056,MDEyOklzc3VlQ29tbWVudDcxMTA3OTA1Ng==,9599,simonw,2020-10-17T20:53:00Z,2020-10-17T20:53:00Z,MEMBER,"I think the safest thing is to sniff the first few lines of the file. Those should be the same no matter the language that was used:
```xml
<?xml version=""1.0"" encoding=""UTF-8""?>
<!DOCTYPE HealthData [
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723838331,export.xml file name varies with different language settings,
https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552129686,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29,552129686,MDEyOklzc3VlQ29tbWVudDU1MjEyOTY4Ng==,9599,simonw,2019-11-09T19:27:39Z,2019-11-09T19:27:39Z,MEMBER,I think this is fixed by the latest version of `sqlite-utils` - https://sqlite-utils.readthedocs.io/en/latest/changelog.html#v1-12-1 - I'll bump the dependency.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",518725064,`import` command fails on empty files,
https://github.com/dogsheep/github-to-sqlite/issues/39#issuecomment-675509550,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/39,675509550,MDEyOklzc3VlQ29tbWVudDY3NTUwOTU1MA==,9599,simonw,2020-08-18T14:23:56Z,2020-08-18T14:23:56Z,MEMBER,I think this is fixed: https://github-to-sqlite.dogsheep.net/github/issues?_facet=repo,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613777056,issues foreign key to repo isn't working,
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622135654,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622135654,MDEyOklzc3VlQ29tbWVudDYyMjEzNTY1NA==,9599,simonw,2020-04-30T21:53:44Z,2020-04-30T21:56:06Z,MEMBER,"I think this is the neatest scraping pattern:
```python
[a[""href""].lstrip(""/"") for a in soup.select(""a[data-hovercard-type=repository]"")]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,Command for retrieving dependents for a repo,
https://github.com/dogsheep/dogsheep-beta/issues/19#issuecomment-688625430,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/19,688625430,MDEyOklzc3VlQ29tbWVudDY4ODYyNTQzMA==,9599,simonw,2020-09-08T05:24:50Z,2020-09-08T05:24:50Z,MEMBER,"I thought about allowing tables to define a incremental indexing SQL query - maybe something that can return just records touched in the past hour, or records since a recorded ""last indexed record"" value.

The problem with this is deletes - if you delete a record, how does the indexer know to remove it? See #18  - that's already caused problems.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695556681,Figure out incremental re-indexing,
https://github.com/dogsheep/pocket-to-sqlite/pull/7#issuecomment-1221621700,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/7,1221621700,IC_kwDODLZ_YM5I0HfE,9599,simonw,2022-08-21T21:11:12Z,2022-08-21T21:11:12Z,MEMBER,I thought this might need a documentation update but `--all` is already covered: https://github.com/dogsheep/pocket-to-sqlite/blob/0.2.1/README.md,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",750141615,Fixed conflicting  CLI flags,
https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-906646452,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13,906646452,IC_kwDOEhK-wc42ClO0,9599,simonw,2021-08-26T18:34:34Z,2021-08-26T18:35:20Z,MEMBER,"I tried this ampersand fix: https://regex101.com/r/ojU2H9/1
```python

# https://regex101.com/r/ojU2H9/1
_invalid_ampersand_re = re.compile(r'&(?![a-z0-9]+;)')

def fix_bad_xml(xml):
    # More fixes for things like '&' not as part of an entity
    return _invalid_ampersand_re.sub('&amp;', xml)
```

Even with that I'm still getting total garbage in the `<en-note>` content - it's just HTML, not even trying to be XML.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",978743426,xml.etree.ElementTree.ParseError: not well-formed (invalid token),
https://github.com/dogsheep/apple-notes-to-sqlite/issues/7#issuecomment-1462570187,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7,1462570187,IC_kwDOJHON9s5XLQzL,9599,simonw,2023-03-09T18:30:24Z,2023-03-09T18:30:24Z,MEMBER,"I used ChatGPT to write this:
```
osascript -e 'tell application ""Notes""
    set allFolders to folders
    repeat with aFolder in allFolders
        set folderId to id of aFolder
        set folderName to name of aFolder
        set folderContainer to container of aFolder
        set folderContainerName to name of folderContainer
        log ""Folder ID: "" & folderId
        log ""Folder Name: "" & folderName
        log ""Folder Container: "" & folderContainerName
        log "" ""
        --check for nested folders
        if count of folders of aFolder > 0 then
            set nestedFolders to folders of aFolder
            repeat with aNestedFolder in nestedFolders
                set nestedFolderId to id of aNestedFolder
                set nestedFolderName to name of aNestedFolder
                set nestedFolderContainer to container of aNestedFolder
                set nestedFolderContainerName to name of nestedFolderContainer
                log ""    Nested Folder ID: "" & nestedFolderId
                log ""    Nested Folder Name: "" & nestedFolderName
                log ""    Nested Folder Container: "" & nestedFolderContainerName
                log "" ""
            end repeat
        end if
    end repeat
end tell
'
```
Which for my account output this:
```
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6113
Folder Name: Blog posts
Folder Container: iCloud
 
    Nested Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p7995
    Nested Folder Name: Nested inside blog posts
    Nested Folder Container: Blog posts
 
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p698
Folder Name: JSK
Folder Container: iCloud
 
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p7995
Folder Name: Nested inside blog posts
Folder Container: Blog posts
 
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p3526
Folder Name: New Folder
Folder Container: iCloud
 
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p3839
Folder Name: New Folder 1
Folder Container: iCloud
 
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p2
Folder Name: Notes
Folder Container: iCloud
 
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6059
Folder Name: Quick Notes
Folder Container: iCloud
 
Folder ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p7283
Folder Name: UK Christmas 2022
Folder Container: iCloud
```
So I think the correct approach here is to run code at the start to list all of the folders (no need to do fancy recursion though, just a flat list with the parent containers is enough) and create a model of that hierarchy in SQLite.

Then when I import notes I can foreign key reference them back to their containing folder.

I'm tempted to use `rowid` for the foreign keys because the official IDs are pretty long.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1617769847,Folder support,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607011972,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,607011972,MDEyOklzc3VlQ29tbWVudDYwNzAxMTk3Mg==,9599,simonw,2020-04-01T03:49:02Z,2020-04-01T03:50:01Z,MEMBER,"I want the datetime value to look like `2020-04-01T03:34:58+00:00` (the format returned by the Twitter API which I am storing in other tables at the moment).
```
>>> datetime.utcnow().isoformat().split('.')[0] + '+00:00'
'2020-04-01T03:49:52+00:00'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,
https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660536265,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660536265,MDEyOklzc3VlQ29tbWVudDY2MDUzNjI2NQ==,9599,simonw,2020-07-18T20:15:12Z,2020-07-18T20:15:12Z,MEMBER,"I want to create a SQL query which shows me all of my repositories that have commits that are NOT in the most recent release.

The releases table doesn't have enough information for this because it doesn't tell you the commit hash associated with each release, just the tag: https://github-to-sqlite.dogsheep.net/github/releases","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904,github-to-sqlite tags command for fetching tags,
https://github.com/dogsheep/twitter-to-sqlite/issues/25#issuecomment-543266947,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/25,543266947,MDEyOklzc3VlQ29tbWVudDU0MzI2Njk0Nw==,9599,simonw,2019-10-17T16:56:06Z,2019-10-17T16:56:06Z,MEMBER,I wrote a test that proves that this is a problem. Should be an easy fix though.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",508578780,Ensure migrations don't accidentally create foreign key twice,
https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711074031,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711074031,MDEyOklzc3VlQ29tbWVudDcxMTA3NDAzMQ==,9599,simonw,2020-10-17T20:14:01Z,2020-10-17T20:14:01Z,MEMBER,I'd be happy to teach the tool to look for `export.xml` or `eksport.xml` - and then expand that list to other languages.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723838331,export.xml file name varies with different language settings,
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693775622,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693775622,MDEyOklzc3VlQ29tbWVudDY5Mzc3NTYyMg==,9599,simonw,2020-09-17T02:48:34Z,2020-09-17T02:48:34Z,MEMBER,I'd like a `--paginate` option that does the same thing as https://github.com/simonw/paginate-json,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls,
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694554584,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694554584,MDEyOklzc3VlQ29tbWVudDY5NDU1NDU4NA==,9599,simonw,2020-09-17T23:31:25Z,2020-09-17T23:31:25Z,MEMBER,"I'd prefer it if errors in these template fragments were displayed as errors inline where the fragment should have been inserted, rather than 500ing the whole page - especially since the template fragments are user-provided and could have all kinds of odd errors in them which should be as easy to debug as possible.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633629944,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,633629944,MDEyOklzc3VlQ29tbWVudDYzMzYyOTk0NA==,9599,simonw,2020-05-25T15:47:42Z,2020-05-25T15:47:42Z,MEMBER,"I'll add a proper section to the README, but for the moment here's how I do this.

First, install `datasette` and the `datasette-media` plugin.

Create a `metadata.yaml` file with the following content:

```yaml
plugins:
  datasette-media:
    photo:
      sql: |-
        select path as filepath, 200 as resize_height from apple_photos where uuid = :key
    photo-big:
      sql: |-
        select path as filepath, 1024 as resize_height from apple_photos where uuid = :key
```
Now run `datasette -m metadata.yaml photos.db` - thumbnails will be served at http://127.0.0.1:8001/-/media/photo/F4469918-13F3-43D8-9EC1-734C0E6B60AD and larger sizes of the image at http://127.0.0.1:8001/-/media/photo-big/A8B02C7D-365E-448B-9510-69F80C26304D

I also made myself two custom pages, one showing recent images and one showing random images.

To do this, install the `datasette-template-sql` plugin and then create a `templates/pages` directory and add these files:

`recent-photos.html`
```html
<h1>Recent photos</h1>

<div>
{% for photo in sql(""select * from apple_photos order by date desc limit 100"") %}
    <img src=""/-/media/photo/{{ photo['uuid'] }}"">
{% endfor %}
</div>
```

`random-photos.html`
```html
<h1>Random photos</h1>

<div>
{% for photo in sql(""with foo as (select * from apple_photos order by date desc limit 5000) select * from foo order by random() limit 100"") %}
    <img src=""/-/media/photo/{{ photo['uuid'] }}"">
{% endfor %}
</div>
```

Now run `datasette -m metadata.yaml photos.db --template-dir=templates/`

Visit http://127.0.0.1:8001/random-photos to see some random photos or http://127.0.0.1:8002/recent-photos for recent photos.

This is using this mechanism: https://datasette.readthedocs.io/en/stable/custom_templates.html#custom-pages","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633644225,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,633644225,MDEyOklzc3VlQ29tbWVudDYzMzY0NDIyNQ==,9599,simonw,2020-05-25T16:30:44Z,2020-05-25T16:30:44Z,MEMBER,I'll add docs on using `datasette-json-html` too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685966707,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7,685966707,MDEyOklzc3VlQ29tbWVudDY4NTk2NjcwNw==,9599,simonw,2020-09-02T20:04:08Z,2020-09-02T20:04:08Z,MEMBER,I'll make `category` a foreign key to a `categories` table so Datasette can automatically show the `name` column.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691265198,"Mechanism for differentiating between ""by me"" and ""liked by me""",
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602862236,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602862236,MDEyOklzc3VlQ29tbWVudDYwMjg2MjIzNg==,9599,simonw,2020-03-23T21:20:26Z,2020-03-23T21:20:26Z,MEMBER,I'll run the `commits` and `issues` and `issue-comments` commands in addition to the `releases` command.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/twitter-to-sqlite/issues/21#issuecomment-542333836,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/21,542333836,MDEyOklzc3VlQ29tbWVudDU0MjMzMzgzNg==,9599,simonw,2019-10-15T18:00:48Z,2019-10-15T18:00:48Z,MEMBER,I'll use `html.unescape()` for this: https://docs.python.org/3/library/html.html#html.unescape,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506432572,Fix &amp; escapes in tweet text,
https://github.com/dogsheep/dogsheep-photos/issues/6#issuecomment-615979923,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6,615979923,MDEyOklzc3VlQ29tbWVudDYxNTk3OTkyMw==,9599,simonw,2020-04-18T23:36:02Z,2020-04-18T23:36:02Z,MEMBER,"I'll use a Click progress bar. To do this I need to first calculate the sum number of bytes in the photos that are going to be uploaded, then run the upload.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602575575,Add progress bar to upload command,
https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-623006004,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,623006004,MDEyOklzc3VlQ29tbWVudDYyMzAwNjAwNA==,9599,simonw,2020-05-02T20:00:26Z,2020-05-02T20:00:26Z,MEMBER,I'm abandoning this in favour of a new implementation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727,"stargazers command, refs #4",
https://github.com/dogsheep/github-to-sqlite/issues/26#issuecomment-614794739,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26,614794739,MDEyOklzc3VlQ29tbWVudDYxNDc5NDczOQ==,9599,simonw,2020-04-16T17:38:28Z,2020-04-16T17:38:28Z,MEMBER,I'm already doing this here: https://github.com/dogsheep/github-to-sqlite/blob/c4aaa50e167cfa9021c7c94260bc3e89e10947bf/github_to_sqlite/utils.py#L246-L250,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601271612,Topics are missing from repositories,
https://github.com/dogsheep/github-to-sqlite/issues/29#issuecomment-616883275,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/29,616883275,MDEyOklzc3VlQ29tbWVudDYxNjg4MzI3NQ==,9599,simonw,2020-04-21T00:43:28Z,2020-04-21T00:43:28Z,MEMBER,"I'm copying repo from issue, which surprisingly is a string, not an integer ID.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",603617013,Milestones should have foreign key to creator and repo,
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861042050,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861042050,MDEyOklzc3VlQ29tbWVudDg2MTA0MjA1MA==,9599,simonw,2021-06-14T22:45:42Z,2021-06-14T22:45:42Z,MEMBER,I'm definitely interested in supporting events in this tool - see #14.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216,"feature: support ""events""",
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902327457,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57,902327457,IC_kwDODEm0Qs41yGyh,9599,simonw,2021-08-19T23:53:25Z,2021-08-19T23:53:25Z,MEMBER,I'm getting this too. Looking into it now.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907645813,"Error: Use either --since or --since_id, not both",
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549226399,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3,549226399,MDEyOklzc3VlQ29tbWVudDU0OTIyNjM5OQ==,9599,simonw,2019-11-04T05:11:57Z,2019-11-04T05:11:57Z,MEMBER,I'm going to add a `hash` column to `search_runs` to support that. It's going to be the sha1 hash of the key-ordered JSON of the search arguments used by that run. Then `--since` can look for an identical hash and use it to identify the highest last fetched tweet to use in `since_id`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833975,Command for running a search and saving tweets for that search,
https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886140431,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2,886140431,IC_kwDODtX3eM400W4P,9599,simonw,2021-07-25T03:12:57Z,2021-07-25T03:12:57Z,MEMBER,"I'm going to build a general-purpose `hacker-new-to-sqlite search ...` command, where one of the options is to search within the URL.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",952179830,Command for fetching Hacker News threads from the search API,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932204,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615932204,MDEyOklzc3VlQ29tbWVudDYxNTkzMjIwNA==,9599,simonw,2020-04-18T19:29:22Z,2020-04-18T19:34:44Z,MEMBER,I'm going to call my bucket `dogsheep-photos-simon`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/twitter-to-sqlite/issues/2#issuecomment-527954898,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2,527954898,MDEyOklzc3VlQ29tbWVudDUyNzk1NDg5OA==,9599,simonw,2019-09-04T15:31:46Z,2019-09-04T15:31:46Z,MEMBER,I'm going to call this `twitter-to-sqlite user-timeline` to reflect the language used to describe the API endpoint: https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833698,"""twitter-to-sqlite user-timeline"" command for pulling tweets by a specific user",
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615941746,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615941746,MDEyOklzc3VlQ29tbWVudDYxNTk0MTc0Ng==,9599,simonw,2020-04-18T20:29:36Z,2020-04-18T20:29:36Z,MEMBER,"I'm going to create another user just for Transmit, with full S3 access.

name: `dogsheep-photos-simon-s3-all-access`

Rather than creating a group for that user, I'm trying the ""Attach existing policies directly"" option:

![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79670182-03513880-8176-11ea-811a-c80aefb4538a.png)

That user DID work with Transmit. I uploaded a test HEIC image. I used Transmit to copy a signed URL for it.

```
~ $ curl -i 'https://dogsheep-photos-simon.s3.us-west-1.amazonaws.com/IMG_7195.HEIC?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWXFXAI...' | head -n 100
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0HTTP/1.1 200 OK
x-amz-id-2: gBOCYqZfbNAnv0R/uJ++qm2NbW5SgD4TapgF9RQjzzeDIThcCz/BkKU+YoxlG4NJHlcmMgAHyh4=
x-amz-request-id: C2FE7FCC3BD53A84
Date: Sat, 18 Apr 2020 20:28:54 GMT
Last-Modified: Sat, 18 Apr 2020 20:13:49 GMT
ETag: ""fe3e081239a123ef745517878c53b854""
Accept-Ranges: bytes
Content-Type: image/heic
Content-Length: 1913097
Server: AmazonS3
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/github-to-sqlite/issues/21#issuecomment-602896434,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/21,602896434,MDEyOklzc3VlQ29tbWVudDYwMjg5NjQzNA==,9599,simonw,2020-03-23T22:43:37Z,2020-03-23T22:43:37Z,MEMBER,"I'm going to do this now to help figure out the latest error in #13:
```
Traceback (most recent call last):
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/bin/github-to-sqlite"", line 11, in <module>
    load_entry_point('github-to-sqlite', 'console_scripts', 'github-to-sqlite')()
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 237, in commits
    utils.save_commits(db, commits, repo_full[""id""])
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 345, in save_commits
    for commit in commits:
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 207, in fetch_commits
    if stop_when(commit):
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 224, in stop_when
    db[""commits""].get(commit[""sha""])
TypeError: string indices must be integers
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",586561727,Turn GitHub API errors into exceptions,
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602861730,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602861730,MDEyOklzc3VlQ29tbWVudDYwMjg2MTczMA==,9599,simonw,2020-03-23T21:19:16Z,2020-03-23T21:19:42Z,MEMBER,I'm going to do this with a scheduled GitHub Action in this repo.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-542876047,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3,542876047,MDEyOklzc3VlQ29tbWVudDU0Mjg3NjA0Nw==,9599,simonw,2019-10-16T20:23:36Z,2019-10-16T20:23:36Z,MEMBER,I'm going to go with `--since=1d/2w/3h` for this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",487600595,Option to fetch only checkins more recent than the current max checkin,
https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685966361,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7,685966361,MDEyOklzc3VlQ29tbWVudDY4NTk2NjM2MQ==,9599,simonw,2020-09-02T20:03:29Z,2020-09-02T20:03:41Z,MEMBER,"I'm going to implement the first version of this as an indexed integer `category` column which has 1 for ""about me"" and 2 for ""liked by me"" - and space for other category numerals in the future, albeit a row can only belong to one category.

I'll think about a full tagging system separately.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691265198,"Mechanism for differentiating between ""by me"" and ""liked by me""",
https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601876051,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35,601876051,MDEyOklzc3VlQ29tbWVudDYwMTg3NjA1MQ==,9599,simonw,2020-03-20T19:31:43Z,2020-03-20T19:31:43Z,MEMBER,I'm going to keep `--screen_name` working for backwards compatibility. Using `hidden=True` on it should hide it from the help output.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585282212,twitter-to-sqlite user-timeline [screen_names] --sql / --attach,
https://github.com/dogsheep/dogsheep-photos/issues/13#issuecomment-620309185,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/13,620309185,MDEyOklzc3VlQ29tbWVudDYyMDMwOTE4NQ==,9599,simonw,2020-04-28T00:39:45Z,2020-04-28T00:39:45Z,MEMBER,I'm going to leave this until I have the mechanism for associating a live photo video with the photo.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",607888367,Also upload movie files,
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831842,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27,614831842,MDEyOklzc3VlQ29tbWVudDYxNDgzMTg0Mg==,9599,simonw,2020-04-16T18:48:18Z,2020-04-16T18:48:18Z,MEMBER,I'm going to make `organization` another foreign key to the `users` table just in case it IS possible (maybe with GitHub Enterprise or similar?),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601330277,Repos have a big blob of JSON in the organization column,
https://github.com/dogsheep/dogsheep-beta/issues/25#issuecomment-695108895,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25,695108895,MDEyOklzc3VlQ29tbWVudDY5NTEwODg5NQ==,9599,simonw,2020-09-18T22:11:32Z,2020-09-18T22:11:32Z,MEMBER,"I'm going to make this a new plugin configuration setting, `template_debug`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",704685890,template_debug mechanism,
https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615509803,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37,615509803,MDEyOklzc3VlQ29tbWVudDYxNTUwOTgwMw==,9599,simonw,2020-04-17T23:36:40Z,2020-04-17T23:36:40Z,MEMBER,"I'm going to print a warning to stderr, skip and continue - because if you have 100 screen names and only one of them is invalid you should still execute for the other 99.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585353598,"Handle ""User not found"" error",
https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-623723026,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15,623723026,MDEyOklzc3VlQ29tbWVudDYyMzcyMzAyNg==,9599,simonw,2020-05-04T21:41:30Z,2020-05-04T21:41:30Z,MEMBER,"I'm going to put these in a table called `apple_photos_scores` - I'll also pull in the following columns from the `ZGENERICASSET` table:

* `ZOVERALLAESTHETICSCORE`
* `ZCURATIONSCORE`
* `ZHIGHLIGHTVISIBILITYSCORE`
* `ZPROMOTIONSCORE`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612151767,Expose scores from ZCOMPUTEDASSETATTRIBUTES,
https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601875405,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35,601875405,MDEyOklzc3VlQ29tbWVudDYwMTg3NTQwNQ==,9599,simonw,2020-03-20T19:30:09Z,2020-03-20T19:32:33Z,MEMBER,"I'm going to redesign the command a little. Right now it defaults to fetching the authenticated user's timeline and allows `--screen_name` to be used to fetch for one other user instead.

I think the design of `users-lookup` is much better. I'm going to copy that:

```
twitter-to-sqlite user-timeline users.db simonw cleopaws
twitter-to-sqlite user-timeline users.db 12497 3166449535 --ids
```
It will also accept `--since` and `--sql` and `--alter` and `--since_id`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585282212,twitter-to-sqlite user-timeline [screen_names] --sql / --attach,
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631227245,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631227245,MDEyOklzc3VlQ29tbWVudDYzMTIyNzI0NQ==,9599,simonw,2020-05-20T04:21:38Z,2020-05-20T04:21:38Z,MEMBER,I'm going to release 0.4 now.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/github-to-sqlite/issues/58#issuecomment-746734412,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58,746734412,MDEyOklzc3VlQ29tbWVudDc0NjczNDQxMg==,9599,simonw,2020-12-16T17:58:56Z,2020-12-16T17:58:56Z,MEMBER,"I'm going to rewrite those `<a href=""#filtering-tables"">` links to `<a href=""#user-content-filtering-tables"">` - but only if a corresponding `id=""user-content-filtering-tables""` element exists.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",769150394,Readme HTML has broken internal links,
https://github.com/dogsheep/healthkit-to-sqlite/issues/5#issuecomment-513514978,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5,513514978,MDEyOklzc3VlQ29tbWVudDUxMzUxNDk3OA==,9599,simonw,2019-07-21T02:55:12Z,2019-07-21T02:55:12Z,MEMBER,I'm going to show this by default. Users can pass `-s` or `--silent` to disable the progress bar.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",470691622,Add progress bar,
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693773191,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693773191,MDEyOklzc3VlQ29tbWVudDY5Mzc3MzE5MQ==,9599,simonw,2020-09-17T02:39:26Z,2020-09-17T02:39:26Z,MEMBER,I'm going to start with `github-to-sqlite get` and `github-to-sqlite post` - I may add `put` and suchlike later on.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615948102,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615948102,MDEyOklzc3VlQ29tbWVudDYxNTk0ODEwMg==,9599,simonw,2020-04-18T20:56:59Z,2020-04-18T20:56:59Z,MEMBER,"I'm going to start with this:

`photos-to-sqlite upload photos.db ~/path/to/directory`

This will scan the provided directory (and all sub-directories) for image files. It will then:

* Calculate a sha256 of the contents of that file
* Upload the file to a key that's `sha256.jpg` or `.heic`
* Upload a `sha256.json` file with the original path to the image
* Add that image to a `uploads` table in `photos.db`

Stretch goal: grab the EXIF data and include that in the `.json` upload AND the `uploads` database table.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695856398,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695856398,MDEyOklzc3VlQ29tbWVudDY5NTg1NjM5OA==,9599,simonw,2020-09-21T00:22:20Z,2020-09-21T00:22:20Z,MEMBER,I'm going to try for keyset pagination sorted by relevance just as a learning exercise.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination,
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623193947,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623193947,MDEyOklzc3VlQ29tbWVudDYyMzE5Mzk0Nw==,9599,simonw,2020-05-03T22:36:17Z,2020-05-03T22:36:17Z,MEMBER,"I'm going to use [osxphotos](https://github.com/RhetTbull/osxphotos) for this.

Since I've already got code to upload photos and insert them into a table based on their `sha256` hash, my first go at this will be to import data using the tool and foreign-key it to the `sha256` hash in the existing table.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-662626901,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48,662626901,MDEyOklzc3VlQ29tbWVudDY2MjYyNjkwMQ==,9599,simonw,2020-07-22T18:54:53Z,2020-07-22T18:54:53Z,MEMBER,I'm going to use a GitHub Action to run `npx markdown-toc README.md -i`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",663976976,Add a table of contents to the README,
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-607010791,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10,607010791,MDEyOklzc3VlQ29tbWVudDYwNzAxMDc5MQ==,9599,simonw,2020-04-01T03:45:48Z,2020-04-01T03:45:48Z,MEMBER,I'm happy with the recent work I did on this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",492297930,Rethink progress bars for various commands,
https://github.com/dogsheep/github-to-sqlite/issues/21#issuecomment-602920163,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/21,602920163,MDEyOklzc3VlQ29tbWVudDYwMjkyMDE2Mw==,9599,simonw,2020-03-23T23:48:22Z,2020-03-23T23:48:22Z,MEMBER,"I'm happy with this pattern:
https://github.com/dogsheep/github-to-sqlite/blob/f78c4e9baaf0970ffab266ba780df7240aae9f32/github_to_sqlite/utils.py#L4-L18","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",586561727,Turn GitHub API errors into exceptions,
https://github.com/dogsheep/dogsheep-beta/issues/18#issuecomment-688623097,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/18,688623097,MDEyOklzc3VlQ29tbWVudDY4ODYyMzA5Nw==,9599,simonw,2020-09-08T05:15:51Z,2020-09-08T05:15:51Z,MEMBER,"I'm inclined to go with the first, simpler option. I have longer term plans for efficient incremental index updates based on clever trickery with triggers.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",695553522,Deleted records stay in the search index,
https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-747034481,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29,747034481,MDEyOklzc3VlQ29tbWVudDc0NzAzNDQ4MQ==,9599,simonw,2020-12-16T21:17:05Z,2020-12-16T21:17:05Z,MEMBER,I'm just going to add `q` for the moment.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",724759588,Add search highlighting snippets,
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623000814,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623000814,MDEyOklzc3VlQ29tbWVudDYyMzAwMDgxNA==,9599,simonw,2020-05-02T19:15:23Z,2020-05-02T19:15:23Z,MEMBER,"I'm not going to do the `--sql` bit just yet. I have patterns for working around this for other commands which are working fine:

https://github.com/dogsheep/github-to-sqlite/blob/d00a53061556dc403c166b443d141c4e1adbd64a/.github/workflows/deploy-demo.yml#L53-L70","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-542854749,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20,542854749,MDEyOklzc3VlQ29tbWVudDU0Mjg1NDc0OQ==,9599,simonw,2019-10-16T19:26:01Z,2019-10-16T19:26:01Z,MEMBER,"I'm not going to do this for ""accounts that have followed me"" and ""new accounts that I have followed"" - instead I will recommend running the `friend_ids` and `followers_ids` commands on a daily basis since that data doesn't really change much by the hour.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506268945,--since support for various commands for refresh-by-cron,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790372621,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790372621,MDEyOklzc3VlQ29tbWVudDc5MDM3MjYyMQ==,9599,simonw,2021-03-04T07:01:18Z,2021-03-04T07:01:18Z,MEMBER,"I'm not sure if it would work, but there is an alternative pattern for showing a progress bar against a really large file that I've used in `healthkit-to-sqlite` - you set the progress bar size to the size of the file in bytes, then update a counter as you read the file.

https://github.com/dogsheep/healthkit-to-sqlite/blob/3eb2b06bfe3b4faaf10e9cf9dfcb28e3d16c14ff/healthkit_to_sqlite/cli.py#L24-L57 and https://github.com/dogsheep/healthkit-to-sqlite/blob/3eb2b06bfe3b4faaf10e9cf9dfcb28e3d16c14ff/healthkit_to_sqlite/utils.py#L4-L19 (the `progress_callback()` bit) is where that happens.

It can be a bit of a convoluted pattern, and I'm not at all sure it would work for `mbox` files since it looks like that library has other reasons it needs to do a file scan rather than streaming it through one chunk of bytes at a time. So I imagine this would not work here.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790695126,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790695126,MDEyOklzc3VlQ29tbWVudDc5MDY5NTEyNg==,9599,simonw,2021-03-04T15:20:42Z,2021-03-04T15:20:42Z,MEMBER,"I'm not sure why but my most recent import, when displayed in Datasette, looks like this:

<img width=""574"" alt=""mbox__mbox_emails__753_446_rows"" src=""https://user-images.githubusercontent.com/9599/109985836-0ab00080-7cba-11eb-97d5-0631a0835b61.png"">

Sorting by `id` in the opposite order gives me the data I would expect - so it looks like a bunch of null/blank messages are being imported at some point and showing up first due to ID ordering.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686688963,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686688963,MDEyOklzc3VlQ29tbWVudDY4NjY4ODk2Mw==,9599,simonw,2020-09-03T18:42:59Z,2020-09-03T18:42:59Z,MEMBER,I'm pleased with how this works now.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623807568,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623807568,MDEyOklzc3VlQ29tbWVudDYyMzgwNzU2OA==,9599,simonw,2020-05-05T02:56:06Z,2020-05-05T02:56:06Z,MEMBER,"I'm pretty sure this is what I'm after. The `groups` table has what looks like identified labels in the rows with category = 2025:

<img width=""1122"" alt=""words__groups__2_528_rows_where_where_category___2025"" src=""https://user-images.githubusercontent.com/9599/81031361-e0df4080-8e40-11ea-9060-6d850aa52140.png"">

Then there's a `ga` table that maps groups to assets:

<img width=""304"" alt=""words__ga__633_653_rows"" src=""https://user-images.githubusercontent.com/9599/81031387-f48aa700-8e40-11ea-9a3d-da23903be928.png"">

And an `assets` table which looks like it has one row for every one of my photos:

<img width=""645"" alt=""words__assets__40_419_rows"" src=""https://user-images.githubusercontent.com/9599/81031402-04a28680-8e41-11ea-8047-e9199d068563.png"">

One major challenge: these UUIDs are split into two integer numbers, `uuid_0` and `uuid_1` - but the main photos database uses regular UUIDs like this:

![image](https://user-images.githubusercontent.com/9599/81031481-39164280-8e41-11ea-983b-005ced641a18.png)

I need to figure out how to match up these two different UUID representations. I asked on Twitter if anyone has any ideas: https://twitter.com/simonw/status/1257500689019703296","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/github-to-sqlite/issues/16#issuecomment-601979709,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16,601979709,MDEyOklzc3VlQ29tbWVudDYwMTk3OTcwOQ==,9599,simonw,2020-03-21T02:20:08Z,2020-03-21T02:20:08Z,MEMBER,I'm pretty sure this means I need to upgrade the underlying `sqlite-utils` dependency. I saw the same bug in `twitter-to-sqlite`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",546051181,Exception running first command: IndexError: list index out of range,
https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514496725,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7,514496725,MDEyOklzc3VlQ29tbWVudDUxNDQ5NjcyNQ==,9599,simonw,2019-07-24T06:20:59Z,2019-07-24T06:20:59Z,MEMBER,"I'm using https://pypi.org/project/memory-profiler/ to explore this in more detail:

```
$ pip install memory-profiler matplotlib
```

Then:

```
$ mprof run healthkit-to-sqlite ~/Downloads/healthkit-export.zip healthkit.db
$ mprof plot
```

<img width=""1128"" alt=""Screen Shot 2019-07-24 at 8 17 06 AM"" src=""https://user-images.githubusercontent.com/9599/61769849-f3294a00-adeb-11e9-9026-02c24209530f.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",472097220,Script uses a lot of RAM,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607003655,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,607003655,MDEyOklzc3VlQ29tbWVudDYwNzAwMzY1NQ==,9599,simonw,2020-04-01T03:18:00Z,2020-04-01T03:18:00Z,MEMBER,I've got this working for the `user-timeline` command.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/evernote-to-sqlite/pull/10#issuecomment-777839351,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/10,777839351,MDEyOklzc3VlQ29tbWVudDc3NzgzOTM1MQ==,9599,simonw,2021-02-11T22:37:55Z,2021-02-11T22:37:55Z,MEMBER,"I've merged these changes by hand now, thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",770712149,BugFix for encoding and not update info.,
https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645599881,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47,645599881,MDEyOklzc3VlQ29tbWVudDY0NTU5OTg4MQ==,9599,simonw,2020-06-17T20:13:48Z,2020-06-17T20:13:48Z,MEMBER,"I've now figured out how to compile specific SQLite versions to help replicate this problem: https://github.com/simonw/til/blob/master/sqlite/ld-preload.md

Next step: replicate the problem!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639542974,Fall back to FTS4 if FTS5 is not available,
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695879237,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695879237,MDEyOklzc3VlQ29tbWVudDY5NTg3OTIzNw==,9599,simonw,2020-09-21T02:53:29Z,2020-09-21T02:53:29Z,MEMBER,"If previous page ended at `2018-02-11T16:32:53+00:00`:
```sql
select
  search_index.rowid,
  search_index.type,
  search_index.key,
  search_index.title,
  search_index.category,
  search_index.timestamp,
  search_index.search_1
from
  search_index
 where 
  date(""timestamp"") = '2018-02-11'
 and timestamp < '2018-02-11T16:32:53+00:00'
order by
  search_index.timestamp desc, rowid
limit 41
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination,
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686154627,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686154627,MDEyOklzc3VlQ29tbWVudDY4NjE1NDYyNw==,9599,simonw,2020-09-03T00:19:22Z,2020-09-03T00:19:22Z,MEMBER,If this performs well enough (100 displayed items will be 100 extra `display_sql` calls) then I'll go with this as the design for the feature.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686158454,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686158454,MDEyOklzc3VlQ29tbWVudDY4NjE1ODQ1NA==,9599,simonw,2020-09-03T00:32:42Z,2020-09-03T00:32:42Z,MEMBER,"If this turns out to be too inefficient I could add a `display` text column to the `search_index` table which is designed to be populated with arbitrary JSON by the indexing query, which can then be used to render the template fragment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/hacker-news-to-sqlite/issues/3#issuecomment-886237834,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3,886237834,IC_kwDODtX3eM400uqK,9599,simonw,2021-07-25T18:05:32Z,2021-07-25T18:05:32Z,MEMBER,"If you hit the endpoint for a comment that's part of a thread you get that comment and its recursive children: https://hn.algolia.com/api/v1/items/27941552

You can tell that it's not the top-level because the `parent_id` isn't `null`. You can use `story_id` to figure out what the top-level item is.

```json
{
  ""id"": 27941552,
  ""created_at"": ""2021-07-24T15:08:39.000Z"",
  ""created_at_i"": 1627139319,
  ""type"": ""comment"",
  ""author"": ""nine_k"",
  ""title"": null,
  ""url"": null,
  ""text"": ""<p>I wish ..."",
  ""points"": null,
  ""parent_id"": 27941108,
  ""story_id"": 27941108
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",952189173,Use HN algolia endpoint to retrieve trees,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/7#issuecomment-1462691466,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7,1462691466,IC_kwDOJHON9s5XLuaK,9599,simonw,2023-03-09T19:59:52Z,2023-03-09T19:59:52Z,MEMBER,"Improved script:
```zsh
osascript -e 'tell application ""Notes""
    set allFolders to folders
    repeat with aFolder in allFolders
        set folderId to id of aFolder
        set folderName to name of aFolder
        set folderContainer to container of aFolder
        if class of folderContainer is folder then
            set folderContainerId to id of folderContainer
        else
            set folderContainerId to """"
        end if
        log ""ID: "" & folderId
        log ""Name: "" & folderName
        log ""Container: "" & folderContainerId
        log "" ""
    end repeat
end tell
'
```
```
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6113
Name: Blog posts
Container: 
 
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p698
Name: JSK
Container: 
 
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p7995
Name: Nested inside blog posts
Container: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6113
 
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p3526
Name: New Folder
Container: 
 
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p3839
Name: New Folder 1
Container: 
 
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p2
Name: Notes
Container: 
 
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p6059
Name: Quick Notes
Container: 
 
ID: x-coredata://D2D50498-BBD1-4097-B122-D15ABD32BDEC/ICFolder/p7283
Name: UK Christmas 2022
Container: 
```
I filtered out things where the parent was an account and not a folder using `if class of folderContainer is folder then`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1617769847,Folder support,
https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-544335363,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20,544335363,MDEyOklzc3VlQ29tbWVudDU0NDMzNTM2Mw==,9599,simonw,2019-10-21T03:32:04Z,2019-10-21T03:32:04Z,MEMBER,"In case anyone is interested, here's an extract from the crontab I'm running these under at the moment:
```
1,11,21,31,41,51 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite user-timeline /home/ubuntu/twitter.db -a /home/ubuntu/auth.json --since
2,7,12,17,22,27,32,37,42,47,52,57 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite home-timeline /home/ubuntu/timeline.db -a /home/ubuntu/auth.json --since
6,16,26,36,46,56 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite favorites /home/ubuntu/twitter.db -a /home/ubuntu/auth.json --stop_after=50
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506268945,--since support for various commands for refresh-by-cron,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623806085,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623806085,MDEyOklzc3VlQ29tbWVudDYyMzgwNjA4NQ==,9599,simonw,2020-05-05T02:47:18Z,2020-05-05T02:47:18Z,MEMBER,"In https://github.com/RhetTbull/osxphotos/issues/121#issuecomment-623249263 Rhet Turnbull spotted a table called `ZSCENEIDENTIFIER` which looked like it might have the right data, but the columns in it aren't particularly helpful:
```
Z_PK,Z_ENT,Z_OPT,ZSCENEIDENTIFIER,ZASSETATTRIBUTES,ZCONFIDENCE
8,49,1,731,5,0.11834716796875
9,49,1,684,6,0.0233648251742125
10,49,1,1702,1,0.026153564453125
```
I love the look of those confidence scores, but what do the numbers mean?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541119834,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18,541119834,MDEyOklzc3VlQ29tbWVudDU0MTExOTgzNA==,9599,simonw,2019-10-11T15:51:22Z,2019-10-11T16:51:33Z,MEMBER,"In order to support multiple user timelines being saved in the same database, I'm going to import the tweets into the `tweets` table AND add a new `timeline_tweets` table recording that a specific tweet showed up in a specific user's timeline.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",505928530,Command to import home-timeline,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/4#issuecomment-1462556829,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/4,1462556829,IC_kwDOJHON9s5XLNid,9599,simonw,2023-03-09T18:20:56Z,2023-03-09T18:20:56Z,MEMBER,"In terms of the UI: I'm tempted to say that the default behaviour is for it to run until it sees a note that it already knows about AND that has matching update/created dates, and then stop.

You can do a full import again ignoring that logic with `apple-notes-to-sqlite notes.db --full`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616429236,Support incremental updates,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-607019151,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,607019151,MDEyOklzc3VlQ29tbWVudDYwNzAxOTE1MQ==,9599,simonw,2020-04-01T04:11:10Z,2020-04-01T04:11:10Z,MEMBER,"In testing this collects a LOT of data. I'm going to skip tracking favourites_count and statuses_count and just track followers, friends and listed instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,
https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735464438,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54,735464438,MDEyOklzc3VlQ29tbWVudDczNTQ2NDQzOA==,9599,simonw,2020-11-29T21:57:08Z,2020-11-29T21:57:08Z,MEMBER,Inspired by this tweet from Michael Heap https://twitter.com/mheap/status/1333108608817631238,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753026003,github-to-sqlite workflows command,
https://github.com/dogsheep/dogsheep-photos/issues/22#issuecomment-626941278,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/22,626941278,MDEyOklzc3VlQ29tbWVudDYyNjk0MTI3OA==,9599,simonw,2020-05-11T20:25:58Z,2020-05-11T20:25:58Z,MEMBER,"Interesting - do you know if there's anything the `exiftool` process handles that `ExifReader` doesn't?

I'm actually just going to extract a subset of the EXIF data at first - since the original photo files will always be available I don't feel the need to get everything out for the first step.

My plan is to use EXIF to help support photo collections that aren't in Apple Photos - I'm going to build a database table keyed by the `sha256` of each photo that extracts the camera make, lens, a few settings (ISO, aperture etc) and the GPS lat/lon.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615626118,Try out ExifReader,
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614831451,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27,614831451,MDEyOklzc3VlQ29tbWVudDYxNDgzMTQ1MQ==,9599,simonw,2020-04-16T18:47:25Z,2020-04-16T18:47:25Z,MEMBER,Is it possible for a repo to have an `owner` that differs from its `organization`?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601330277,Repos have a big blob of JSON in the organization column,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461262577,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2,1461262577,IC_kwDOJHON9s5XGRjx,9599,simonw,2023-03-09T04:30:00Z,2023-03-09T04:30:00Z,MEMBER,It doesn't have tests yet. I guess I'll need to mock `subprocess` to test this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616354999,First working version,
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695855723,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695855723,MDEyOklzc3VlQ29tbWVudDY5NTg1NTcyMw==,9599,simonw,2020-09-21T00:16:52Z,2020-09-21T00:17:53Z,MEMBER,"It feels a bit weird to implement keyset pagination against results sorted by `rank` because the ranks could change substantially if the search index gets updated while the user is paginating.

I may just ignore that though. If you want reliable pagination you can get it by sorting by date. Maybe it doesn't even make sense to offer pagination if you sort by relevance?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination,
https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550783316,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10,550783316,MDEyOklzc3VlQ29tbWVudDU1MDc4MzMxNg==,9599,simonw,2019-11-07T05:16:56Z,2019-11-07T05:34:29Z,MEMBER,"It looks like Apple changed the location of these in iOS 13 - they are now in separate `.gpx` files:

![2FF70E95-CDEE-4241-A5C5-EE95A862E519](https://user-images.githubusercontent.com/9599/68362042-be12e000-00da-11ea-8925-7397410332d8.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",519038979,Failed to import workout points,
https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-623723687,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15,623723687,MDEyOklzc3VlQ29tbWVudDYyMzcyMzY4Nw==,9599,simonw,2020-05-04T21:43:06Z,2020-05-04T21:43:06Z,MEMBER,It looks like I can map the photos I'm importing to these tables using the `ZUUID` column on `ZGENERICASSET` to get a `Z_PK` which then maps to the rows in `ZGENERICASSET`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612151767,Expose scores from ZCOMPUTEDASSETATTRIBUTES,
https://github.com/dogsheep/pocket-to-sqlite/issues/3#issuecomment-684424396,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3,684424396,MDEyOklzc3VlQ29tbWVudDY4NDQyNDM5Ng==,9599,simonw,2020-09-01T06:17:45Z,2020-09-01T06:17:45Z,MEMBER,It looks like I could ignore the `image` column and synthesize a unique key from the data in the `images` column using `$item_id/$image_id`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503243784,Extract images into separate tables,
https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-906635938,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13,906635938,IC_kwDOEhK-wc42Ciqi,9599,simonw,2021-08-26T18:18:27Z,2021-08-26T18:18:27Z,MEMBER,"It looks like I was using the round-trip to dump the `<?xml version=""1.0"" encoding=""UTF-8"" standalone=""no""?>` and `<!DOCTYPE` prefixes.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",978743426,xml.etree.ElementTree.ParseError: not well-formed (invalid token),
https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095641,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27,549095641,MDEyOklzc3VlQ29tbWVudDU0OTA5NTY0MQ==,9599,simonw,2019-11-03T01:12:58Z,2019-11-03T01:12:58Z,MEMBER,"It looks like Twitter really want you to subscribe to a premium API for this kind of thing and consume retweets via webhooks: https://developer.twitter.com/en/docs/accounts-and-users/subscribe-account-activity/api-reference

I'm going to give up on this for the moment.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",514459062,retweets-of-me command,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623865250,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623865250,MDEyOklzc3VlQ29tbWVudDYyMzg2NTI1MA==,9599,simonw,2020-05-05T05:38:16Z,2020-05-05T05:38:16Z,MEMBER,It looks like `groups.content_string` often has a null byte in it. I should clean this up as part of the import.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790378658,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790378658,MDEyOklzc3VlQ29tbWVudDc5MDM3ODY1OA==,9599,simonw,2021-03-04T07:12:48Z,2021-03-04T07:12:48Z,MEMBER,"It looks like the `body` is being loaded into a BLOB column - so in Datasette default it looks like this:

<img width=""1650"" alt=""mbox__mbox_emails__753_446_rows"" src=""https://user-images.githubusercontent.com/9599/109924808-b4b96980-7c75-11eb-8c9e-307f2ae32d5a.png"">

If I `datasette install datasette-render-binary` and then try again I get this:

<img width=""1487"" alt=""mbox__mbox_emails__753_446_rows"" src=""https://user-images.githubusercontent.com/9599/109924944-ea5e5280-7c75-11eb-9a32-404f3d68455f.png"">

It would be great if we could store the `body` as unicode text instead. May have to do something clever to decode it based on some kind of charset header?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/4#issuecomment-1462554175,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/4,1462554175,IC_kwDOJHON9s5XLM4_,9599,simonw,2023-03-09T18:19:34Z,2023-03-09T18:19:34Z,MEMBER,It looks like the iteration order is most-recently-modified-first - I tried editing a note a bit further back in my notes app and it was the first one output by `apple-notes-to-sqlite --dump`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616429236,Support incremental updates,
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622133422,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622133422,MDEyOklzc3VlQ29tbWVudDYyMjEzMzQyMg==,9599,simonw,2020-04-30T21:48:39Z,2020-04-30T21:48:39Z,MEMBER,It looks like the only option is to scrape them. I'll do that and then replace with an API as soon as one becomes available.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,Command for retrieving dependents for a repo,
https://github.com/dogsheep/evernote-to-sqlite/issues/6#issuecomment-706785201,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6,706785201,MDEyOklzc3VlQ29tbWVudDcwNjc4NTIwMQ==,9599,simonw,2020-10-11T23:29:39Z,2020-10-11T23:29:39Z,MEMBER,"It looks to me like each of those `<item>` blocks has a number of guesses in order of confidence:
```xml
  <item x=""215"" y=""190"" w=""187"" h=""39"">
    <t w=""57"">wonders,</t>
    <t w=""55"">wanders,</t>
    <t w=""52"">wonders ?</t>
    <t w=""45"">wonders</t>
    <t w=""42"">wonders.</t>
  </item>
```
So maybe the best approach here is to just take the first `t` element within each `item`.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718949182,Better handling of OCR data,
https://github.com/dogsheep/healthkit-to-sqlite/issues/5#issuecomment-513625406,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5,513625406,MDEyOklzc3VlQ29tbWVudDUxMzYyNTQwNg==,9599,simonw,2019-07-22T03:20:16Z,2019-07-22T03:20:16Z,MEMBER,"It now renders like this:
```
Importing from HealthKit  [#-----------------------------------]    5%  00:01:33
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",470691622,Add progress bar,
https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735483604,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735483604,MDEyOklzc3VlQ29tbWVudDczNTQ4MzYwNA==,9599,simonw,2020-11-30T00:26:50Z,2020-11-30T00:26:50Z,MEMBER,"It seems like there's a lot missing from that - those aren't particularly interesting given the data that is returned.

From the docs at https://docs.github.com/en/free-pro-team@latest/rest/reference/pulls#reviews it looks like each review consists of multiple comments, and the comments are where the useful material is - https://docs.github.com/en/free-pro-team@latest/rest/reference/pulls#list-comments-for-a-pull-request-review

`github-to-sqlite get https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48/reviews/503368921/comments --accept 'application/vnd.github.v3+json'`

```json
[
    {
        ""id"": 500603838,
        ""node_id"": ""MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDUwMDYwMzgzOA=="",
        ""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500603838"",
        ""pull_request_review_id"": 503368921,
        ""diff_hunk"": ""@@ -0,0 +1,370 @@\n+[\n+    {\n+        \""url\"": \""https://api.github.com/repos/simonw/datasette/pulls/571\"",\n+        \""id\"": 313384926,\n+        \""node_id\"": \""MDExOlB1bGxSZXF1ZXN0MzEzMzg0OTI2\"",\n+        \""html_url\"": \""https://github.com/simonw/datasette/pull/571\"",\n+        \""diff_url\"": \""https://github.com/simonw/datasette/pull/571.diff\"",\n+        \""patch_url\"": \""https://github.com/simonw/datasette/pull/571.patch\"",\n+        \""issue_url\"": \""https://api.github.com/repos/simonw/datasette/issues/571\"",\n+        \""number\"": 571,\n+        \""state\"": \""closed\"",\n+        \""locked\"": false,\n+        \""title\"": \""detect_fts now works with alternative table escaping\"",\n+        \""user\"": {\n+          \""login\"": \""simonw\"",\n+          \""id\"": 9599,\n+          \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+          \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+          \""gravatar_id\"": \""\"",\n+          \""url\"": \""https://api.github.com/users/simonw\"",\n+          \""html_url\"": \""https://github.com/simonw\"",\n+          \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+          \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+          \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+          \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+          \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+          \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+          \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+          \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+          \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+          \""type\"": \""User\"",\n+          \""site_admin\"": false\n+        },\n+        \""body\"": \""Fixes #570\"",\n+        \""created_at\"": \""2019-09-03T00:23:39Z\"",\n+        \""updated_at\"": \""2019-09-03T00:32:28Z\"",\n+        \""closed_at\"": \""2019-09-03T00:32:28Z\"",\n+        \""merged_at\"": \""2019-09-03T00:32:28Z\"",\n+        \""merge_commit_sha\"": \""2dc5c8dc259a0606162673d394ba8cc1c6f54428\"",\n+        \""assignee\"": null,\n+        \""assignees\"": [\n+\n+        ],\n+        \""requested_reviewers\"": [\n+\n+        ],\n+        \""requested_teams\"": [\n+\n+        ],\n+        \""labels\"": [\n+\n+        ],\n+        \""milestone\"": null,\n+        \""draft\"": false,\n+        \""commits_url\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/commits\"",\n+        \""review_comments_url\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/comments\"",\n+        \""review_comment_url\"": \""https://api.github.com/repos/simonw/datasette/pulls/comments{/number}\"",\n+        \""comments_url\"": \""https://api.github.com/repos/simonw/datasette/issues/571/comments\"",\n+        \""statuses_url\"": \""https://api.github.com/repos/simonw/datasette/statuses/a85239f69261c10f1a9f90514c8b5d113cb94585\"",\n+        \""head\"": {\n+          \""label\"": \""simonw:detect-fts\"",\n+          \""ref\"": \""detect-fts\"",\n+          \""sha\"": \""a85239f69261c10f1a9f90514c8b5d113cb94585\"",\n+          \""user\"": {\n+            \""login\"": \""simonw\"",\n+            \""id\"": 9599,\n+            \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+            \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+            \""gravatar_id\"": \""\"",\n+            \""url\"": \""https://api.github.com/users/simonw\"",\n+            \""html_url\"": \""https://github.com/simonw\"",\n+            \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+            \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+            \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+            \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+            \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+            \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+            \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+            \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+            \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+            \""type\"": \""User\"",\n+            \""site_admin\"": false\n+          },\n+          \""repo\"": {\n+            \""id\"": 107914493,\n+            \""node_id\"": \""MDEwOlJlcG9zaXRvcnkxMDc5MTQ0OTM=\"",\n+            \""name\"": \""datasette\"",\n+            \""full_name\"": \""simonw/datasette\"",\n+            \""private\"": false,\n+            \""owner\"": {\n+              \""login\"": \""simonw\"",\n+              \""id\"": 9599,\n+              \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+              \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+              \""gravatar_id\"": \""\"",\n+              \""url\"": \""https://api.github.com/users/simonw\"",\n+              \""html_url\"": \""https://github.com/simonw\"",\n+              \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+              \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+              \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+              \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+              \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+              \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+              \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+              \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+              \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+              \""type\"": \""User\"",\n+              \""site_admin\"": false\n+            },\n+            \""html_url\"": \""https://github.com/simonw/datasette\"",\n+            \""description\"": \""An open source multi-tool for exploring and publishing data\"",\n+            \""fork\"": false,\n+            \""url\"": \""https://api.github.com/repos/simonw/datasette\"",\n+            \""forks_url\"": \""https://api.github.com/repos/simonw/datasette/forks\"",\n+            \""keys_url\"": \""https://api.github.com/repos/simonw/datasette/keys{/key_id}\"",\n+            \""collaborators_url\"": \""https://api.github.com/repos/simonw/datasette/collaborators{/collaborator}\"",\n+            \""teams_url\"": \""https://api.github.com/repos/simonw/datasette/teams\"",\n+            \""hooks_url\"": \""https://api.github.com/repos/simonw/datasette/hooks\"",\n+            \""issue_events_url\"": \""https://api.github.com/repos/simonw/datasette/issues/events{/number}\"",\n+            \""events_url\"": \""https://api.github.com/repos/simonw/datasette/events\"",\n+            \""assignees_url\"": \""https://api.github.com/repos/simonw/datasette/assignees{/user}\"",\n+            \""branches_url\"": \""https://api.github.com/repos/simonw/datasette/branches{/branch}\"",\n+            \""tags_url\"": \""https://api.github.com/repos/simonw/datasette/tags\"",\n+            \""blobs_url\"": \""https://api.github.com/repos/simonw/datasette/git/blobs{/sha}\"",\n+            \""git_tags_url\"": \""https://api.github.com/repos/simonw/datasette/git/tags{/sha}\"",\n+            \""git_refs_url\"": \""https://api.github.com/repos/simonw/datasette/git/refs{/sha}\"",\n+            \""trees_url\"": \""https://api.github.com/repos/simonw/datasette/git/trees{/sha}\"",\n+            \""statuses_url\"": \""https://api.github.com/repos/simonw/datasette/statuses/{sha}\"",\n+            \""languages_url\"": \""https://api.github.com/repos/simonw/datasette/languages\"",\n+            \""stargazers_url\"": \""https://api.github.com/repos/simonw/datasette/stargazers\"",\n+            \""contributors_url\"": \""https://api.github.com/repos/simonw/datasette/contributors\"",\n+            \""subscribers_url\"": \""https://api.github.com/repos/simonw/datasette/subscribers\"",\n+            \""subscription_url\"": \""https://api.github.com/repos/simonw/datasette/subscription\"",\n+            \""commits_url\"": \""https://api.github.com/repos/simonw/datasette/commits{/sha}\"",\n+            \""git_commits_url\"": \""https://api.github.com/repos/simonw/datasette/git/commits{/sha}\"",\n+            \""comments_url\"": \""https://api.github.com/repos/simonw/datasette/comments{/number}\"",\n+            \""issue_comment_url\"": \""https://api.github.com/repos/simonw/datasette/issues/comments{/number}\"",\n+            \""contents_url\"": \""https://api.github.com/repos/simonw/datasette/contents/{+path}\"",\n+            \""compare_url\"": \""https://api.github.com/repos/simonw/datasette/compare/{base}...{head}\"",\n+            \""merges_url\"": \""https://api.github.com/repos/simonw/datasette/merges\"",\n+            \""archive_url\"": \""https://api.github.com/repos/simonw/datasette/{archive_format}{/ref}\"",\n+            \""downloads_url\"": \""https://api.github.com/repos/simonw/datasette/downloads\"",\n+            \""issues_url\"": \""https://api.github.com/repos/simonw/datasette/issues{/number}\"",\n+            \""pulls_url\"": \""https://api.github.com/repos/simonw/datasette/pulls{/number}\"",\n+            \""milestones_url\"": \""https://api.github.com/repos/simonw/datasette/milestones{/number}\"",\n+            \""notifications_url\"": \""https://api.github.com/repos/simonw/datasette/notifications{?since,all,participating}\"",\n+            \""labels_url\"": \""https://api.github.com/repos/simonw/datasette/labels{/name}\"",\n+            \""releases_url\"": \""https://api.github.com/repos/simonw/datasette/releases{/id}\"",\n+            \""deployments_url\"": \""https://api.github.com/repos/simonw/datasette/deployments\"",\n+            \""created_at\"": \""2017-10-23T00:39:03Z\"",\n+            \""updated_at\"": \""2020-07-27T20:42:15Z\"",\n+            \""pushed_at\"": \""2020-07-26T01:21:05Z\"",\n+            \""git_url\"": \""git://github.com/simonw/datasette.git\"",\n+            \""ssh_url\"": \""git@github.com:simonw/datasette.git\"",\n+            \""clone_url\"": \""https://github.com/simonw/datasette.git\"",\n+            \""svn_url\"": \""https://github.com/simonw/datasette\"",\n+            \""homepage\"": \""http://datasette.readthedocs.io/\"",\n+            \""size\"": 3487,\n+            \""stargazers_count\"": 3642,\n+            \""watchers_count\"": 3642,\n+            \""language\"": \""Python\"",\n+            \""has_issues\"": true,\n+            \""has_projects\"": false,\n+            \""has_downloads\"": true,\n+            \""has_wiki\"": true,\n+            \""has_pages\"": false,\n+            \""forks_count\"": 206,\n+            \""mirror_url\"": null,\n+            \""archived\"": false,\n+            \""disabled\"": false,\n+            \""open_issues_count\"": 190,\n+            \""license\"": {\n+              \""key\"": \""apache-2.0\"",\n+              \""name\"": \""Apache License 2.0\"",\n+              \""spdx_id\"": \""Apache-2.0\"",\n+              \""url\"": \""https://api.github.com/licenses/apache-2.0\"",\n+              \""node_id\"": \""MDc6TGljZW5zZTI=\""\n+            },\n+            \""forks\"": 206,\n+            \""open_issues\"": 190,\n+            \""watchers\"": 3642,\n+            \""default_branch\"": \""master\""\n+          }\n+        },\n+        \""base\"": {\n+          \""label\"": \""simonw:master\"",\n+          \""ref\"": \""master\"",\n+          \""sha\"": \""f04deebec4f3842f7bd610cd5859de529f77d50e\"",\n+          \""user\"": {\n+            \""login\"": \""simonw\"",\n+            \""id\"": 9599,\n+            \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+            \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+            \""gravatar_id\"": \""\"",\n+            \""url\"": \""https://api.github.com/users/simonw\"",\n+            \""html_url\"": \""https://github.com/simonw\"",\n+            \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+            \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+            \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+            \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+            \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+            \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+            \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+            \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+            \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+            \""type\"": \""User\"",\n+            \""site_admin\"": false\n+          },\n+          \""repo\"": {\n+            \""id\"": 107914493,\n+            \""node_id\"": \""MDEwOlJlcG9zaXRvcnkxMDc5MTQ0OTM=\"",\n+            \""name\"": \""datasette\"",\n+            \""full_name\"": \""simonw/datasette\"",\n+            \""private\"": false,\n+            \""owner\"": {\n+              \""login\"": \""simonw\"",\n+              \""id\"": 9599,\n+              \""node_id\"": \""MDQ6VXNlcjk1OTk=\"",\n+              \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"",\n+              \""gravatar_id\"": \""\"",\n+              \""url\"": \""https://api.github.com/users/simonw\"",\n+              \""html_url\"": \""https://github.com/simonw\"",\n+              \""followers_url\"": \""https://api.github.com/users/simonw/followers\"",\n+              \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"",\n+              \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"",\n+              \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"",\n+              \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"",\n+              \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"",\n+              \""repos_url\"": \""https://api.github.com/users/simonw/repos\"",\n+              \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"",\n+              \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"",\n+              \""type\"": \""User\"",\n+              \""site_admin\"": false\n+            },\n+            \""html_url\"": \""https://github.com/simonw/datasette\"",\n+            \""description\"": \""An open source multi-tool for exploring and publishing data\"",\n+            \""fork\"": false,\n+            \""url\"": \""https://api.github.com/repos/simonw/datasette\"",\n+            \""forks_url\"": \""https://api.github.com/repos/simonw/datasette/forks\"",\n+            \""keys_url\"": \""https://api.github.com/repos/simonw/datasette/keys{/key_id}\"",\n+            \""collaborators_url\"": \""https://api.github.com/repos/simonw/datasette/collaborators{/collaborator}\"",\n+            \""teams_url\"": \""https://api.github.com/repos/simonw/datasette/teams\"",\n+            \""hooks_url\"": \""https://api.github.com/repos/simonw/datasette/hooks\"",\n+            \""issue_events_url\"": \""https://api.github.com/repos/simonw/datasette/issues/events{/number}\"",\n+            \""events_url\"": \""https://api.github.com/repos/simonw/datasette/events\"",\n+            \""assignees_url\"": \""https://api.github.com/repos/simonw/datasette/assignees{/user}\"",\n+            \""branches_url\"": \""https://api.github.com/repos/simonw/datasette/branches{/branch}\"",\n+            \""tags_url\"": \""https://api.github.com/repos/simonw/datasette/tags\"",\n+            \""blobs_url\"": \""https://api.github.com/repos/simonw/datasette/git/blobs{/sha}\"",\n+            \""git_tags_url\"": \""https://api.github.com/repos/simonw/datasette/git/tags{/sha}\"",\n+            \""git_refs_url\"": \""https://api.github.com/repos/simonw/datasette/git/refs{/sha}\"",\n+            \""trees_url\"": \""https://api.github.com/repos/simonw/datasette/git/trees{/sha}\"",\n+            \""statuses_url\"": \""https://api.github.com/repos/simonw/datasette/statuses/{sha}\"",\n+            \""languages_url\"": \""https://api.github.com/repos/simonw/datasette/languages\"",\n+            \""stargazers_url\"": \""https://api.github.com/repos/simonw/datasette/stargazers\"",\n+            \""contributors_url\"": \""https://api.github.com/repos/simonw/datasette/contributors\"",\n+            \""subscribers_url\"": \""https://api.github.com/repos/simonw/datasette/subscribers\"",\n+            \""subscription_url\"": \""https://api.github.com/repos/simonw/datasette/subscription\"",\n+            \""commits_url\"": \""https://api.github.com/repos/simonw/datasette/commits{/sha}\"",\n+            \""git_commits_url\"": \""https://api.github.com/repos/simonw/datasette/git/commits{/sha}\"",\n+            \""comments_url\"": \""https://api.github.com/repos/simonw/datasette/comments{/number}\"",\n+            \""issue_comment_url\"": \""https://api.github.com/repos/simonw/datasette/issues/comments{/number}\"",\n+            \""contents_url\"": \""https://api.github.com/repos/simonw/datasette/contents/{+path}\"",\n+            \""compare_url\"": \""https://api.github.com/repos/simonw/datasette/compare/{base}...{head}\"",\n+            \""merges_url\"": \""https://api.github.com/repos/simonw/datasette/merges\"",\n+            \""archive_url\"": \""https://api.github.com/repos/simonw/datasette/{archive_format}{/ref}\"",\n+            \""downloads_url\"": \""https://api.github.com/repos/simonw/datasette/downloads\"",\n+            \""issues_url\"": \""https://api.github.com/repos/simonw/datasette/issues{/number}\"",\n+            \""pulls_url\"": \""https://api.github.com/repos/simonw/datasette/pulls{/number}\"",\n+            \""milestones_url\"": \""https://api.github.com/repos/simonw/datasette/milestones{/number}\"",\n+            \""notifications_url\"": \""https://api.github.com/repos/simonw/datasette/notifications{?since,all,participating}\"",\n+            \""labels_url\"": \""https://api.github.com/repos/simonw/datasette/labels{/name}\"",\n+            \""releases_url\"": \""https://api.github.com/repos/simonw/datasette/releases{/id}\"",\n+            \""deployments_url\"": \""https://api.github.com/repos/simonw/datasette/deployments\"",\n+            \""created_at\"": \""2017-10-23T00:39:03Z\"",\n+            \""updated_at\"": \""2020-07-27T20:42:15Z\"",\n+            \""pushed_at\"": \""2020-07-26T01:21:05Z\"",\n+            \""git_url\"": \""git://github.com/simonw/datasette.git\"",\n+            \""ssh_url\"": \""git@github.com:simonw/datasette.git\"",\n+            \""clone_url\"": \""https://github.com/simonw/datasette.git\"",\n+            \""svn_url\"": \""https://github.com/simonw/datasette\"",\n+            \""homepage\"": \""http://datasette.readthedocs.io/\"",\n+            \""size\"": 3487,\n+            \""stargazers_count\"": 3642,\n+            \""watchers_count\"": 3642,\n+            \""language\"": \""Python\"",\n+            \""has_issues\"": true,\n+            \""has_projects\"": false,\n+            \""has_downloads\"": true,\n+            \""has_wiki\"": true,\n+            \""has_pages\"": false,\n+            \""forks_count\"": 206,\n+            \""mirror_url\"": null,\n+            \""archived\"": false,\n+            \""disabled\"": false,\n+            \""open_issues_count\"": 190,\n+            \""license\"": {\n+              \""key\"": \""apache-2.0\"",\n+              \""name\"": \""Apache License 2.0\"",\n+              \""spdx_id\"": \""Apache-2.0\"",\n+              \""url\"": \""https://api.github.com/licenses/apache-2.0\"",\n+              \""node_id\"": \""MDc6TGljZW5zZTI=\""\n+            },\n+            \""forks\"": 206,\n+            \""open_issues\"": 190,\n+            \""watchers\"": 3642,\n+            \""default_branch\"": \""master\""\n+          }\n+        },\n+        \""_links\"": {\n+          \""self\"": {\n+            \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/571\""\n+          },\n+          \""html\"": {\n+            \""href\"": \""https://github.com/simonw/datasette/pull/571\""\n+          },\n+          \""issue\"": {\n+            \""href\"": \""https://api.github.com/repos/simonw/datasette/issues/571\""\n+          },\n+          \""comments\"": {\n+            \""href\"": \""https://api.github.com/repos/simonw/datasette/issues/571/comments\""\n+          },\n+          \""review_comments\"": {\n+            \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/comments\""\n+          },\n+          \""review_comment\"": {\n+            \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/comments{/number}\""\n+          },\n+          \""commits\"": {\n+            \""href\"": \""https://api.github.com/repos/simonw/datasette/pulls/571/commits\""\n+          },\n+          \""statuses\"": {\n+            \""href\"": \""https://api.github.com/repos/simonw/datasette/statuses/a85239f69261c10f1a9f90514c8b5d113cb94585\""\n+          }\n+        },\n+        \""author_association\"": \""OWNER\"",\n+        \""active_lock_reason\"": null,\n+        \""merged\"": true,\n+        \""mergeable\"": null,\n+        \""rebaseable\"": null,\n+        \""mergeable_state\"": \""unknown\"",\n+        \""merged_by\"": {"",
        ""path"": ""tests/pull_requests.json"",
        ""position"": 342,
        ""original_position"": 342,
        ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3"",
        ""user"": {
            ""login"": ""simonw"",
            ""id"": 9599,
            ""node_id"": ""MDQ6VXNlcjk1OTk="",
            ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"",
            ""gravatar_id"": """",
            ""url"": ""https://api.github.com/users/simonw"",
            ""html_url"": ""https://github.com/simonw"",
            ""followers_url"": ""https://api.github.com/users/simonw/followers"",
            ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"",
            ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"",
            ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"",
            ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"",
            ""organizations_url"": ""https://api.github.com/users/simonw/orgs"",
            ""repos_url"": ""https://api.github.com/users/simonw/repos"",
            ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"",
            ""received_events_url"": ""https://api.github.com/users/simonw/received_events"",
            ""type"": ""User"",
            ""site_admin"": false
        },
        ""body"": ""Running this should create a `merged_by` column on the `pull_requests` table which is a foreign key to the `users` table."",
        ""created_at"": ""2020-10-06T21:22:47Z"",
        ""updated_at"": ""2020-10-20T20:56:33Z"",
        ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500603838"",
        ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"",
        ""author_association"": ""MEMBER"",
        ""_links"": {
            ""self"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500603838""
            },
            ""html"": {
                ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500603838""
            },
            ""pull_request"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48""
            }
        },
        ""original_commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c""
    },
    {
        ""id"": 500606198,
        ""node_id"": ""MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDUwMDYwNjE5OA=="",
        ""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606198"",
        ""pull_request_review_id"": 503368921,
        ""diff_hunk"": ""@@ -0,0 +1,124 @@\n+from github_to_sqlite import utils\n+import pytest\n+import pathlib\n+import sqlite_utils\n+from sqlite_utils.db import ForeignKey\n+import json\n+\n+\n+@pytest.fixture\n+def pull_requests():\n+    return json.load(open(pathlib.Path(__file__).parent / \""pull_requests.json\""))\n+\n+\n+@pytest.fixture\n+def db(pull_requests):\n+    db = sqlite_utils.Database(memory=True)\n+    db[\""repos\""].insert(\n+        {\""id\"": 1},\n+        pk=\""id\"",\n+        columns={\""organization\"": int, \""topics\"": str, \""name\"": str, \""description\"": str},\n+    )\n+    utils.save_pull_requests(db, pull_requests, {\""id\"": 1})\n+    return db\n+\n+\n+def test_tables(db):\n+    assert {\""pull_requests\"", \""users\"", \""repos\"", \""milestones\""} == set(\n+        db.table_names()\n+    )\n+    assert {\n+        ForeignKey(\n+            table=\""pull_requests\"", column=\""repo\"", other_table=\""repos\"", other_column=\""id\""\n+        ),\n+        ForeignKey(\n+            table=\""pull_requests\"",\n+            column=\""milestone\"",\n+            other_table=\""milestones\"",\n+            other_column=\""id\"",\n+        ),\n+        ForeignKey(\n+            table=\""pull_requests\"", column=\""assignee\"", other_table=\""users\"", other_column=\""id\""\n+        ),\n+        ForeignKey(\n+            table=\""pull_requests\"", column=\""user\"", other_table=\""users\"", other_column=\""id\""\n+        ),\n+    } == set(db[\""pull_requests\""].foreign_keys)\n+\n+\n+def test_pull_requests(db):\n+    pull_request_rows = list(db[\""pull_requests\""].rows)\n+    assert [\n+        {\n+            'id': 313384926,\n+            'node_id': 'MDExOlB1bGxSZXF1ZXN0MzEzMzg0OTI2',\n+            'number': 571,\n+            'state': 'closed',\n+            'locked': 0,\n+            'title': 'detect_fts now works with alternative table escaping',\n+            'user': 9599,\n+            'body': 'Fixes #570',\n+            'created_at': '2019-09-03T00:23:39Z',\n+            'updated_at': '2019-09-03T00:32:28Z',\n+            'closed_at': '2019-09-03T00:32:28Z',\n+            'merged_at': '2019-09-03T00:32:28Z',\n+            'merge_commit_sha': '2dc5c8dc259a0606162673d394ba8cc1c6f54428',\n+            'assignee': None,\n+            'milestone': None,\n+            'draft': 0,\n+            'head': 'a85239f69261c10f1a9f90514c8b5d113cb94585',\n+            'base': 'f04deebec4f3842f7bd610cd5859de529f77d50e',\n+            'author_association': 'OWNER',\n+            'merged': 1,\n+            'mergeable': None,\n+            'rebaseable': None,\n+            'mergeable_state': 'unknown',\n+            'merged_by': '{\""login\"": \""simonw\"", \""id\"": 9599, \""node_id\"": \""MDQ6VXNlcjk1OTk=\"", \""avatar_url\"": \""https://avatars0.githubusercontent.com/u/9599?v=4\"", \""gravatar_id\"": \""\"", \""url\"": \""https://api.github.com/users/simonw\"", \""html_url\"": \""https://github.com/simonw\"", \""followers_url\"": \""https://api.github.com/users/simonw/followers\"", \""following_url\"": \""https://api.github.com/users/simonw/following{/other_user}\"", \""gists_url\"": \""https://api.github.com/users/simonw/gists{/gist_id}\"", \""starred_url\"": \""https://api.github.com/users/simonw/starred{/owner}{/repo}\"", \""subscriptions_url\"": \""https://api.github.com/users/simonw/subscriptions\"", \""organizations_url\"": \""https://api.github.com/users/simonw/orgs\"", \""repos_url\"": \""https://api.github.com/users/simonw/repos\"", \""events_url\"": \""https://api.github.com/users/simonw/events{/privacy}\"", \""received_events_url\"": \""https://api.github.com/users/simonw/received_events\"", \""type\"": \""User\"", \""site_admin\"": false}',"",
        ""path"": ""tests/test_pull_requests.py"",
        ""position"": null,
        ""original_position"": 76,
        ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3"",
        ""user"": {
            ""login"": ""simonw"",
            ""id"": 9599,
            ""node_id"": ""MDQ6VXNlcjk1OTk="",
            ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"",
            ""gravatar_id"": """",
            ""url"": ""https://api.github.com/users/simonw"",
            ""html_url"": ""https://github.com/simonw"",
            ""followers_url"": ""https://api.github.com/users/simonw/followers"",
            ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"",
            ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"",
            ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"",
            ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"",
            ""organizations_url"": ""https://api.github.com/users/simonw/orgs"",
            ""repos_url"": ""https://api.github.com/users/simonw/repos"",
            ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"",
            ""received_events_url"": ""https://api.github.com/users/simonw/received_events"",
            ""type"": ""User"",
            ""site_admin"": false
        },
        ""body"": ""See above - this should be 9599, an integer reference to the row in the users table."",
        ""created_at"": ""2020-10-06T21:27:43Z"",
        ""updated_at"": ""2020-10-20T20:56:33Z"",
        ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606198"",
        ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"",
        ""author_association"": ""MEMBER"",
        ""_links"": {
            ""self"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606198""
            },
            ""html"": {
                ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606198""
            },
            ""pull_request"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48""
            }
        },
        ""original_commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c""
    },
    {
        ""id"": 500606665,
        ""node_id"": ""MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDUwMDYwNjY2NQ=="",
        ""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606665"",
        ""pull_request_review_id"": 503368921,
        ""diff_hunk"": ""@@ -0,0 +1,124 @@\n+from github_to_sqlite import utils\n+import pytest\n+import pathlib\n+import sqlite_utils\n+from sqlite_utils.db import ForeignKey\n+import json\n+\n+\n+@pytest.fixture\n+def pull_requests():\n+    return json.load(open(pathlib.Path(__file__).parent / \""pull_requests.json\""))\n+\n+\n+@pytest.fixture\n+def db(pull_requests):\n+    db = sqlite_utils.Database(memory=True)\n+    db[\""repos\""].insert(\n+        {\""id\"": 1},\n+        pk=\""id\"",\n+        columns={\""organization\"": int, \""topics\"": str, \""name\"": str, \""description\"": str},\n+    )\n+    utils.save_pull_requests(db, pull_requests, {\""id\"": 1})\n+    return db\n+\n+\n+def test_tables(db):\n+    assert {\""pull_requests\"", \""users\"", \""repos\"", \""milestones\""} == set(\n+        db.table_names()\n+    )\n+    assert {\n+        ForeignKey(\n+            table=\""pull_requests\"", column=\""repo\"", other_table=\""repos\"", other_column=\""id\""\n+        ),\n+        ForeignKey(\n+            table=\""pull_requests\"",\n+            column=\""milestone\"",\n+            other_table=\""milestones\"",\n+            other_column=\""id\"",\n+        ),\n+        ForeignKey(\n+            table=\""pull_requests\"", column=\""assignee\"", other_table=\""users\"", other_column=\""id\""\n+        ),\n+        ForeignKey(\n+            table=\""pull_requests\"", column=\""user\"", other_table=\""users\"", other_column=\""id\""\n+        ),\n+    } == set(db[\""pull_requests\""].foreign_keys)\n+\n+\n+def test_pull_requests(db):\n+    pull_request_rows = list(db[\""pull_requests\""].rows)\n+    assert [\n+        {\n+            'id': 313384926,"",
        ""path"": ""tests/test_pull_requests.py"",
        ""position"": null,
        ""original_position"": 53,
        ""commit_id"": ""3a0d5c498f9faae4e40aab204cd01b965a4f61f3"",
        ""user"": {
            ""login"": ""simonw"",
            ""id"": 9599,
            ""node_id"": ""MDQ6VXNlcjk1OTk="",
            ""avatar_url"": ""https://avatars0.githubusercontent.com/u/9599?u=5968723deb1a55b82620e106f5ca58e9b11a0942&v=4"",
            ""gravatar_id"": """",
            ""url"": ""https://api.github.com/users/simonw"",
            ""html_url"": ""https://github.com/simonw"",
            ""followers_url"": ""https://api.github.com/users/simonw/followers"",
            ""following_url"": ""https://api.github.com/users/simonw/following{/other_user}"",
            ""gists_url"": ""https://api.github.com/users/simonw/gists{/gist_id}"",
            ""starred_url"": ""https://api.github.com/users/simonw/starred{/owner}{/repo}"",
            ""subscriptions_url"": ""https://api.github.com/users/simonw/subscriptions"",
            ""organizations_url"": ""https://api.github.com/users/simonw/orgs"",
            ""repos_url"": ""https://api.github.com/users/simonw/repos"",
            ""events_url"": ""https://api.github.com/users/simonw/events{/privacy}"",
            ""received_events_url"": ""https://api.github.com/users/simonw/received_events"",
            ""type"": ""User"",
            ""site_admin"": false
        },
        ""body"": ""Minor detail: I use Black for this repo, which requires double quotes - running \""black .\"" in the root directory (with the latest version of Black) should handle this for you."",
        ""created_at"": ""2020-10-06T21:28:31Z"",
        ""updated_at"": ""2020-10-20T20:56:33Z"",
        ""html_url"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606665"",
        ""pull_request_url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48"",
        ""author_association"": ""MEMBER"",
        ""_links"": {
            ""self"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/comments/500606665""
            },
            ""html"": {
                ""href"": ""https://github.com/dogsheep/github-to-sqlite/pull/48#discussion_r500606665""
            },
            ""pull_request"": {
                ""href"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/pulls/48""
            }
        },
        ""original_commit_id"": ""4f33b850bd37829262dd29e1c520afffebedc19c""
    }
]
```
That's a lot more interesting.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022,Feature: pull request reviews and comments,
https://github.com/dogsheep/twitter-to-sqlite/issues/17#issuecomment-541112108,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17,541112108,MDEyOklzc3VlQ29tbWVudDU0MTExMjEwOA==,9599,simonw,2019-10-11T15:30:15Z,2019-10-11T15:30:15Z,MEMBER,It should delete the tables entirely. That way it will work even if the table schema has changed.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",505674949,import command should empty all archive-* tables first,
https://github.com/dogsheep/github-to-sqlite/issues/37#issuecomment-622461948,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37,622461948,MDEyOklzc3VlQ29tbWVudDYyMjQ2MTk0OA==,9599,simonw,2020-05-01T16:36:42Z,2020-05-01T16:36:42Z,MEMBER,It should only create views if the underlying tables exist.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610843136,Mechanism for creating views if they don't yet exist,
https://github.com/dogsheep/github-to-sqlite/issues/35#issuecomment-622213950,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35,622213950,MDEyOklzc3VlQ29tbWVudDYyMjIxMzk1MA==,9599,simonw,2020-05-01T02:09:04Z,2020-05-01T02:09:04Z,MEMBER,"It sped up this query a lot - 2.5s down to 300ms:
```sql
select
  repos.full_name,
  json_object(
    'href', 'https://github.com/' || repos.full_name || '/issues/' || issues.number,
    'label', '#' || issues.number
  ) as issue,
  issues.title,
  users.login,
  users.id,
  issues.state,
  issues.locked,
  issues.assignee,
  issues.milestone,
  issues.comments,
  issues.created_at,
  issues.updated_at,
  issues.closed_at,
  issues.author_association,
  issues.pull_request,
  issues.repo,
  issues.type
from
  issues
  join repos on repos.id = issues.repo
  join users on issues.user = users.id
where
  issues.state = 'open'
  and issues.user not in (9599, 27856297)
  and not exists (
    select
      id
    from
      issue_comments
    where
      issue_comments.user = 9599
      and issues.id = issue_comments.issue
  )
order by
  issues.updated_at desc;
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610511450,Create index on issue_comments(user) and other foreign keys,
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-549096321,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3,549096321,MDEyOklzc3VlQ29tbWVudDU0OTA5NjMyMQ==,9599,simonw,2019-11-03T01:27:55Z,2019-11-03T01:28:17Z,MEMBER,"It would be neat if this could support `--since`, with that argument automatically finding the maximum tweet ID from a previous search that used the same exact arguments (using the `search_runs` table).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833975,Command for running a search and saving tweets for that search,
https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-693794700,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,693794700,MDEyOklzc3VlQ29tbWVudDY5Mzc5NDcwMA==,9599,simonw,2020-09-17T04:02:39Z,2020-09-17T04:02:39Z,MEMBER,It would be useful if you could pass an `--accept` option to this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls,
https://github.com/dogsheep/github-to-sqlite/issues/79#issuecomment-1498167714,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/79,1498167714,IC_kwDODFdgUs5ZTDmi,9599,simonw,2023-04-05T21:12:55Z,2023-04-05T21:12:55Z,MEMBER,"It's a rate limiting problem: https://github.com/dogsheep/github-to-sqlite/actions/runs/4622674009/jobs/8175633155

```
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 407, in commits
    utils.save_commits(db, commits, repo_full[""id""])
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 635, in save_commits
    for commit in commits:
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 417, in fetch_commits
    for commits in paginate(url, headers):
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 478, in paginate
    raise GitHubError.from_response(response)
github_to_sqlite.utils.GitHubError: ('API rate limit exceeded for user ID 9599.', 403)
Error: Process completed with exit code 1.
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1570375808,Deploy demo job is failing due to rate limit,
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695856967,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695856967,MDEyOklzc3VlQ29tbWVudDY5NTg1Njk2Nw==,9599,simonw,2020-09-21T00:26:59Z,2020-09-21T00:26:59Z,MEMBER,It's a shame Datasette doesn't currently have an easy way to implement sorted-by-rank keyset-paginated using a TableView or QueryView. I'll have to do this using the custom SQL query constructed in the plugin: https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L8-L43,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination,
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-543273540,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3,543273540,MDEyOklzc3VlQ29tbWVudDU0MzI3MzU0MA==,9599,simonw,2019-10-17T17:12:51Z,2019-10-17T17:12:51Z,MEMBER,"Just importing tweets here isn't enough - how are we supposed to know which tweets were imported by which search?

So I think the right thing to do here is to also create a `search_runs` table, which records each individual run of this tool (with a timestamp and the search terms used). Then have a `search_runs_tweets` m2m table which shows which Tweets were found by that search.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833975,Command for running a search and saving tweets for that search,
https://github.com/dogsheep/pocket-to-sqlite/issues/2#issuecomment-605338322,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2,605338322,MDEyOklzc3VlQ29tbWVudDYwNTMzODMyMg==,9599,simonw,2020-03-27T22:18:02Z,2020-03-27T22:18:02Z,MEMBER,Just needs documentation now.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503234169,Track and use the 'since' value,
https://github.com/dogsheep/pocket-to-sqlite/issues/11#issuecomment-1221621529,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11,1221621529,IC_kwDODLZ_YM5I0HcZ,9599,simonw,2022-08-21T21:10:15Z,2022-08-21T21:11:26Z,MEMBER,"Just saw that's what's implemented here already!
- #7","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1345452427,"-a option is used for ""--auth"" and for ""--all""",
https://github.com/dogsheep/dogsheep-photos/issues/18#issuecomment-624364557,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/18,624364557,MDEyOklzc3VlQ29tbWVudDYyNDM2NDU1Nw==,9599,simonw,2020-05-05T23:49:18Z,2020-05-05T23:49:18Z,MEMBER,Label is `macos-latest`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612860758,Switch CI solution to GitHub Actions with a macOS runner,
https://github.com/dogsheep/dogsheep-beta/issues/31#issuecomment-748426663,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/31,748426663,MDEyOklzc3VlQ29tbWVudDc0ODQyNjY2Mw==,9599,simonw,2020-12-19T06:14:06Z,2020-12-19T06:14:06Z,MEMBER,Looks like I already do that here: https://github.com/dogsheep/dogsheep-beta/blob/9ba4401017ac24ffa3bc1db38e0910ea49de7616/dogsheep_beta/__init__.py#L141-L146,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771316301,"Searching for ""github-to-sqlite"" throws an error",
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694551646,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694551646,MDEyOklzc3VlQ29tbWVudDY5NDU1MTY0Ng==,9599,simonw,2020-09-17T23:22:48Z,2020-09-17T23:22:48Z,MEMBER,"Looks like its happening in a Jinja fragment template for one of the results:
```
  /Users/simon/Dropbox/Development/dogsheep-beta/dogsheep_beta/__init__.py(169)process_results()
-> output = compiled.render({**result, **{""json"": json}})
  /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/asyncsupport.py(71)render()
-> return original_render(self, *args, **kwargs)
  /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(1090)render()
-> self.environment.handle_exception()
  /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(832)handle_exception()
-> reraise(*rewrite_traceback_stack(source=source))
  /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/_compat.py(28)reraise()
-> raise value.with_traceback(tb)
  <template>(5)top-level template code()
> /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py(341)loads()
-> raise TypeError(f'the JSON object must be str, bytes or bytearray, '
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/swarm-to-sqlite/issues/8#issuecomment-669241341,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/8,669241341,MDEyOklzc3VlQ29tbWVudDY2OTI0MTM0MQ==,9599,simonw,2020-08-05T14:55:14Z,2020-08-05T14:55:14Z,MEMBER,"Looks like there's a column that's not consistently there, so the table got created without it.

Easiest fix is to add `alter=True` to this line:

https://github.com/dogsheep/swarm-to-sqlite/blob/f4a82633da927cde672c9d9af92930bfca2e3ddf/swarm_to_sqlite/utils.py#L94

That will cause `sqlite-utils` to notice if there's a missing column and add it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",648245071,Error thrown: table photos has no column named hasSticker,
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-543271000,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10,543271000,MDEyOklzc3VlQ29tbWVudDU0MzI3MTAwMA==,9599,simonw,2019-10-17T17:05:59Z,2019-10-17T17:05:59Z,MEMBER,Looks like tqdm already does a TTY check here: https://github.com/tqdm/tqdm/blob/89b73bdc30c099c5b53725806e7edf3a121c9b3a/tqdm/std.py#L889-L890,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",492297930,Rethink progress bars for various commands,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790380839,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790380839,MDEyOklzc3VlQ29tbWVudDc5MDM4MDgzOQ==,9599,simonw,2021-03-04T07:17:05Z,2021-03-04T07:17:05Z,MEMBER,"Looks like you're doing this:
```python
    elif message.get_content_type() == ""text/plain"":
        body = message.get_payload(decode=True)
```
So presumably that decodes to a unicode string?

I imagine the reason the column is a `BLOB` for me is that `sqlite-utils` determines the column type based on the first batch of items - https://github.com/simonw/sqlite-utils/blob/09c3386f55f766b135b6a1c00295646c4ae29bec/sqlite_utils/db.py#L1927-L1928 - and I got unlucky and had something in my first batch that wasn't a unicode string.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/twitter-to-sqlite/pull/33#issuecomment-601871930,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/33,601871930,MDEyOklzc3VlQ29tbWVudDYwMTg3MTkzMA==,9599,simonw,2020-03-20T19:21:41Z,2020-03-20T19:21:41Z,MEMBER,Made obsolete by https://github.com/dogsheep/twitter-to-sqlite/commit/d3887d1535c3edd59d3ebf6aea4495bacd82e59b,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",561469252,Upgrade to sqlite-utils 2.2.1,
https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554162,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660554162,MDEyOklzc3VlQ29tbWVudDY2MDU1NDE2Mg==,9599,simonw,2020-07-18T22:56:58Z,2020-07-18T22:56:58Z,MEMBER,"Manually fixing the database:
```
$ wget 'https://github-to-sqlite.dogsheep.net/github.db'
--2020-07-18 15:52:33--  https://github-to-sqlite.dogsheep.net/github.db
Resolving github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)... 172.217.5.115
Connecting to github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)|172.217.5.115|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14626816 (14M) [application/octet-stream]
Saving to: ‘github.db’

github.db                              100%[============================================================================>]  13.95M  1.22MB/s    in 18s     

2020-07-18 15:52:53 (773 KB/s) - ‘github.db’ saved [14626816/14626816]

$ sqlite3 github.db 
SQLite version 3.28.0 2019-04-15 14:49:49
Enter "".help"" for usage hints.
sqlite> drop table tags;
sqlite> ^D
$ github-to-sqlite tags github.db simonw/datasette simonw/sqlite-utils dogsheep/healthkit-to-sqlite dogsheep/swarm-to-sqlite dogsheep/twitter-to-sqlite dogsheep/inaturalist-to-sqlite dogsheep/google-takeout-to-sqlite dogsheep/github-to-sqlite dogsheep/genome-to-sqlite dogsheep/pocket-to-sqlite dogsheep/hacker-news-to-sqlite dogsheep/dogsheep-photos 
$ sqlite-utils tables github.db --counts
[{""table"": ""users"", ""count"": 4048},
 {""table"": ""repos"", ""count"": 210},
 ...
 {""table"": ""stars"", ""count"": 4140},
 {""table"": ""tags"", ""count"": 188}]
$ sqlite-utils rows github.db tags    
[{""repo"": 107914493, ""name"": ""0.45"", ""sha"": ""f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb""},
 {""repo"": 107914493, ""name"": ""0.45a5"", ""sha"": ""676bb64c877d73f8ff496cef4632f5a8a5a9283c""},
 ...
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601,Fix the demo - it breaks because of the tags table change,
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902329884,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57,902329884,IC_kwDODEm0Qs41yHYc,9599,simonw,2021-08-20T00:01:05Z,2021-08-20T00:01:05Z,MEMBER,Maybe Click changed something which meant that this broke things when it didn't used to?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907645813,"Error: Use either --since or --since_id, not both",
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686153967,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686153967,MDEyOklzc3VlQ29tbWVudDY4NjE1Mzk2Nw==,9599,simonw,2020-09-03T00:17:16Z,2020-09-03T00:17:55Z,MEMBER,"Maybe I can take advantage of https://sqlite.org/np1queryprob.html here - I could define a SQL query for fetching the ""display"" version of each item, and include a Jinja template fragment in the configuration as well. Maybe something like this:
```yaml
photos.db:
    photos_with_apple_metadata:
        sql: |-
            select
              sha256 as key,
              'Photo in ' || coalesce(place_city, 'unknown') as title,
              (
                select
                  group_concat(normalized_string, ' ')
                from
                  labels
                where
                  labels.uuid = photos_with_apple_metadata.uuid
              ) as search_1,
              date as timestamp,
              1 as category
            from
              photos_with_apple_metadata
        display_sql: |-
            select
              sha256, place_city, date
            from photos_with_apple_metadata
            where sha256 = :key
        display: |-
          <img src=""https://photos.simonwillison.net/i/{{ display.sha256 }}.jpeg?w=600"">
          <p>Taken in {{ display.place_city }} on {{ display.date }}</p>
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776680,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776680,MDEyOklzc3VlQ29tbWVudDcwNjc3NjY4MA==,9599,simonw,2020-10-11T22:22:16Z,2020-10-11T22:22:16Z,MEMBER,"Maybe the best way do this is with a custom route, `/-/evernote/note-id` - that way I can clean the HTML and resolve the other things in the `<en-note>` structure without using `render_cell()` and the like. My concern about using `render_cell()` is that it could lead to weird security problems when combined with `?sql=` queries.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from <en-media> tags inline in Datasette,
https://github.com/dogsheep/twitter-to-sqlite/issues/12#issuecomment-542858025,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12,542858025,MDEyOklzc3VlQ29tbWVudDU0Mjg1ODAyNQ==,9599,simonw,2019-10-16T19:35:31Z,2019-10-16T19:36:09Z,MEMBER,Maybe this means I need an `upgrade` command to apply these kinds of migrations? Total feature creep!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503053800,"Extract ""source"" into a separate lookup table",
https://github.com/dogsheep/dogsheep-beta/issues/3#issuecomment-684250044,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3,684250044,MDEyOklzc3VlQ29tbWVudDY4NDI1MDA0NA==,9599,simonw,2020-09-01T05:01:09Z,2020-09-01T05:01:23Z,MEMBER,Maybe this starts out as a custom templated canned query.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",689810340,"Datasette plugin to provide custom page for running faceted, ranked searches",
https://github.com/dogsheep/apple-notes-to-sqlite/issues/1#issuecomment-1461226402,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/1,1461226402,IC_kwDOJHON9s5XGIui,9599,simonw,2023-03-09T03:47:11Z,2023-03-09T03:47:11Z,MEMBER,"Me:
> Write applescript to loop through all of my Apple Notes and output their contents

ChatGPT:

```applescript
tell application ""Notes""
   repeat with eachNote in every note
      set noteTitle to the name of eachNote
      set noteBody to the body of eachNote
      display dialog noteTitle & return & return & noteBody
   end repeat
end tell
```
Me:
> How can I turn that into a zsh script which will output the results to my terminal

ChatGPT:
```zsh
#!/usr/bin/zsh

osascript -e 'tell application ""Notes""
   repeat with eachNote in every note
      set noteTitle to the name of eachNote
      set noteBody to the body of eachNote
      set output to noteTitle & ""\n"" & noteBody & ""\n""
      display dialog output
      log output
   end repeat
end tell'
```
I had to change that first line to `#!/bin/zsh` - then when I ran it it spammed my screen with dialog boxes.

Me:

> I want to log the notes contents to the terminal, not display it in a dialog

ChatGPT:

```zsh
#!/bin/zsh

osascript -e 'tell application ""Notes""
   repeat with eachNote in every note
      set noteTitle to the name of eachNote
      set noteBody to the body of eachNote
      set output to noteTitle & ""\n"" & noteBody & ""\n""
      log output
   end repeat
end tell'
```
This really helps!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616347574,Initial proof of concept with ChatGPT,
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653947916,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653947916,MDEyOklzc3VlQ29tbWVudDY1Mzk0NzkxNg==,9599,simonw,2020-07-05T22:40:47Z,2020-07-05T22:40:47Z,MEMBER,Might be that it's not got enough RAM. I'll try deploying to a larger instance.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727,Demo is failing to deploy,
https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-712266834,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29,712266834,MDEyOklzc3VlQ29tbWVudDcxMjI2NjgzNA==,9599,simonw,2020-10-19T16:01:23Z,2020-10-19T16:01:23Z,MEMBER,Might just be a documented pattern for how to configure this in YAML templates.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",724759588,Add search highlighting snippets,
https://github.com/dogsheep/github-to-sqlite/issues/5#issuecomment-549094229,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/5,549094229,MDEyOklzc3VlQ29tbWVudDU0OTA5NDIyOQ==,9599,simonw,2019-11-03T00:44:03Z,2019-11-03T00:44:03Z,MEMBER,Might not need an incomplete boolean - may be possible to handle this with `alter=True` and then by filtering for users with null values in certain columns.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493671014,"Add ""incomplete"" boolean to users table for incomplete profiles",
https://github.com/dogsheep/twitter-to-sqlite/issues/23#issuecomment-543222239,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/23,543222239,MDEyOklzc3VlQ29tbWVudDU0MzIyMjIzOQ==,9599,simonw,2019-10-17T15:12:33Z,2019-10-17T15:12:33Z,MEMBER,"Migrations will run only if you open a database that previously existed (as opposed to opening a brand new empty database).

This means that the first time you run a command against a fresh database, migrations will not run and the `migrations` table will not be created. The _second_ time you run any command against that database the migrations will execute and populate the `migrations` table.

This also means that each migration needs to be able to sanity check the database to see if it should run or not. If it should NOT run, it will do nothing but still be marked as having executed by adding to the `migrations` table.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",508190730,Extremely simple migration system,
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626388764,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626388764,MDEyOklzc3VlQ29tbWVudDYyNjM4ODc2NA==,9599,simonw,2020-05-10T20:58:52Z,2020-05-10T20:58:52Z,MEMBER,"More from the debugger:
```
> /Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py(614)place()
-> self._place = PlaceInfo5(self._info[""reverse_geolocation""])
```
And:
```
> /Users/simon/Dropbox/Development/photos-to-sqlite/photos_to_sqlite/utils.py(91)osxphoto_to_row()
-> place = photo.place
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,bpylist.archiver.CircularReference: archive has a cycle with uid(13),
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694553579,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694553579,MDEyOklzc3VlQ29tbWVudDY5NDU1MzU3OQ==,9599,simonw,2020-09-17T23:28:37Z,2020-09-17T23:28:37Z,MEMBER,"More investigation in pdb:
```
(dogsheep-beta) dogsheep-beta % datasette . --get '/-/beta?q=pycon&sort=oldest' --pdb
> /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py(341)loads()
-> raise TypeError(f'the JSON object must be str, bytes or bytearray, '
(Pdb) list
336  	        if s.startswith('\ufeff'):
337  	            raise JSONDecodeError(""Unexpected UTF-8 BOM (decode using utf-8-sig)"",
338  	                                  s, 0)
339  	    else:
340  	        if not isinstance(s, (bytes, bytearray)):
341  ->	            raise TypeError(f'the JSON object must be str, bytes or bytearray, '
342  	                            f'not {s.__class__.__name__}')
343  	        s = s.decode(detect_encoding(s), 'surrogatepass')
344  	
345  	    if ""encoding"" in kw:
346  	        import warnings
(Pdb) bytes
<class 'bytes'>
(Pdb) locals()['s']
Undefined
(Pdb) type(locals()['s'])
<class 'jinja2.runtime.Undefined'>
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/dogsheep-photos/issues/19#issuecomment-624406285,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19,624406285,MDEyOklzc3VlQ29tbWVudDYyNDQwNjI4NQ==,9599,simonw,2020-05-06T02:10:03Z,2020-05-06T02:10:03Z,MEMBER,"Most annoying part of this is the difficulty of actually showing a photo.

Maybe I need to run a local proxy that I can link to? A custom Datasette plugin perhaps?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613002220,apple-photos command should work even if upload has not run,
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602917713,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602917713,MDEyOklzc3VlQ29tbWVudDYwMjkxNzcxMw==,9599,simonw,2020-03-23T23:40:29Z,2020-03-23T23:40:29Z,MEMBER,"Most recently updated issues across all Dogsheep repos, with faceting: https://github-to-sqlite-j7hipcg4aq-uc.a.run.app/github/issues?_facet=repo&_facet=user&_facet=state&_facet=author_association&_facet=type&_sort_desc=updated_at","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/7#issuecomment-1462693867,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7,1462693867,IC_kwDOJHON9s5XLu_r,9599,simonw,2023-03-09T20:01:39Z,2023-03-09T20:02:11Z,MEMBER,"My `folders` table will have:

- `id` - rowid
- `long_id` - that long unique string ID
- `name` - the name
- `parent` - foreign key to `id`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1617769847,Folder support,
https://github.com/dogsheep/evernote-to-sqlite/issues/7#issuecomment-777827396,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/7,777827396,MDEyOklzc3VlQ29tbWVudDc3NzgyNzM5Ng==,9599,simonw,2021-02-11T22:13:14Z,2021-02-11T22:13:14Z,MEMBER,My best guess is that you have an older version of `sqlite-utils` installed here - the `replace=True` argument was added in version 2.0. I've bumped the dependency in `setup.py`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",743297582,evernote-to-sqlite on windows 10 give this error: TypeError: insert() got an unexpected keyword argument 'replace',
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461234591,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2,1461234591,IC_kwDOJHON9s5XGKuf,9599,simonw,2023-03-09T03:56:45Z,2023-03-09T03:56:45Z,MEMBER,"My prototype showed that images embedded in notes come out in the HTML export as bas64 image URLs, which is neat.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616354999,First working version,
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694551406,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694551406,MDEyOklzc3VlQ29tbWVudDY5NDU1MTQwNg==,9599,simonw,2020-09-17T23:22:07Z,2020-09-17T23:22:07Z,MEMBER,"Neat, I can debug this with the new `--pdb` option:

    datasette . --get '/-/beta?q=pycon&sort=oldest' --pdb
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631226572,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631226572,MDEyOklzc3VlQ29tbWVudDYzMTIyNjU3Mg==,9599,simonw,2020-05-20T04:18:52Z,2020-05-20T04:18:52Z,MEMBER,Need to reconfigure Circle CI.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/pocket-to-sqlite/issues/2#issuecomment-605337941,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2,605337941,MDEyOklzc3VlQ29tbWVudDYwNTMzNzk0MQ==,9599,simonw,2020-03-27T22:16:32Z,2020-03-27T22:16:32Z,MEMBER,"Need to test this. I have 7,394 items in my database right now. I'm going to save a new thing.

Then I ran this:
```
pocket-to-sqlite fetch pocket-simon.db
```
And it worked!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503234169,Track and use the 'since' value,
https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601890795,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35,601890795,MDEyOklzc3VlQ29tbWVudDYwMTg5MDc5NQ==,9599,simonw,2020-03-20T20:13:08Z,2020-03-20T20:13:08Z,MEMBER,Needs documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585282212,twitter-to-sqlite user-timeline [screen_names] --sql / --attach,
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623232984,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623232984,MDEyOklzc3VlQ29tbWVudDYyMzIzMjk4NA==,9599,simonw,2020-05-04T02:41:32Z,2020-05-04T02:41:32Z,MEMBER,Needs documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686689122,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686689122,MDEyOklzc3VlQ29tbWVudDY4NjY4OTEyMg==,9599,simonw,2020-09-03T18:43:20Z,2020-09-03T18:43:20Z,MEMBER,Needs documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-623006154,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,623006154,MDEyOklzc3VlQ29tbWVudDYyMzAwNjE1NA==,9599,simonw,2020-05-02T20:01:39Z,2020-05-02T20:01:54Z,MEMBER,Needs tests and documentation. I shipped it early to check that the live demo works.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/twitter-to-sqlite/issues/38#issuecomment-601939656,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/38,601939656,MDEyOklzc3VlQ29tbWVudDYwMTkzOTY1Ng==,9599,simonw,2020-03-20T22:37:04Z,2020-03-20T22:37:04Z,MEMBER,"New formatting:
```
@Alaska_DHSS      [####################################]  3205
@BWhippleKS       [####################################]  1
@CAPublicHealth   [####################################]  2
@CDPHE            [####################################]  67
@CHFSKy           [####################################]  3216
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585359363,Screen name display for user-timeline is uneven,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615942116,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615942116,MDEyOklzc3VlQ29tbWVudDYxNTk0MjExNg==,9599,simonw,2020-04-18T20:30:56Z,2020-04-18T20:30:56Z,MEMBER,"Next step: attempt a programmatic upload using the `dogsheep-photos-simon-read-write` credentials from a Jupyter notebook.

Also attempt a programmatic bucket listing and read using `dogsheep-photos-simon-read` credentials.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615936880,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615936880,MDEyOklzc3VlQ29tbWVudDYxNTkzNjg4MA==,9599,simonw,2020-04-18T20:04:31Z,2020-04-18T20:04:31Z,MEMBER,"Next step: create two IAM users, one for each of those groups.

https://console.aws.amazon.com/iam/home#/users$new?step=details

![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669931-1bc05380-8174-11ea-9657-0e0c6a692d42.png)

![IAM_Management_Console](https://user-images.githubusercontent.com/9599/79669941-27137f00-8174-11ea-8ce7-249f0d4f96f6.png)

I copied the keys into a secure note in 1password.

Couldn't get into Transmit with them though! https://library.panic.com/transmit/transmit5/iam-roles/ may help.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602919058,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602919058,MDEyOklzc3VlQ29tbWVudDYwMjkxOTA1OA==,9599,simonw,2020-03-23T23:44:48Z,2020-03-23T23:44:48Z,MEMBER,Next step: use a `metadata.json` file to add some extras. And add the `datasette-render-markdown` plugin as soon as I ship https://github.com/simonw/datasette-render-markdown/issues/2 (GFM support).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631227020,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631227020,MDEyOklzc3VlQ29tbWVudDYzMTIyNzAyMA==,9599,simonw,2020-05-20T04:20:48Z,2020-05-20T04:21:16Z,MEMBER,Next time I push a release it will create `dogsheep-photos` on PyPI.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/dogsheep-beta/issues/5#issuecomment-686689366,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/5,686689366,MDEyOklzc3VlQ29tbWVudDY4NjY4OTM2Ng==,9599,simonw,2020-09-03T18:43:50Z,2020-09-03T18:43:50Z,MEMBER,No longer needed thanks to #9,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",689847361,Add a context column that's not searchable,
https://github.com/dogsheep/twitter-to-sqlite/issues/17#issuecomment-541112588,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/17,541112588,MDEyOklzc3VlQ29tbWVudDU0MTExMjU4OA==,9599,simonw,2019-10-11T15:31:30Z,2019-10-11T15:31:30Z,MEMBER,"No need for an option:

> This command will delete and recreate all of your `archive-*` tables every time you run it. If this is not what you want, run the command against a fresh SQLite database rather than running it again one that already exists.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",505674949,import command should empty all archive-* tables first,
https://github.com/dogsheep/github-to-sqlite/issues/38#issuecomment-623038378,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38,623038378,MDEyOklzc3VlQ29tbWVudDYyMzAzODM3OA==,9599,simonw,2020-05-03T01:21:13Z,2020-05-03T01:21:13Z,MEMBER,No this is really useful feedback! I'm so close to this project that I miss what's not obvious to people dropping in for the first time.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",611284481,[Feature Request] Support Repo Name in Search 🥺,
https://github.com/dogsheep/github-to-sqlite/issues/15#issuecomment-602311701,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15,602311701,MDEyOklzc3VlQ29tbWVudDYwMjMxMTcwMQ==,9599,simonw,2020-03-23T01:05:15Z,2020-03-23T01:06:14Z,MEMBER,"None of my own releases use assets (they are all pushed to PyPI instead) but I spotted that your project here uses assets, so I'll test against that: https://github.com/instrumenta/conftest/releases/tag/v0.18.0

```
github-to-sqlite releases releases.db instrumenta/conftest
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",544571092,Assets table with downloads,
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653966670,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653966670,MDEyOklzc3VlQ29tbWVudDY1Mzk2NjY3MA==,9599,simonw,2020-07-06T01:07:02Z,2020-07-06T01:07:02Z,MEMBER,OK that fix worked.https://github.com/dogsheep/github-to-sqlite/runs/839764768?check_suite_focus=true,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727,Demo is failing to deploy,
https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502629219,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502629219,IC_kwDODD6af85ZkE1j,9599,simonw,2023-04-11T03:15:26Z,2023-04-11T03:15:26Z,MEMBER,"OK, I figured this out. Unfortunately it's an error that occurs on Python versions that have defensive mode turned on, and it doesn't look like there's a way to turn that mode off. See notes above.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR:  table sqlite_master may not be modified,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-607010634,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,607010634,MDEyOklzc3VlQ29tbWVudDYwNzAxMDYzNA==,9599,simonw,2020-04-01T03:45:16Z,2020-04-01T03:45:16Z,MEMBER,"OK, fix is applied to everything now.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711081703,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711081703,MDEyOklzc3VlQ29tbWVudDcxMTA4MTcwMw==,9599,simonw,2020-10-17T21:18:35Z,2020-10-17T21:18:35Z,MEMBER,"OK, if you upgrade to the just-released 1.0 this should work (it worked against my Spanish export).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723838331,export.xml file name varies with different language settings,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/2#issuecomment-1461285545,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2,1461285545,IC_kwDOJHON9s5XGXKp,9599,simonw,2023-03-09T05:06:24Z,2023-03-09T05:06:24Z,MEMBER,"OK, this works!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616354999,First working version,
https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-675259273,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,675259273,MDEyOklzc3VlQ29tbWVudDY3NTI1OTI3Mw==,9599,simonw,2020-08-18T05:28:32Z,2020-08-18T05:28:32Z,MEMBER,"Oh that's interesting - i didn't realize ""reviews"" were a separate concept. I'd definitely accept a pull request adding those!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022,Feature: pull request reviews and comments,
https://github.com/dogsheep/dogsheep-beta/issues/31#issuecomment-748426581,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/31,748426581,MDEyOklzc3VlQ29tbWVudDc0ODQyNjU4MQ==,9599,simonw,2020-12-19T06:13:17Z,2020-12-19T06:13:17Z,MEMBER,"One fix for this could be to try running the raw query, but if it throws an error run it again with the query escaped.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771316301,"Searching for ""github-to-sqlite"" throws an error",
https://github.com/dogsheep/twitter-to-sqlite/issues/4#issuecomment-527682713,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/4,527682713,MDEyOklzc3VlQ29tbWVudDUyNzY4MjcxMw==,9599,simonw,2019-09-03T23:48:57Z,2019-09-03T23:48:57Z,MEMBER,"One interesting challenge here is that the JSON format for tweets in the archive is subtly different from the JSON format currently returned by the API.

If we want to keep the tweets in the same database table (which feels like the right thing to me) we'll need to handle this.

One thing we can do is have a column for `from_archive` which is set to 1 for tweets that were recovered from the archive.

We can also ensure that tweets from the API always over-write the version that came from the archive (using `.upsert()`) while tweets from the archive use `.insert(..., ignore=True)` to avoid over-writing a better version that came from the API.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488835586,Command for importing data from a Twitter Export file,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606304837,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606304837,MDEyOklzc3VlQ29tbWVudDYwNjMwNDgzNw==,9599,simonw,2020-03-30T23:27:50Z,2020-03-30T23:29:31Z,MEMBER,"One option would be something like this:
```sql
select max(id) from tweets
where user = ?
and not exists (select id from tweets where retweeted_status = id)
and not exists (select id from tweets where quoted_status = id)
and not exists (select id from tweets where in_reply_to_status_id = id)
```
Might be a good idea to index those columns (after confirming that doing so would indeed speed up the query).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-605327655,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1,605327655,MDEyOklzc3VlQ29tbWVudDYwNTMyNzY1NQ==,9599,simonw,2020-03-27T21:42:49Z,2020-03-27T21:42:49Z,MEMBER,Or maybe it was because of the current Google Cloud outage? https://news.ycombinator.com/item?id=22706677,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503233021,Use better pagination (and implement progress bar),
https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776447,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776447,MDEyOklzc3VlQ29tbWVudDcwNjc3NjQ0Nw==,9599,simonw,2020-10-11T22:20:32Z,2020-10-11T22:20:32Z,MEMBER,Or... I could do this client-side. JavaScript that looks for `<en-media>` tags and fetches the data using `fetch()` wouldn't be too hard to write.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from <en-media> tags inline in Datasette,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606843224,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606843224,MDEyOklzc3VlQ29tbWVudDYwNjg0MzIyNA==,9599,simonw,2020-03-31T19:59:11Z,2020-03-31T20:06:32Z,MEMBER,"Or... have a single `since_ids` table to track since values, and have its primary key be a string that looks something like this:

`user:123145`
`home:23441`
`mentions:23425`
`search:99ff9cefff5cbfd804f7cd43e2b27ced8addbe8d`

That last example would use the hash generated here:

https://github.com/dogsheep/twitter-to-sqlite/blob/810cb2af5a175837204389fd7f4b5721f8b325ab/twitter_to_sqlite/cli.py#L792-L808","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/github-to-sqlite/issues/4#issuecomment-531517138,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4,531517138,MDEyOklzc3VlQ29tbWVudDUzMTUxNzEzOA==,9599,simonw,2019-09-14T21:59:59Z,2019-09-14T21:59:59Z,MEMBER,"Paginate through https://api.github.com/repos/simonw/datasette/stargazers

Send `Accept: application/vnd.github.v3.star+json` to get the `starred_at` dates.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670730,Command to fetch stargazers for one or more repos,
https://github.com/dogsheep/dogsheep-photos/issues/14#issuecomment-620769348,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14,620769348,MDEyOklzc3VlQ29tbWVudDYyMDc2OTM0OA==,9599,simonw,2020-04-28T18:09:21Z,2020-04-28T18:09:21Z,MEMBER,"Pricing is pretty good: free for first 1,000 calls per month, then $1.50 per thousand after that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",608512747,Annotate photos using the Google Cloud Vision API,
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622133775,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622133775,MDEyOklzc3VlQ29tbWVudDYyMjEzMzc3NQ==,9599,simonw,2020-04-30T21:49:27Z,2020-04-30T21:49:27Z,MEMBER,"Proposed command:

    github-to-sqlite scrape-dependents github.db simonw/datasette

I'll pull full details of the scraped repos from the regular API. I'll also record when they were ""first seen"" by the command.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,Command for retrieving dependents for a repo,
https://github.com/dogsheep/pocket-to-sqlite/issues/1#issuecomment-538847446,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1,538847446,MDEyOklzc3VlQ29tbWVudDUzODg0NzQ0Ng==,9599,simonw,2019-10-07T05:41:17Z,2019-10-07T05:41:17Z,MEMBER,"Prototype code:
```python
offset = 0
fetched = []
size = 500
while True:
    page = requests.get(""https://getpocket.com/v3/get"", {
        ""consumer_key"": consumer_key,
        ""access_token"": access_token,
        ""sort"": ""oldest"",
        ""detailType"": ""complete"",
        ""count"": size,
        ""offset"": offset,
    }).json()
    print(offset)
    fetched.append(page)
    offset += size
    if not len(page[""list""]):
        break
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503233021,Use better pagination (and implement progress bar),
https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886136224,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2,886136224,IC_kwDODtX3eM400V2g,9599,simonw,2021-07-25T02:08:29Z,2021-07-25T02:08:29Z,MEMBER,"Prototype:

    curl ""https://hn.algolia.com/api/v1/search_by_date?query=simonwillison.net&restrictSearchableAttributes=url&hitsPerPage=1000"" | \
      jq .hits | sqlite-utils insert hn.db items - --pk objectID --alter","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",952179830,Command for fetching Hacker News threads from the search API,
https://github.com/dogsheep/hacker-news-to-sqlite/issues/3#issuecomment-886142671,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/3,886142671,IC_kwDODtX3eM400XbP,9599,simonw,2021-07-25T03:51:05Z,2021-07-25T03:51:05Z,MEMBER,"Prototype:

    curl 'https://hn.algolia.com/api/v1/items/27941108' \
      | jq '[recurse(.children[]) | del(.children)]' \
      | sqlite-utils insert hn.db items - --pk id
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",952189173,Use HN algolia endpoint to retrieve trees,
https://github.com/dogsheep/github-to-sqlite/issues/14#issuecomment-559902818,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14,559902818,MDEyOklzc3VlQ29tbWVudDU1OTkwMjgxOA==,9599,simonw,2019-11-30T01:32:38Z,2019-11-30T01:32:38Z,MEMBER,"Prototype:
```
pip install sqlite-utils paginate-json
paginate-json ""https://api.github.com/users/simonw/events"" | sqlite-utils insert /tmp/events.db events - --pk=id
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",530491074,Command for importing events,
https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735482187,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735482187,MDEyOklzc3VlQ29tbWVudDczNTQ4MjE4Nw==,9599,simonw,2020-11-30T00:20:11Z,2020-11-30T00:20:11Z,MEMBER,"Pull request are now added, thanks to  @adamjonas.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022,Feature: pull request reviews and comments,
https://github.com/dogsheep/dogsheep-photos/issues/14#issuecomment-620771067,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14,620771067,MDEyOklzc3VlQ29tbWVudDYyMDc3MTA2Nw==,9599,simonw,2020-04-28T18:12:34Z,2020-04-28T18:15:38Z,MEMBER,"Python library docs: https://googleapis.dev/python/vision/latest/index.html

I'm creating a new project for this called simonwillison-photos: https://console.cloud.google.com/projectcreate

https://console.cloud.google.com/home/dashboard?project=simonwillison-photos

Then I enabled the Vision API. The direct link to https://console.cloud.google.com/flows/enableapi?apiid=vision-json.googleapis.com which they provided in the docs didn't work - it gave me a ""You don't have sufficient permissions to use the requested API"" error - but starting at the ""Enable APIs"" page and searching for it worked fine.

I created a new service account as an ""owner"" of that project: https://console.cloud.google.com/apis/credentials/serviceaccountkey (and complained about it on Twitter and through their feedback form)

`pip install google-cloud-vision`

```python
from google.cloud import vision
client = vision.ImageAnnotatorClient.from_service_account_file(""simonwillison-photos-18c570b301fe.json"")
# Photo of a lemur
response = client.annotate_image(
    {
        ""image"": {
            ""source"": {
                ""image_uri"": ""https://photos.simonwillison.net/i/1b3414ee9ade67ce04ade9042e6d4b433d1e523c9a16af17f490e2c0a619755b.jpeg""
            }
        },
        ""features"": [
            {""type"": vision.enums.Feature.Type.IMAGE_PROPERTIES},
            {""type"": vision.enums.Feature.Type.OBJECT_LOCALIZATION},
            {""type"": vision.enums.Feature.Type.LABEL_DETECTION},
        ],
    }
)
response
```
Output is:
```
label_annotations {
  mid: ""/m/09686""
  description: ""Vertebrate""
  score: 0.9851104021072388
  topicality: 0.9851104021072388
}
label_annotations {
  mid: ""/m/04rky""
  description: ""Mammal""
  score: 0.975814163684845
  topicality: 0.975814163684845
}
label_annotations {
  mid: ""/m/01280g""
  description: ""Wildlife""
  score: 0.8973650336265564
  topicality: 0.8973650336265564
}
label_annotations {
  mid: ""/m/02f9pk""
  description: ""Lemur""
  score: 0.8270352482795715
  topicality: 0.8270352482795715
}
label_annotations {
  mid: ""/m/0fbf1m""
  description: ""Terrestrial animal""
  score: 0.7443860769271851
  topicality: 0.7443860769271851
}
label_annotations {
  mid: ""/m/06z_nw""
  description: ""Tail""
  score: 0.6934166550636292
  topicality: 0.6934166550636292
}
label_annotations {
  mid: ""/m/0b5gs""
  description: ""Branch""
  score: 0.6203985214233398
  topicality: 0.6203985214233398
}
label_annotations {
  mid: ""/m/05s2s""
  description: ""Plant""
  score: 0.585474967956543
  topicality: 0.585474967956543
}
label_annotations {
  mid: ""/m/089v3""
  description: ""Zoo""
  score: 0.5488107800483704
  topicality: 0.5488107800483704
}
label_annotations {
  mid: ""/m/02tcwp""
  description: ""Trunk""
  score: 0.5200017690658569
  topicality: 0.5200017690658569
}
image_properties_annotation {
  dominant_colors {
    colors {
      color {
        red: 172.0
        green: 146.0
        blue: 116.0
      }
      score: 0.24523821473121643
      pixel_fraction: 0.027533333748579025
    }
    colors {
      color {
        red: 54.0
        green: 50.0
        blue: 42.0
      }
      score: 0.10449723154306412
      pixel_fraction: 0.12893334031105042
    }
    colors {
      color {
        red: 141.0
        green: 121.0
        blue: 97.0
      }
      score: 0.1391485631465912
      pixel_fraction: 0.039133332669734955
    }
    colors {
      color {
        red: 28.0
        green: 25.0
        blue: 20.0
      }
      score: 0.08589499443769455
      pixel_fraction: 0.11506666988134384
    }
    colors {
      color {
        red: 87.0
        green: 82.0
        blue: 74.0
      }
      score: 0.0845794677734375
      pixel_fraction: 0.16113333404064178
    }
    colors {
      color {
        red: 121.0
        green: 117.0
        blue: 108.0
      }
      score: 0.05901569500565529
      pixel_fraction: 0.13379999995231628
    }
    colors {
      color {
        red: 94.0
        green: 83.0
        blue: 66.0
      }
      score: 0.049011144787073135
      pixel_fraction: 0.03946666792035103
    }
    colors {
      color {
        red: 155.0
        green: 117.0
        blue: 90.0
      }
      score: 0.04164913296699524
      pixel_fraction: 0.0023333332501351833
    }
    colors {
      color {
        red: 178.0
        green: 143.0
        blue: 102.0
      }
      score: 0.02993861958384514
      pixel_fraction: 0.0012666666880249977
    }
    colors {
      color {
        red: 61.0
        green: 51.0
        blue: 35.0
      }
      score: 0.027391711249947548
      pixel_fraction: 0.01953333243727684
    }
  }
}
crop_hints_annotation {
  crop_hints {
    bounding_poly {
      vertices {
        x: 2073
      }
      vertices {
        x: 4008
      }
      vertices {
        x: 4008
        y: 3455
      }
      vertices {
        x: 2073
        y: 3455
      }
    }
    confidence: 0.65625
    importance_fraction: 0.746666669845581
  }
}
localized_object_annotations {
  mid: ""/m/0jbk""
  name: ""Animal""
  score: 0.7008256912231445
  bounding_poly {
    normalized_vertices {
      x: 0.0390297956764698
      y: 0.26235100626945496
    }
    normalized_vertices {
      x: 0.8466796875
      y: 0.26235100626945496
    }
    normalized_vertices {
      x: 0.8466796875
      y: 0.9386426210403442
    }
    normalized_vertices {
      x: 0.0390297956764698
      y: 0.9386426210403442
    }
  }
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",608512747,Annotate photos using the Google Cloud Vision API,
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694552393,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694552393,MDEyOklzc3VlQ29tbWVudDY5NDU1MjM5Mw==,9599,simonw,2020-09-17T23:25:01Z,2020-09-17T23:25:17Z,MEMBER,"Ran `locals()` In the debugger:
`{'range': <class 'range'>, 'dict': <class 'dict'>, 'lipsum': <function generate_lorem_ipsum at 0x10aeff430>, 'cycler': <class 'jinja2.utils.Cycler'>, 'joiner': <class 'jinja2.utils.Joiner'>, 'namespace': <class 'jinja2.utils.Namespace'>, 'rank': -9.383801886431414, 'rowid': 14297, 'type': 'twitter.db/tweets', 'key': '312658917933076480', 'title': 'Tweet by @chrisstreeter', 'category': 2, 'timestamp': '2013-03-15T20:17:49+00:00', 'search_1': '@simonw are you at pycon? Would love to meet you.', 'display': {'avatar_url': 'https://pbs.twimg.com/profile_images/806275088597204993/38yLHfJi_normal.jpg', 'user_name': 'Chris Streeter', 'screen_name': 'chrisstreeter', 'followers_count': 280, 'tweet_id': 312658917933076480, 'created_at': '2013-03-15T20:17:49+00:00', 'full_text': '@simonw are you at pycon? Would love to meet you.', 'media_urls_2': '[]', 'media_urls': '[]'}, 'json': <module 'json' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>}`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/twitter-to-sqlite/issues/2#issuecomment-527955302,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/2,527955302,MDEyOklzc3VlQ29tbWVudDUyNzk1NTMwMg==,9599,simonw,2019-09-04T15:32:39Z,2019-09-04T15:32:39Z,MEMBER,Rate limit is 900 / 15 minutes which is 1 call per second.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833698,"""twitter-to-sqlite user-timeline"" command for pulling tweets by a specific user",
https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541118934,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18,541118934,MDEyOklzc3VlQ29tbWVudDU0MTExODkzNA==,9599,simonw,2019-10-11T15:48:54Z,2019-10-11T15:48:54Z,MEMBER,Rate limit is tight: 15 requests every 15 mins!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",505928530,Command to import home-timeline,
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623199750,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623199750,MDEyOklzc3VlQ29tbWVudDYyMzE5OTc1MA==,9599,simonw,2020-05-03T23:17:58Z,2020-05-03T23:17:58Z,MEMBER,Reading this source code is really useful for figuring out how to store a photo in a DB table: https://github.com/RhetTbull/osxphotos/blob/7444b6d173918a3ad2a07aefce5ecf054786c787/osxphotos/photoinfo.py,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/dogsheep-beta/issues/16#issuecomment-695851036,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16,695851036,MDEyOklzc3VlQ29tbWVudDY5NTg1MTAzNg==,9599,simonw,2020-09-20T23:34:57Z,2020-09-20T23:34:57Z,MEMBER,Really basic starting point is to add facet by date.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694493566,Timeline view,
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623199701,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623199701,MDEyOklzc3VlQ29tbWVudDYyMzE5OTcwMQ==,9599,simonw,2020-05-03T23:17:38Z,2020-05-03T23:17:38Z,MEMBER,"Record burst_uuid as a column:
```
(Pdb) with_bursts[0]._info[""burstUUID""]
'703FAA23-57BF-40B4-8A33-D9CEB143391B'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/swarm-to-sqlite/issues/13#issuecomment-1502546045,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13,1502546045,IC_kwDODD6af85Zjwh9,9599,simonw,2023-04-11T01:14:50Z,2023-04-11T01:14:50Z,MEMBER,"Related:
- https://github.com/simonw/sqlite-utils/issues/235","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1373210675,fails before generating views. ERR:  table sqlite_master may not be modified,
https://github.com/dogsheep/twitter-to-sqlite/issues/35#issuecomment-601892338,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35,601892338,MDEyOklzc3VlQ29tbWVudDYwMTg5MjMzOA==,9599,simonw,2020-03-20T20:17:00Z,2020-03-20T20:17:00Z,MEMBER,Released as 0.18.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585282212,twitter-to-sqlite user-timeline [screen_names] --sql / --attach,
https://github.com/dogsheep/hacker-news-to-sqlite/issues/1#issuecomment-602094386,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/1,602094386,MDEyOklzc3VlQ29tbWVudDYwMjA5NDM4Ng==,9599,simonw,2020-03-21T19:47:46Z,2020-03-21T19:47:46Z,MEMBER,Released as 0.3.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585526292,Set up full text search,
https://github.com/dogsheep/twitter-to-sqlite/issues/43#issuecomment-615513491,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/43,615513491,MDEyOklzc3VlQ29tbWVudDYxNTUxMzQ5MQ==,9599,simonw,2020-04-17T23:48:28Z,2020-04-17T23:48:28Z,MEMBER,Released in 0.21.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602176870,"""twitter-to-sqlite lists"" command for retrieving a user's owned lists",
https://github.com/dogsheep/twitter-to-sqlite/issues/29#issuecomment-552133449,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/29,552133449,MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ0OQ==,9599,simonw,2019-11-09T20:15:15Z,2019-11-09T20:15:15Z,MEMBER,Released: https://github.com/dogsheep/twitter-to-sqlite/releases/tag/0.15,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",518725064,`import` command fails on empty files,
https://github.com/dogsheep/twitter-to-sqlite/issues/30#issuecomment-552133468,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/30,552133468,MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ2OA==,9599,simonw,2019-11-09T20:15:27Z,2019-11-09T20:15:27Z,MEMBER,Released: https://github.com/dogsheep/twitter-to-sqlite/releases/tag/0.15,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",518739697,`followers` fails because `transform_user` is called twice,
https://github.com/dogsheep/twitter-to-sqlite/issues/28#issuecomment-552133488,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/28,552133488,MDEyOklzc3VlQ29tbWVudDU1MjEzMzQ4OA==,9599,simonw,2019-11-09T20:15:42Z,2019-11-09T20:15:42Z,MEMBER,Released: https://github.com/dogsheep/twitter-to-sqlite/releases/tag/0.15,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",515658861,Add indexes to followers table,
https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550824838,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10,550824838,MDEyOklzc3VlQ29tbWVudDU1MDgyNDgzOA==,9599,simonw,2019-11-07T05:47:07Z,2019-11-07T05:47:07Z,MEMBER,"Relevant code:

https://github.com/dogsheep/healthkit-to-sqlite/blob/d16f45f06fbae6ec8a78cc9ca7b5b7db0413f139/healthkit_to_sqlite/utils.py#L58-L64","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",519038979,Failed to import workout points,
https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685965516,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7,685965516,MDEyOklzc3VlQ29tbWVudDY4NTk2NTUxNg==,9599,simonw,2020-09-02T20:01:54Z,2020-09-02T20:01:54Z,MEMBER,"Relevant post: https://sqlite.org/forum/forumpost/9f06fedaa5 - drh says:

> Indexes are one-to-one. There is one entry in the index for each row in the table.
>
> You are asking for an index that is many-to-one - multiple index entries for each table row.
>
> A Full-Text Index is basically a many-to-one index. So if all of your array entries really are words, you could probably get this to work using a Full-Text Index.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691265198,"Mechanism for differentiating between ""by me"" and ""liked by me""",
https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685962280,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7,685962280,MDEyOklzc3VlQ29tbWVudDY4NTk2MjI4MA==,9599,simonw,2020-09-02T19:55:26Z,2020-09-02T19:59:58Z,MEMBER,"Relevant: https://charlesleifer.com/blog/a-tour-of-tagging-schemas-many-to-many-bitmaps-and-more/

SQLite supports bitwise operators Binary AND (&) and Binary OR (|) - I could try those. Not sure how they interact with indexes though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691265198,"Mechanism for differentiating between ""by me"" and ""liked by me""",
https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631251707,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25,631251707,MDEyOklzc3VlQ29tbWVudDYzMTI1MTcwNw==,9599,simonw,2020-05-20T05:49:27Z,2020-05-21T15:58:42Z,MEMBER,Renaming this demo to `dogsheep-photos.dogsheep.net`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621332242,Create a public demo,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932007,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615932007,MDEyOklzc3VlQ29tbWVudDYxNTkzMjAwNw==,9599,simonw,2020-04-18T19:27:55Z,2020-04-18T19:27:55Z,MEMBER,"Research thread: https://twitter.com/simonw/status/1249049694984011776
> I want to build some software that lets people store their own data in their own S3 bucket, but if possible I'd like not to have to teach people the incantations needed to get their bucket setup and minimum-permission credentials figures out

https://testdriven.io/blog/storing-django-static-and-media-files-on-amazon-s3/ looks useful","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623806687,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623806687,MDEyOklzc3VlQ29tbWVudDYyMzgwNjY4Nw==,9599,simonw,2020-05-05T02:51:16Z,2020-05-05T02:51:16Z,MEMBER,"Running datasette against it directly doesn't work:
```
simon@Simons-MacBook-Pro search % datasette psi.sqlite
Serve! files=('psi.sqlite',) (immutables=()) on port 8001
Usage: datasette serve [OPTIONS] [FILES]...

Error: Connection to psi.sqlite failed check: no such tokenizer: PSITokenizer
```
Instead, I created a new SQLite database with a copy of some of the key tables, like this:
```
sqlite-utils rows psi.sqlite groups | sqlite-utils insert /tmp/search.db groups -
sqlite-utils rows psi.sqlite assets | sqlite-utils insert /tmp/search.db assets -
sqlite-utils rows psi.sqlite ga | sqlite-utils insert /tmp/search.db ga -
sqlite-utils rows psi.sqlite collections | sqlite-utils insert /tmp/search.db collections -
sqlite-utils rows psi.sqlite gc | sqlite-utils insert /tmp/search.db gc -
sqlite-utils rows psi.sqlite lookup | sqlite-utils insert /tmp/search.db lookup -
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947229,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615947229,MDEyOklzc3VlQ29tbWVudDYxNTk0NzIyOQ==,9599,simonw,2020-04-18T20:51:26Z,2020-04-18T20:51:26Z,MEMBER,"Running the upload again like this resulted in the correct content-type:
```python
client.upload_file(
    ""/Users/simonw/Desktop/this_is_fine.jpg"",
    ""dogsheep-photos-simon"",
    ""this_is_fine.jpg"",
    ExtraArgs={
        ""ContentType"": ""image/jpeg""
    }
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653960989,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653960989,MDEyOklzc3VlQ29tbWVudDY1Mzk2MDk4OQ==,9599,simonw,2020-07-06T00:32:34Z,2020-07-06T00:32:34Z,MEMBER,Same error.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727,Demo is failing to deploy,
https://github.com/dogsheep/swarm-to-sqlite/issues/8#issuecomment-707332912,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/8,707332912,MDEyOklzc3VlQ29tbWVudDcwNzMzMjkxMg==,9599,simonw,2020-10-12T20:35:06Z,2020-10-12T20:35:06Z,MEMBER,Shipped a fix for this in [swarm-to-sqlite 0.3.2](https://github.com/dogsheep/swarm-to-sqlite/releases/tag/0.3.2).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",648245071,Error thrown: table photos has no column named hasSticker,
https://github.com/dogsheep/twitter-to-sqlite/issues/11#issuecomment-538711918,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/11,538711918,MDEyOklzc3VlQ29tbWVudDUzODcxMTkxOA==,9599,simonw,2019-10-06T04:54:17Z,2019-10-06T04:54:17Z,MEMBER,Shipped in 0.6. Here's the documentation: https://github.com/dogsheep/twitter-to-sqlite#capturing-tweets-in-real-time-with-track-and-follow,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503045221,Commands for recording real-time tweets from the streaming API,
https://github.com/dogsheep/twitter-to-sqlite/issues/26#issuecomment-547713287,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/26,547713287,MDEyOklzc3VlQ29tbWVudDU0NzcxMzI4Nw==,9599,simonw,2019-10-30T02:36:13Z,2019-10-30T02:36:13Z,MEMBER,"Shipped this in 0.13: https://github.com/dogsheep/twitter-to-sqlite/releases/tag/0.13

See also this Twitter thread: https://twitter.com/simonw/status/1189369677509623809","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",513074501,Command for importing mentions timeline,
https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695855646,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695855646,MDEyOklzc3VlQ29tbWVudDY5NTg1NTY0Ng==,9599,simonw,2020-09-21T00:16:11Z,2020-09-21T00:16:11Z,MEMBER,"Should I do this with offset/limit or should I do proper keyset pagination?

I think keyset because then it will work well for the full search interface with no filters or search string.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination,
https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-747031608,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29,747031608,MDEyOklzc3VlQ29tbWVudDc0NzAzMTYwOA==,9599,simonw,2020-12-16T21:15:18Z,2020-12-16T21:15:18Z,MEMBER,Should I pass any other details to the `display_sql` here as well?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",724759588,Add search highlighting snippets,
https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706786548,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4,706786548,MDEyOklzc3VlQ29tbWVudDcwNjc4NjU0OA==,9599,simonw,2020-10-11T23:39:46Z,2020-10-11T23:39:46Z,MEMBER,Should have used porter stemming for this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938508,Configure FTS + add an index on the date columns,
https://github.com/dogsheep/github-to-sqlite/issues/14#issuecomment-613641947,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/14,613641947,MDEyOklzc3VlQ29tbWVudDYxMzY0MTk0Nw==,9599,simonw,2020-04-14T19:38:24Z,2020-04-14T19:38:34Z,MEMBER,"Since events include payloads with full object representations in them (for issues, repos and more) running this command every few minutes may be all it takes to keep a constant copy of everything updated in a very rate-limit friendly manner (thanks to the ETags).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",530491074,Command for importing events,
https://github.com/dogsheep/dogsheep-beta/issues/28#issuecomment-711089647,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/28,711089647,MDEyOklzc3VlQ29tbWVudDcxMTA4OTY0Nw==,9599,simonw,2020-10-17T22:43:13Z,2020-10-17T22:43:13Z,MEMBER,"Since my personal Dogsheep uses Datasette authentication, I'm going to need to pass through cookies. https://github.com/simonw/datasette/issues/1020 will solve that in the future but for now I need to solve it explicitly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723861683,Switch to using datasette.client,
https://github.com/dogsheep/dogsheep-beta/issues/10#issuecomment-686238498,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/10,686238498,MDEyOklzc3VlQ29tbWVudDY4NjIzODQ5OA==,9599,simonw,2020-09-03T04:05:05Z,2020-09-03T04:05:05Z,MEMBER,Since the first two categories are `created` and `saved` this one should be called `received`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691557547,Category 3: received,
https://github.com/dogsheep/github-to-sqlite/issues/31#issuecomment-617348174,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31,617348174,MDEyOklzc3VlQ29tbWVudDYxNzM0ODE3NA==,9599,simonw,2020-04-21T18:50:29Z,2020-04-21T18:50:29Z,MEMBER,"Since this represents a breaking schema change for anyone running SQL queries against these tables, I'm going to do a major version bump to 2.0 when I release this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",603624862,Issue and milestone should have foreign key to repo,
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626388837,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626388837,MDEyOklzc3VlQ29tbWVudDYyNjM4ODgzNw==,9599,simonw,2020-05-10T20:59:32Z,2020-05-10T20:59:32Z,MEMBER,So it appears it's possible for `photo.place` to raise that exception. A workaround could be to catch that and treat those photos as not having a place.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,bpylist.archiver.CircularReference: archive has a cycle with uid(13),
https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-735483820,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46,735483820,MDEyOklzc3VlQ29tbWVudDczNTQ4MzgyMA==,9599,simonw,2020-11-30T00:27:47Z,2020-11-30T00:27:47Z,MEMBER,"So it looks like anything that pulls reviews needs to pull each review, then for each one pull the comments.

I'm going to consider this blocked on smarter rate limit handling in #51.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",664485022,Feature: pull request reviews and comments,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/7#issuecomment-1462564717,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7,1462564717,IC_kwDOJHON9s5XLPdt,9599,simonw,2023-03-09T18:25:39Z,2023-03-09T18:25:39Z,MEMBER,So it looks like folders can be hierarchical?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1617769847,Folder support,
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962530,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962530,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjUzMA==,9599,simonw,2020-07-06T00:42:13Z,2020-07-06T00:42:13Z,MEMBER,So it looks like it's the ASGI lifespan change I made in https://github.com/simonw/datasette/commit/16f592247a2a0e140ada487e9972645406dcae69 - It must be incompatible with one of the plugins.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727,Demo is failing to deploy,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602813731,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,602813731,MDEyOklzc3VlQ29tbWVudDYwMjgxMzczMQ==,9599,simonw,2020-03-23T19:38:03Z,2020-03-23T20:44:49Z,MEMBER,"So it turns out `""author""` and `""committer""` on the commit are `null` if the email address in the nested `""commit""` doesn't match an existing GitHub user.

Maybe I should be storing the nested data somewhere as well?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550828084,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10,550828084,MDEyOklzc3VlQ29tbWVudDU1MDgyODA4NA==,9599,simonw,2019-11-07T05:49:24Z,2019-11-07T05:49:24Z,MEMBER,"So the fix there is going to be to detect the new `FileReference` element and load the corresponding points data from it.

This will be a little tricky because that function will need access to the zip file.

It probably won't work at all for the mode where the `export.xml` file is passed directly using the `--xml` option.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",519038979,Failed to import workout points,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790669767,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790669767,MDEyOklzc3VlQ29tbWVudDc5MDY2OTc2Nw==,9599,simonw,2021-03-04T14:46:06Z,2021-03-04T14:46:06Z,MEMBER,"Solution could be to pre-process that string by splitting on `(` and dropping everything afterwards, assuming that the `(...)` bit isn't necessary for correctly parsing the date.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/healthkit-to-sqlite/issues/4#issuecomment-513440090,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/4,513440090,MDEyOklzc3VlQ29tbWVudDUxMzQ0MDA5MA==,9599,simonw,2019-07-20T06:11:50Z,2019-07-20T06:11:50Z,MEMBER,"Some examples:

https://github.com/dogsheep/healthkit-to-sqlite/blob/d016e70c31cf84ba0f5ec3102546db54a51aaffb/tests/export.xml#L4-L13","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",470640505,Import Records,
https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685895540,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7,685895540,MDEyOklzc3VlQ29tbWVudDY4NTg5NTU0MA==,9599,simonw,2020-09-02T17:46:44Z,2020-09-02T17:46:44Z,MEMBER,"Some opet questions about this:

- Should I restrict to two exclusive categories here, or should I have a generic category mechanism that can be expanded to more than two?
- Should an item be able to exist in more than one category? Do I want to be able to mark an indexed item as both by-me and liked-by-me for example? This question is more interesting if the number of categories is greater than two.
- How should this be modeled? Single column, multiple boolean columns, JSON array, m2m against separate table?
- What's the best way to make this performant","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691265198,"Mechanism for differentiating between ""by me"" and ""liked by me""",
https://github.com/dogsheep/github-to-sqlite/pull/48#issuecomment-704553385,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/48,704553385,MDEyOklzc3VlQ29tbWVudDcwNDU1MzM4NQ==,9599,simonw,2020-10-06T21:07:44Z,2020-10-06T21:07:44Z,MEMBER,"Sorry for not looking at this sooner, trying it out now - pull request looks great!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681228542,Add pull requests,
https://github.com/dogsheep/github-to-sqlite/issues/3#issuecomment-531517083,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/3,531517083,MDEyOklzc3VlQ29tbWVudDUzMTUxNzA4Mw==,9599,simonw,2019-09-14T21:58:42Z,2019-09-14T21:58:42Z,MEMBER,Split stargazers into #4,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670426,Command to fetch all repos belonging to a user or organization,
https://github.com/dogsheep/github-to-sqlite/issues/15#issuecomment-602309825,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15,602309825,MDEyOklzc3VlQ29tbWVudDYwMjMwOTgyNQ==,9599,simonw,2020-03-23T00:52:38Z,2020-03-23T00:52:38Z,MEMBER,Splitting assets out into a separate table totally makes sense to me. They can still be fetched as part of the `releases` command.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",544571092,Assets table with downloads,
https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549233778,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,549233778,MDEyOklzc3VlQ29tbWVudDU0OTIzMzc3OA==,9599,simonw,2019-11-04T06:14:40Z,2019-11-04T06:14:40Z,MEMBER,"Spotted a tricky problem: running `github-to-sqlite starred stargazers.db` results in an incomplete `simonw` record. It creates a proper record for me thanks to this bit:

https://github.com/dogsheep/github-to-sqlite/blob/ea07274667a08c67907e8bfbbccb6f0fb95ce817/github_to_sqlite/cli.py#L120-L126

But then... when it gets to the `datasette` repository which I have starred it over-writes my full user record with one that's missing most of the details, thanks to this bit:

https://github.com/dogsheep/github-to-sqlite/blob/ea07274667a08c67907e8bfbbccb6f0fb95ce817/github_to_sqlite/utils.py#L117-L124

I need to find a way of NOT over-writing a good record with a thinner one.

","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727,"stargazers command, refs #4",
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633626741,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,633626741,MDEyOklzc3VlQ29tbWVudDYzMzYyNjc0MQ==,9599,simonw,2020-05-25T15:38:55Z,2020-05-25T15:38:55Z,MEMBER,"Sure, I should absolutely document this!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/github-to-sqlite/issues/33#issuecomment-622169728,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/33,622169728,MDEyOklzc3VlQ29tbWVudDYyMjE2OTcyOA==,9599,simonw,2020-04-30T23:18:51Z,2020-04-30T23:18:51Z,MEMBER,"Sure, that sounds fine to me.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",609950090,Fall back to authentication via ENV,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606850008,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606850008,MDEyOklzc3VlQ29tbWVudDYwNjg1MDAwOA==,9599,simonw,2020-03-31T20:13:59Z,2020-04-01T00:23:00Z,MEMBER,"Table design for `since_ids` table:

type | key | since_id
--- | --- | ---
1 | 124324 | 2347239847293
2 | 99ff9cefff5cbfd804f7cd43e2b27ced8addbe8d | 2125947927344

Primary compound key on `(category, key)`

`type` is also a foreign key to a `since_id_types` table with `id` and `name` columns (probably created using https://sqlite-utils.readthedocs.io/en/stable/python-api.html#working-with-lookup-tables )","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188233729,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188233729,IC_kwDODFdgUs5G0wIB,9599,simonw,2022-07-18T19:51:02Z,2022-07-18T19:51:02Z,MEMBER,Takes 30m to deploy the demo!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063,500 error in github-to-sqlite demo,
https://github.com/dogsheep/github-to-sqlite/pull/59#issuecomment-844250232,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59,844250232,MDEyOklzc3VlQ29tbWVudDg0NDI1MDIzMg==,9599,simonw,2021-05-19T16:08:10Z,2021-05-19T16:08:10Z,MEMBER,Thanks for catching this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771872303,Remove unneeded exists=True for -a/--auth flag.,
https://github.com/dogsheep/evernote-to-sqlite/issues/11#issuecomment-777798330,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/11,777798330,MDEyOklzc3VlQ29tbWVudDc3Nzc5ODMzMA==,9599,simonw,2021-02-11T21:18:58Z,2021-02-11T21:18:58Z,MEMBER,Thanks for the fix!,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",792851444,XML parse error,
https://github.com/dogsheep/dogsheep-photos/pull/29#issuecomment-739058820,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/29,739058820,MDEyOklzc3VlQ29tbWVudDczOTA1ODgyMA==,9599,simonw,2020-12-04T22:32:35Z,2020-12-04T22:32:35Z,MEMBER,Thanks for this!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",638375985,Fixed bug in SQL query for photo scores,
https://github.com/dogsheep/twitter-to-sqlite/pull/59#issuecomment-924209583,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/59,924209583,IC_kwDODEm0Qs43FlGv,9599,simonw,2021-09-21T17:37:34Z,2021-09-21T17:37:34Z,MEMBER,Thanks for this!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",984942782,"Fix for since_id bug, closes #58",
https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-544646516,https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1,544646516,MDEyOklzc3VlQ29tbWVudDU0NDY0NjUxNg==,9599,simonw,2019-10-21T18:30:14Z,2019-10-21T18:30:14Z,MEMBER,"Thanks to help from Dr. Laura Cantino at Science Hack Day San Francisco I've been able to pull together this query:

```sql
select rsid, genotype, case genotype
  when 'AA' then 'brown eye color, 80% of the time'
  when 'AG' then 'brown eye color'
  when 'GG' then 'blue eye color, 99% of the time'
end as interpretation from genome where rsid = 'rs12913832'
```

See also https://www.snpedia.com/index.php/Rs12913832 - in particular this table:

<img width=""321"" alt=""rs12913832_-_SNPedia"" src=""https://user-images.githubusercontent.com/9599/67232392-216ff300-f3f6-11e9-8e14-b5f50c0c0d16.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",496415321,Figure out some interesting example SQL queries,
https://github.com/dogsheep/swarm-to-sqlite/pull/6#issuecomment-605382373,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/6,605382373,MDEyOklzc3VlQ29tbWVudDYwNTM4MjM3Mw==,9599,simonw,2020-03-28T02:27:32Z,2020-03-28T02:27:32Z,MEMBER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",543355051,don't break if source is missing,
https://github.com/dogsheep/github-to-sqlite/pull/61#issuecomment-844249385,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/61,844249385,MDEyOklzc3VlQ29tbWVudDg0NDI0OTM4NQ==,9599,simonw,2021-05-19T16:07:06Z,2021-05-19T16:07:06Z,MEMBER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",797108702,fixing typo in get cli help text,
https://github.com/dogsheep/twitter-to-sqlite/pull/49#issuecomment-902330301,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/49,902330301,IC_kwDODEm0Qs41yHe9,9599,simonw,2021-08-20T00:01:56Z,2021-08-20T00:01:56Z,MEMBER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681575714,"Document the use of --stop_after with favorites, refs #20",
https://github.com/dogsheep/github-to-sqlite/pull/73#issuecomment-1188223299,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/73,1188223299,IC_kwDODFdgUs5G0tlD,9599,simonw,2022-07-18T19:40:06Z,2022-07-18T19:40:06Z,MEMBER,Thanks!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1261884917,Fixing 'NoneType' object has no attribute 'items',
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-786925280,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,786925280,MDEyOklzc3VlQ29tbWVudDc4NjkyNTI4MA==,9599,simonw,2021-02-26T22:23:10Z,2021-02-26T22:23:10Z,MEMBER,"Thanks!

I requested my Gmail export from takeout - once that arrives I'll test it against this and then merge the PR.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/swarm-to-sqlite/issues/11#issuecomment-727692413,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11,727692413,MDEyOklzc3VlQ29tbWVudDcyNzY5MjQxMw==,9599,simonw,2020-11-16T02:15:22Z,2020-11-16T02:15:22Z,MEMBER,"Thanks, I'll look into this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",743400216,Error thrown: sqlite3.OperationalError: table users has no column named lastName,
https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643414646,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40,643414646,MDEyOklzc3VlQ29tbWVudDY0MzQxNDY0Ng==,9599,simonw,2020-06-12T18:06:48Z,2020-06-12T18:06:48Z,MEMBER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",637899539,Demo deploy is broken,
https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188317682,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188317682,IC_kwDODFdgUs5G1Eny,9599,simonw,2022-07-18T21:14:22Z,2022-07-18T21:14:22Z,MEMBER,That fixed it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063,500 error in github-to-sqlite demo,
https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615883687,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28,615883687,MDEyOklzc3VlQ29tbWVudDYxNTg4MzY4Nw==,9599,simonw,2020-04-18T14:49:58Z,2020-04-18T14:49:58Z,MEMBER,"That happened trying to pull contributors for `dogsheep/beta` - an empty repository. Turns out it was returning a `204 no content`:
```
~ $ curl -i 'https://api.github.com/repos/dogsheep/beta/contributors'
HTTP/1.1 204 No Content
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601333634,Pull repository contributors,
https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645512127,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47,645512127,MDEyOklzc3VlQ29tbWVudDY0NTUxMjEyNw==,9599,simonw,2020-06-17T17:24:22Z,2020-06-17T17:24:22Z,MEMBER,"That means your version of SQLite is old enough that it doesn't support the FTS5 extension.

Could you share what operating system you're running, and what the output is that you get from running this?

    python -c 'import sqlite3; print(sqlite3.connect("":memory:"").execute(""select sqlite_version()"").fetchone()[0])'

I can teach this tool to fall back on FTS4 if FTS5 isn't available.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",639542974,Fall back to FTS4 if FTS5 is not available,
https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886135562,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2,886135562,IC_kwDODtX3eM400VsK,9599,simonw,2021-07-25T02:01:11Z,2021-07-25T02:01:11Z,MEMBER,"That page doesn't have an API but does look easy to scrape.

The other option here is the HN Search API powered by Algolia, documented at https://hn.algolia.com/api","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",952179830,Command for fetching Hacker News threads from the search API,
https://github.com/dogsheep/pocket-to-sqlite/issues/10#issuecomment-1221623052,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/10,1221623052,IC_kwDODLZ_YM5I0H0M,9599,simonw,2022-08-21T21:20:33Z,2022-08-21T21:20:33Z,MEMBER,"That was clearly the intention from the description of this issue:
- #4","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1246826792,"When running `auth` command, don't overwrite an existing auth.json file",
https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615510361,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37,615510361,MDEyOklzc3VlQ29tbWVudDYxNTUxMDM2MQ==,9599,simonw,2020-04-17T23:38:27Z,2020-04-17T23:38:27Z,MEMBER,That's a bit tricky since I'd have to rewrite the internals of a bunch of other commands. For the moment I'll exit the script with an error but at least it will be a decent error!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585353598,"Handle ""User not found"" error",
https://github.com/dogsheep/github-to-sqlite/issues/32#issuecomment-617364956,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/32,617364956,MDEyOklzc3VlQ29tbWVudDYxNzM2NDk1Ng==,9599,simonw,2020-04-21T19:24:45Z,2020-04-21T19:24:45Z,MEMBER,"That's because I just broke this code:

https://github.com/dogsheep/github-to-sqlite/blob/2cf75a0a036719eb7e57fdc7c5c2ea0f4c26978a/github_to_sqlite/utils.py#L131-L139

It expects the `repo` column to be `simonw/datasette` but it's now an ID instead.

I should add a test for this as part of the fix.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",604222295,Issue comments don't appear to populate issues foreign key,
https://github.com/dogsheep/github-to-sqlite/issues/72#issuecomment-1105474232,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/72,1105474232,IC_kwDODFdgUs5B5DK4,9599,simonw,2022-04-21T17:02:15Z,2022-04-21T17:02:15Z,MEMBER,"That's interesting - yeah it looks like the number of pages can be derived from the `Link` header, which is enough information to show a progress bar, probably using Click just to avoid adding another dependency.

https://docs.github.com/en/rest/guides/traversing-with-pagination","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1211283427,feature: display progress bar when downloading multi-page responses,
https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711074306,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711074306,MDEyOklzc3VlQ29tbWVudDcxMTA3NDMwNg==,9599,simonw,2020-10-17T20:16:22Z,2020-10-17T20:16:22Z,MEMBER,"The ""first XML file in the root"" solution is probably easier though!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723838331,export.xml file name varies with different language settings,
https://github.com/dogsheep/github-to-sqlite/issues/31#issuecomment-616884647,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31,616884647,MDEyOklzc3VlQ29tbWVudDYxNjg4NDY0Nw==,9599,simonw,2020-04-21T00:49:16Z,2020-04-21T00:50:20Z,MEMBER,"The API just gives us the `repository_url`: https://api.github.com/repos/simonw/datasette/issues

![Mozilla_Firefox_and_Topic__Week_2__Discussion__Submit_your_six_story_points_here](https://user-images.githubusercontent.com/9599/79812950-283cdb80-832f-11ea-8759-9633087d1e7e.png)

We currently turn that into a `simonw/datasette` string here:
https://github.com/dogsheep/github-to-sqlite/blob/e0e8d8caa9657b04bfb8a2cf16c9b580f38b1805/github_to_sqlite/utils.py#L43-L46","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",603624862,Issue and milestone should have foreign key to repo,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790369076,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790369076,MDEyOklzc3VlQ29tbWVudDc5MDM2OTA3Ng==,9599,simonw,2021-03-04T06:54:46Z,2021-03-04T06:54:46Z,MEMBER,"The Rich-powered progress bar is pretty:

![rich](https://user-images.githubusercontent.com/9599/109923307-71f69200-7c73-11eb-9ee2-8f0a240f3994.gif)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/evernote-to-sqlite/issues/6#issuecomment-706785086,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6,706785086,MDEyOklzc3VlQ29tbWVudDcwNjc4NTA4Ng==,9599,simonw,2020-10-11T23:28:50Z,2020-10-11T23:28:50Z,MEMBER,"The XML for the OCR stuff is a bit weird. Currently I'm doing this to it:

https://github.com/dogsheep/evernote-to-sqlite/blob/c33d7b043a45eb3e88676e5fa3ce31755199d9f8/evernote_to_sqlite/utils.py#L70-L78

This can produce some odd results, for example:

> Sure 'Sure, 'Sure. Sure, Sure. sure sure. sure ? If you If Yau [you live jive In m 1n an area devoid of natural wonders, wanders, wonders ? wonders wonders. your mind will be blown, blown' blown. blown ? -e i ? ,1 IL it ? at ? KY ? fl ft bat at

Which came from this image:

![image](https://user-images.githubusercontent.com/9599/95692952-5dd7c880-0bde-11eb-939a-d10b800a4105.png)

The XML for that is:

```xml
<recoIndex docType=""unknown"" objType=""image"" objID=""05ffb72b307bf495f064243c7099d94f"" engineVersion=""6.5.17.7"" recoType=""service"" lang=""en"" objWidth=""1000"" objHeight=""1504"">
  <item x=""68"" y=""75"" w=""104"" h=""37"">
    <t w=""60"">Sure</t>
    <t w=""52"">'Sure,</t>
    <t w=""47"">'Sure.</t>
    <t w=""33"">Sure,</t>
    <t w=""26"">Sure.</t>
  </item>
  <item x=""182"" y=""83"" w=""92"" h=""26"">
    <t w=""62"">sure</t>
    <t w=""58"">sure.</t>
    <t w=""46"">sure ?</t>
  </item>
  <item x=""69"" y=""132"" w=""107"" h=""45"">
    <t w=""81"">If you</t>
    <t w=""64"">If Yau</t>
    <t w=""31"">[you</t>
  </item>
  <item x=""186"" y=""132"" w=""67"" h=""35"">
    <t w=""85"">live</t>
    <t w=""51"">jive</t>
  </item>
  <item x=""263"" y=""140"" w=""36"" h=""27"">
    <t w=""82"">In</t>
    <t w=""56"">m</t>
    <t w=""53"">1n</t>
  </item>
  <item x=""309"" y=""140"" w=""53"" h=""27"">
    <t w=""82"">an</t>
  </item>
  <item x=""372"" y=""141"" w=""90"" h=""26"">
    <t w=""94"">area</t>
  </item>
  <item x=""472"" y=""132"" w=""138"" h=""35"">
    <t w=""85"">devoid</t>
  </item>
  <item x=""620"" y=""132"" w=""43"" h=""35"">
    <t w=""82"">of</t>
  </item>
  <item x=""68"" y=""190"" w=""137"" h=""35"">
    <t w=""87"">natural</t>
  </item>
  <item x=""215"" y=""190"" w=""187"" h=""39"">
    <t w=""57"">wonders,</t>
    <t w=""55"">wanders,</t>
    <t w=""52"">wonders ?</t>
    <t w=""45"">wonders</t>
    <t w=""42"">wonders.</t>
  </item>
  <item x=""410"" y=""198"" w=""98"" h=""36"">
    <t w=""88"">your</t>
  </item>
  <item x=""518"" y=""190"" w=""102"" h=""35"">
    <t w=""86"">mind</t>
  </item>
  <item x=""630"" y=""190"" w=""69"" h=""34"">
    <t w=""87"">will</t>
  </item>
  <item x=""709"" y=""190"" w=""55"" h=""35"">
    <t w=""82"">be</t>
  </item>
  <item x=""774"" y=""190"" w=""137"" h=""34"">
    <t w=""56"">blown,</t>
    <t w=""55"">blown'</t>
    <t w=""48"">blown.</t>
    <t w=""48"">blown ?</t>
  </item>
  <item x=""166"" y=""736"" w=""8"" h=""6"">
    <t w=""66"">-e</t>
  </item>
  <item x=""273"" y=""966"" w=""29"" h=""21"">
    <t w=""11"">i ?</t>
  </item>
  <item x=""281"" y=""1004"" w=""28"" h=""11"">
    <t w=""11"">,1</t>
  </item>
  <item x=""512"" y=""1083"" w=""10"" h=""7"">
    <t w=""10"">IL</t>
  </item>
  <item x=""29"" y=""1447"" w=""7"" h=""23"">
    <t w=""17"">it ?</t>
    <t w=""15"">at ?</t>
    <t w=""13"">KY ?</t>
  </item>
  <item x=""414"" y=""841"" w=""8"" h=""16"">
    <t w=""22"">fl</t>
    <t w=""20"">ft</t>
    <t w=""20"">bat</t>
    <t w=""19"">at</t>
  </item>
</recoIndex>
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718949182,Better handling of OCR data,
https://github.com/dogsheep/healthkit-to-sqlite/issues/10#issuecomment-550806302,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/10,550806302,MDEyOklzc3VlQ29tbWVudDU1MDgwNjMwMg==,9599,simonw,2019-11-07T05:33:31Z,2019-11-07T05:33:31Z,MEMBER,"The XML now includes references to these new files:

![CBBA54FC-51FB-4BB3-927C-C2CA99237B04](https://user-images.githubusercontent.com/9599/68362716-121ec400-00dd-11ea-9846-387c7cd64c8b.jpeg)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",519038979,Failed to import workout points,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606824992,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606824992,MDEyOklzc3VlQ29tbWVudDYwNjgyNDk5Mg==,9599,simonw,2020-03-31T19:24:23Z,2020-03-31T19:24:23Z,MEMBER,"The `--since` option is actually used by four commands:

* `user-timeline`
* `home-timeline`
* `mentions-timeline`
* `search`

All of them use the same `fetch_timeline()` utility function under the hood. I should move the logic that looks up the last `since_id` into that shared function.

Question: should I have a table for each of those four methods or a single table that is used by them all? I'm leaning towards four separate tables.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/twitter-to-sqlite/issues/19#issuecomment-542832952,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19,542832952,MDEyOklzc3VlQ29tbWVudDU0MjgzMjk1Mg==,9599,simonw,2019-10-16T18:30:11Z,2019-10-16T18:30:11Z,MEMBER,"The `--since` option will derive the `since_id` from the max ID in the `timeline_tweets` table:

    $ twitter-to-sqlite home-timeline --since

The `--since_id=xxx` option lets you specify that ID directly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506087267,since_id support for home-timeline,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408220,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,624408220,MDEyOklzc3VlQ29tbWVudDYyNDQwODIyMA==,9599,simonw,2020-05-06T02:18:47Z,2020-05-06T02:18:47Z,MEMBER,"The `apple_photos` table has an indexed `uuid` column and a `path` column which stores the full path to that photo file on disk.

I can write a custom Datasette plugin which takes the `uuid` from the URL, looks up the path, then serves up a thumbnail of the jpeg or heic image file.

I'll prototype this is a one-off plugin first, then package it on PyPI for other people to install.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/github-to-sqlite/issues/33#issuecomment-622171097,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/33,622171097,MDEyOklzc3VlQ29tbWVudDYyMjE3MTA5Nw==,9599,simonw,2020-04-30T23:22:45Z,2020-04-30T23:23:57Z,MEMBER,"The `auth.json` mechanism this uses is standard across all of the other Dogsheep tools - it's actually designed so you can have one `auth.json` with a bunch of different credentials for different tools:

```json
{
    ""goodreads_personal_token"": ""..."",
    ""goodreads_user_id"": ""..."",
    ""github_personal_token"": ""..."",
    ""pocket_consumer_key"": ""..."",
    ""pocket_username"": ""..."",
    ""pocket_access_token"": ""...""
}
```
But... `github-to-sqlite` does feel like it deserves a special case here, since it's such a good fit for running inside of GitHub Actions - which even provide a `GITHUB_TOKEN` for you to use!

So I don't think it will harm the family of tools too much if this has an environment variable alternative to the `-a` file.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",609950090,Fall back to authentication via ENV,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790370485,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790370485,MDEyOklzc3VlQ29tbWVudDc5MDM3MDQ4NQ==,9599,simonw,2021-03-04T06:57:25Z,2021-03-04T06:57:48Z,MEMBER,"The command takes quite a while to start running, presumably because this line causes it to have to scan the WHOLE file in order to generate a count:

https://github.com/dogsheep/google-takeout-to-sqlite/blob/a3de045eba0fae4b309da21aa3119102b0efc576/google_takeout_to_sqlite/utils.py#L66-L67

I'm fine with waiting though. It's not like this is a command people run every day - and without that count we can't show a progress bar, which seems pretty important for a process that takes this long.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/twitter-to-sqlite/issues/40#issuecomment-606307019,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40,606307019,MDEyOklzc3VlQ29tbWVudDYwNjMwNzAxOQ==,9599,simonw,2020-03-30T23:34:27Z,2020-03-30T23:34:27Z,MEMBER,"The count properties available for a user are:

* followers_count
* friends_count
* listed_count
* favourites_count
* statuses_count

May as well track history for all of them? Should be pretty cheap to store.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590669793,Feature: record history of follower counts,
https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-905203570,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13,905203570,IC_kwDOEhK-wc419E9y,9599,simonw,2021-08-25T05:51:22Z,2021-08-25T05:53:27Z,MEMBER,"The debugger showed me that it broke on a string that looked like this:
```xml
<?xml version=""1.0"" encoding=""UTF-8"" standalone=""no""?>
<!DOCTYPE en-note SYSTEM ""http://xml.evernote.com/pub/enml2.dtd"">
<en-note>
  <h1 title=""Q3 2018 Reflection & Development"">
    <span title=Q3 2018 Reflection & Development"">
      Q3 2018 Reflection & Development
    </span>
  </h1>
  ...
```
Yeah that is not valid XML!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",978743426,xml.etree.ElementTree.ParseError: not well-formed (invalid token),
https://github.com/dogsheep/dogsheep-photos/issues/14#issuecomment-620774507,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14,620774507,MDEyOklzc3VlQ29tbWVudDYyMDc3NDUwNw==,9599,simonw,2020-04-28T18:19:06Z,2020-04-28T18:19:06Z,MEMBER,"The default timeout is a bit aggressive and sometimes failed for me if my resizing proxy took too long to fetch and resize the image.

`client.annotate_image(..., timeout=3.0)` may be worth trying.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",608512747,Annotate photos using the Google Cloud Vision API,
https://github.com/dogsheep/google-takeout-to-sqlite/issues/2#issuecomment-747126777,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/2,747126777,MDEyOklzc3VlQ29tbWVudDc0NzEyNjc3Nw==,9599,simonw,2020-12-17T00:36:52Z,2020-12-17T00:36:52Z,MEMBER,The memory profiler tricks I used in https://github.com/dogsheep/healthkit-to-sqlite/issues/7 could help figure out what's going on here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",769376447,killed by oomkiller on large location-history,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408370,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,624408370,MDEyOklzc3VlQ29tbWVudDYyNDQwODM3MA==,9599,simonw,2020-05-06T02:19:27Z,2020-05-06T02:19:27Z,MEMBER,"The plugin can be generalized: it can be configured to know how to take the URL path, look it up in ANY table (via a custom SQL query) to get a path on disk and then serve that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/twitter-to-sqlite/issues/39#issuecomment-606998669,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/39,606998669,MDEyOklzc3VlQ29tbWVudDYwNjk5ODY2OQ==,9599,simonw,2020-04-01T02:57:36Z,2020-04-01T02:57:36Z,MEMBER,"The tricky thing here is thinking about the interaction between the recorded since_id and a desire to run the initial import.

The first time you run `twitter-to-sqlite user-timeline db.db username` we want to fetch as many tweets from that user as possible - probably around 3,200 before the API limitations cut us off.

We need to record the maximum ID from those as the `since_id` - which we will see on the very first page we paginate through. That way next time we run the command with `--since` we will only fetch new tweets.

But what happens if our initial import is cancelled after only a few tweets? We risk never pulling in the rest of the tweets.

Not sure if I need to solve this at all or if I should instead trust users to run the command a second time without `--since` if they think they didn't retrieve anything the first time.

I had considered letting `--stop_after=` over-ride `--since` but that doesn't actually make sense - if you send a since_id to the Twitter API you'll never get back more tweets than exist after that ID, so the `--stop_after` would not make a meaningful difference.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",590666760,--since feature can be confused by retweets,
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631227105,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631227105,MDEyOklzc3VlQ29tbWVudDYzMTIyNzEwNQ==,9599,simonw,2020-05-20T04:21:06Z,2020-05-20T04:21:06Z,MEMBER,Then I just need to push a final photos-to-sqlite release that updates the README to tell people about the name change.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623855841,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623855841,MDEyOklzc3VlQ29tbWVudDYyMzg1NTg0MQ==,9599,simonw,2020-05-05T04:54:28Z,2020-05-05T04:54:28Z,MEMBER,"Things were not matching up for me correctly:

<img width=""1143"" alt=""search__select_json_object__img_src____https___photos_simonwillison_net_i______photos_sha256___________photos_ext______w_400___as_photo__groups_content_string__assets_uuid_0__assets_uuid_1__to_uuid_assets_uuid_0__assets_uuid_1__as_uuid__pho"" src=""https://user-images.githubusercontent.com/9599/81035923-ca8db080-8e51-11ea-95a7-6ee60bae7502.png"">

I think that's because my import script didn't correctly import the existing `rowid` values.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/twitter-to-sqlite/issues/23#issuecomment-543217890,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/23,543217890,MDEyOklzc3VlQ29tbWVudDU0MzIxNzg5MA==,9599,simonw,2019-10-17T15:03:10Z,2019-10-17T15:03:10Z,MEMBER,"Thinking about this further: the concept of migrations may end up being in direct conflict with the `sqlite-utils` concept of creating tables on demand the first time they are used - and of creating table schemas automatically to fit the shape of the JSON that is being inserted into them.

I'm going to forge ahead anyway and build this because I think it will be an interesting exploration, but it's very likely this turns out to be a bad idea in the long run!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",508190730,Extremely simple migration system,
https://github.com/dogsheep/dogsheep-beta/issues/15#issuecomment-695124698,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/15,695124698,MDEyOklzc3VlQ29tbWVudDY5NTEyNDY5OA==,9599,simonw,2020-09-18T23:17:38Z,2020-09-18T23:17:38Z,MEMBER,This can be part of the demo instance in #6.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694136490,Add a bunch of config examples,
https://github.com/dogsheep/github-to-sqlite/issues/10#issuecomment-549230337,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10,549230337,MDEyOklzc3VlQ29tbWVudDU0OTIzMDMzNw==,9599,simonw,2019-11-04T05:47:18Z,2019-11-04T05:47:18Z,MEMBER,This definition isn't quite right - it's not pulling the identity of the user who starred the repo (`users.login` ends up being the owner login instead).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516967682,Add this repos_starred view,
https://github.com/dogsheep/apple-notes-to-sqlite/issues/1#issuecomment-1461230436,https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/1,1461230436,IC_kwDOJHON9s5XGJtk,9599,simonw,2023-03-09T03:51:52Z,2023-03-09T03:51:52Z,MEMBER,This did the job! Next step is to turn that into a Python script.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1616347574,Initial proof of concept with ChatGPT,
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-614843406,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27,614843406,MDEyOklzc3VlQ29tbWVudDYxNDg0MzQwNg==,9599,simonw,2020-04-16T19:11:53Z,2020-04-16T19:20:23Z,MEMBER,"This didn't quite work: the column type is incorrect, so the foreign key relationship isn't sticking:

https://github-to-sqlite.dogsheep.net/github/repos?organization=53015001

`[organization] TEXT REFERENCES [users]([id])` - should be `INTEGER`.

The problem is that if the first repo inserted has no organization it's set to `null`, which `sqlite-utils` derives as a `TEXT` column.

One solution would be to create the column explicitly with a type, but this could get messy.

I think I want a new sqlite-utils feature for this instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601330277,Repos have a big blob of JSON in the organization column,
https://github.com/dogsheep/dogsheep-beta/issues/8#issuecomment-685960072,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/8,685960072,MDEyOklzc3VlQ29tbWVudDY4NTk2MDA3Mg==,9599,simonw,2020-09-02T19:50:47Z,2020-09-02T19:50:47Z,MEMBER,"This doesn't actually help, because the Datasette table view page doesn't then support adding the `where search_index_fts match :query` bit.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691369691,Create a view for running faceted searches,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623846880,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623846880,MDEyOklzc3VlQ29tbWVudDYyMzg0Njg4MA==,9599,simonw,2020-05-05T04:06:08Z,2020-05-05T04:06:08Z,MEMBER,"This function seems to convert them into UUIDs that match my photos:
```python
def to_uuid(uuid_0, uuid_1):
    b = uuid_0.to_bytes(8, 'little', signed=True) + uuid_1.to_bytes(8, 'little', signed=True)
    return str(uuid.UUID(bytes=b)).upper()
```","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602924714,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602924714,MDEyOklzc3VlQ29tbWVudDYwMjkyNDcxNA==,9599,simonw,2020-03-24T00:03:25Z,2020-03-24T00:03:25Z,MEMBER,This is good enough for the 1.0 release.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615947370,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615947370,MDEyOklzc3VlQ29tbWVudDYxNTk0NzM3MA==,9599,simonw,2020-04-18T20:52:13Z,2020-04-18T20:52:13Z,MEMBER,"This is great! I now have a key that can upload photos, and a separate key that can download photos OR generate signed URLs to access those photos.

Next step: a script that starts uploading my photos.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/twitter-to-sqlite/issues/56#issuecomment-769957751,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/56,769957751,MDEyOklzc3VlQ29tbWVudDc2OTk1Nzc1MQ==,9599,simonw,2021-01-29T17:59:40Z,2021-01-29T17:59:40Z,MEMBER,"This is interesting - how did you create that initial table? Was this using the `twitter-to-sqlite import archive.db ~/Downloads/twitter-2019-06-25-b31f2.zip` command, or something else?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",796736607,Not all quoted statuses get fetched?,
https://github.com/dogsheep/github-to-sqlite/issues/38#issuecomment-623027889,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38,623027889,MDEyOklzc3VlQ29tbWVudDYyMzAyNzg4OQ==,9599,simonw,2020-05-02T23:15:11Z,2020-05-02T23:15:11Z,MEMBER,"This is one of the use-cases for the `repos_starred` view: it allows you to easily run this kid of query without having to construct the SQL by hand. Here's a demo:

https://github-to-sqlite.dogsheep.net/github/repos_starred?name__contains=twitter

My philosophy here is to keep the raw tables (like `stars`) as normalized as possible, then use SQL views which expose the data in a form that's easier to query.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",611284481,[Feature Request] Support Repo Name in Search 🥺,
https://github.com/dogsheep/dogsheep-beta/issues/13#issuecomment-686774592,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/13,686774592,MDEyOklzc3VlQ29tbWVudDY4Njc3NDU5Mg==,9599,simonw,2020-09-03T21:30:21Z,2020-09-03T21:30:21Z,MEMBER,"This is partially supported: the custom search SQL we run doesn't escape them, but the `?_search` used to calculate facet counts does. So this is a bug.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",692386625,Support advanced FTS queries,
https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694552681,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694552681,MDEyOklzc3VlQ29tbWVudDY5NDU1MjY4MQ==,9599,simonw,2020-09-17T23:25:54Z,2020-09-17T23:25:54Z,MEMBER,"This is the template fragment it's rendering:
```html+jinja
            <div style=""overflow: hidden;"">
              <p>Tweet by <a href=""https://twitter.com/{{ display.screen_name }}"">@{{ display.screen_name }}</a> ({{ display.user_name }}, {{ ""{:,}"".format(display.followers_count or 0) }} followers)
                on <a href=""https://twitter.com/{{ display.screen_name }}/status/{{ display.tweet_id }}"">{{ display.created_at }}</a></p>
              </p>
              <blockquote>{{ display.full_text }}</blockquote>
              {% if display.media_urls and json.loads(display.media_urls) %}
                {% for url in json.loads(display.media_urls) %}
                  <img src=""{{ url }}"" style=""height: 200px;"">
                {% endfor %}
              {% endif %}
            </div>
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'",
https://github.com/dogsheep/github-to-sqlite/issues/51#issuecomment-735484186,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51,735484186,MDEyOklzc3VlQ29tbWVudDczNTQ4NDE4Ng==,9599,simonw,2020-11-30T00:29:31Z,2020-11-30T00:29:31Z,MEMBER,"This just caused a failure in deploying the demo: https://github.com/dogsheep/github-to-sqlite/runs/1471304407?check_suite_focus=true

```
  File ""/opt/hostedtoolcache/Python/3.8.6/x64/bin/github-to-sqlite"", line 33, in <module>
    sys.exit(load_entry_point('github-to-sqlite', 'console_scripts', 'github-to-sqlite')())
  File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 142, in issue_comments
    for comment in utils.fetch_issue_comments(repo, token, issue):
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 380, in fetch_issue_comments
    for comments in paginate(url, headers):
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 472, in paginate
    raise GitHubError.from_response(response)
github_to_sqlite.utils.GitHubError: ('API rate limit exceeded for user ID 9599.', 403)
Error: Process completed with exit code 1.
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703246031,github-to-sqlite should handle rate limits better,
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902329455,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57,902329455,IC_kwDODEm0Qs41yHRv,9599,simonw,2021-08-19T23:59:56Z,2021-08-19T23:59:56Z,MEMBER,"This looks like the bug to me:

https://github.com/dogsheep/twitter-to-sqlite/blob/197e69cec40052c423a5ed071feb5f7cccea41b9/twitter_to_sqlite/cli.py#L239-L241

`type=str, default=False`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907645813,"Error: Use either --since or --since_id, not both",
https://github.com/dogsheep/twitter-to-sqlite/issues/12#issuecomment-542855081,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/12,542855081,MDEyOklzc3VlQ29tbWVudDU0Mjg1NTA4MQ==,9599,simonw,2019-10-16T19:26:56Z,2019-10-16T19:26:56Z,MEMBER,This may be the first case where I want to be able to repair existing databases rather than discarding their contents.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503053800,"Extract ""source"" into a separate lookup table",
https://github.com/dogsheep/twitter-to-sqlite/issues/25#issuecomment-543265058,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/25,543265058,MDEyOklzc3VlQ29tbWVudDU0MzI2NTA1OA==,9599,simonw,2019-10-17T16:51:12Z,2019-10-17T16:51:12Z,MEMBER,"This migration function only runs if there is a table called `tweets` and the migration has not run before.

I think this can happen if the database has just been freshly created (by a command that fetches the user's user timeline for example) and is then run a SECOND time.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",508578780,Ensure migrations don't accidentally create foreign key twice,
https://github.com/dogsheep/dogsheep-photos/issues/24#issuecomment-631255206,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/24,631255206,MDEyOklzc3VlQ29tbWVudDYzMTI1NTIwNg==,9599,simonw,2020-05-20T06:00:25Z,2020-05-20T06:00:25Z,MEMBER,This needs documentation.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621323348,Configurable URL for images,
https://github.com/dogsheep/github-to-sqlite/issues/6#issuecomment-541387822,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/6,541387822,MDEyOklzc3VlQ29tbWVudDU0MTM4NzgyMg==,9599,simonw,2019-10-13T05:27:39Z,2019-10-13T05:27:39Z,MEMBER,This should be fixed by https://github.com/dogsheep/github-to-sqlite/commit/552543a74970f8a3a3f87f887be23a0c6eb1cb5b,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",504238461,sqlite3.OperationalError: table users has no column named bio,
https://github.com/dogsheep/github-to-sqlite/issues/16#issuecomment-613611455,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16,613611455,MDEyOklzc3VlQ29tbWVudDYxMzYxMTQ1NQ==,9599,simonw,2020-04-14T18:37:21Z,2020-04-14T18:37:21Z,MEMBER,"This should have been fixed by #20 and #23 

@jayvdb I'm definitely interested in this tool working as a library - it's purely designed as a CLI tool at the moment, but cleaning it up to work better as a dependency is totally in-scope for the project. https://sqlite-utils.readthedocs.io/ is an example of a tool I've built that works for both.

Feel free to open a new issue here with some notes on what you would need for this to work as a library for your project!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",546051181,Exception running first command: IndexError: list index out of range,
https://github.com/dogsheep/dogsheep-beta/issues/3#issuecomment-685961809,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3,685961809,MDEyOklzc3VlQ29tbWVudDY4NTk2MTgwOQ==,9599,simonw,2020-09-02T19:54:24Z,2020-09-02T19:54:24Z,MEMBER,"This should implement search highlighting too, as seen on https://til.simonwillison.net/til/search?q=cloud

<img width=""1029"" alt=""TIL_search__cloud"" src=""https://user-images.githubusercontent.com/9599/92029959-32c6a300-ed1b-11ea-8b5e-971950980c38.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",689810340,"Datasette plugin to provide custom page for running faceted, ranked searches",
https://github.com/dogsheep/github-to-sqlite/issues/35#issuecomment-622214262,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35,622214262,MDEyOklzc3VlQ29tbWVudDYyMjIxNDI2Mg==,9599,simonw,2020-05-01T02:10:32Z,2020-05-01T02:11:19Z,MEMBER,"This sped that query up even more - down to 4ms.
```sql
create index issue_comments_issue on issue_comments(issue);
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610511450,Create index on issue_comments(user) and other foreign keys,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-601980724,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,601980724,MDEyOklzc3VlQ29tbWVudDYwMTk4MDcyNA==,9599,simonw,2020-03-21T02:28:05Z,2020-03-21T02:28:05Z,MEMBER,"This suggests that `commit[""author""]` can be `None` in some cases?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/github-to-sqlite/issues/37#issuecomment-622461537,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37,622461537,MDEyOklzc3VlQ29tbWVudDYyMjQ2MTUzNw==,9599,simonw,2020-05-01T16:35:40Z,2020-05-01T16:35:40Z,MEMBER,"This will check if the view exists and has the exact same matching definition as the one we want. If it doesn't, we will drop it (if it exists) and recreate it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610843136,Mechanism for creating views if they don't yet exist,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615944806,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615944806,MDEyOklzc3VlQ29tbWVudDYxNTk0NDgwNg==,9599,simonw,2020-04-18T20:41:39Z,2020-04-18T20:41:39Z,MEMBER,"This worked!

![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670712-d868e380-8179-11ea-82a5-5dfd17356113.png)

And this worked:

![Dogsheep_Photos_S3_access](https://user-images.githubusercontent.com/9599/79670777-50370e00-817a-11ea-83cd-18ebf5702878.png)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-747030964,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29,747030964,MDEyOklzc3VlQ29tbWVudDc0NzAzMDk2NA==,9599,simonw,2020-12-16T21:14:54Z,2020-12-16T21:14:54Z,MEMBER,"To do this I'll need the search term to be passed to the `display_sql` SQL query:
https://github.com/dogsheep/dogsheep-beta/blob/4890ec87b5e2ec48940f32c9ad1f5aae25c75a4d/dogsheep_beta/__init__.py#L164-L171","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",724759588,Add search highlighting snippets,
https://github.com/dogsheep/dogsheep-photos/issues/1#issuecomment-623198986,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/1,623198986,MDEyOklzc3VlQ29tbWVudDYyMzE5ODk4Ng==,9599,simonw,2020-05-03T23:12:31Z,2020-05-03T23:12:46Z,MEMBER,"To get the taken date in UTC:
```
from datetime import timezone
(Pdb) photo.date.astimezone(timezone.utc).isoformat()
'2018-02-13T20:21:31.620000+00:00'
(Pdb) photo.date.astimezone(timezone.utc).isoformat().split(""."")
['2018-02-13T20:21:31', '620000+00:00']
(Pdb) photo.date.astimezone(timezone.utc).isoformat().split(""."")[0]
'2018-02-13T20:21:31'
(Pdb) photo.date.astimezone(timezone.utc).isoformat().split(""."")[0] + ""+00:00""
'2018-02-13T20:21:31+00:00'

```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533300,Import photo metadata from Apple Photos into SQLite,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623855885,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623855885,MDEyOklzc3VlQ29tbWVudDYyMzg1NTg4NQ==,9599,simonw,2020-05-05T04:54:39Z,2020-05-05T04:54:53Z,MEMBER,"Trying this import mechanism instead:
`sqlite3 /Users/simon/Pictures/Photos\ Library.photoslibrary/database/search/psi.sqlite .dump | grep -v 'CREATE INDEX' | grep -v 'CREATE TRIGGER' | grep -v 'CREATE VIRTUAL TABLE' | sqlite3 search.db`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/github-to-sqlite/issues/25#issuecomment-614810417,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/25,614810417,MDEyOklzc3VlQ29tbWVudDYxNDgxMDQxNw==,9599,simonw,2020-04-16T18:07:11Z,2020-04-16T18:07:11Z,MEMBER,Turns out the main problem was #26 - now fixed.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601265023,Improvements to demo instance,
https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-623739934,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15,623739934,MDEyOklzc3VlQ29tbWVudDYyMzczOTkzNA==,9599,simonw,2020-05-04T22:24:26Z,2020-05-04T22:24:26Z,MEMBER,Twitter thread with some examples of photos that are coming up from queries against these scores: https://twitter.com/simonw/status/1257434670750408705,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612151767,Expose scores from ZCOMPUTEDASSETATTRIBUTES,
https://github.com/dogsheep/healthkit-to-sqlite/issues/12#issuecomment-879477586,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/12,879477586,MDEyOklzc3VlQ29tbWVudDg3OTQ3NzU4Ng==,9599,simonw,2021-07-13T23:50:06Z,2021-07-13T23:50:06Z,MEMBER,"Unfortunately I don't think updating the database is practical, because the export doesn't include unique identifiers which can be used to update existing records and create new ones. Recreating from scratch works around that limitation.

I've not explored workouts with SpatiaLite but that's a really good idea.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",727848625,"Some workout columns should be float, not text",
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622133298,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622133298,MDEyOklzc3VlQ29tbWVudDYyMjEzMzI5OA==,9599,simonw,2020-04-30T21:48:24Z,2020-04-30T21:48:24Z,MEMBER,"Unfortunately it's not available through any GitHub API - I managed to figure out how to get dependencies, but I need dependents. https://github.com/simonw/til/blob/master/github/dependencies-graphql-api.md","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,Command for retrieving dependents for a repo,
https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631253136,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25,631253136,MDEyOklzc3VlQ29tbWVudDYzMTI1MzEzNg==,9599,simonw,2020-05-20T05:53:58Z,2020-05-20T05:53:58Z,MEMBER,"Updated deploy command:
```
datasette publish now public.db --project dogsheep-photos \
    --about=dogsheep/dogsheep-photos \
    --about_url=""https://github.com/dogsheep/dogsheep-photos"" \
    --install=datasette-json-html \
    --install=datasette-cluster-map \
    --title ""Dogsheep Photos demo""
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621332242,Create a public demo,
https://github.com/dogsheep/twitter-to-sqlite/issues/19#issuecomment-542849963,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19,542849963,MDEyOklzc3VlQ29tbWVudDU0Mjg0OTk2Mw==,9599,simonw,2019-10-16T19:13:06Z,2019-10-16T19:13:06Z,MEMBER,Updated documentation: https://github.com/dogsheep/twitter-to-sqlite/blob/fced2a9b67d2cbdf9817f1eb75f7c28e413c963b/README.md#retrieving-tweets-from-your-home-timeline,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506087267,since_id support for home-timeline,
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602895896,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602895896,MDEyOklzc3VlQ29tbWVudDYwMjg5NTg5Ng==,9599,simonw,2020-03-23T22:42:25Z,2020-03-23T22:42:25Z,MEMBER,"Urgh this is such a mess! I should have done this on a branch / pull request to avoid polluting my main master history, but never mind.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/pocket-to-sqlite/issues/2#issuecomment-538847796,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2,538847796,MDEyOklzc3VlQ29tbWVudDUzODg0Nzc5Ng==,9599,simonw,2019-10-07T05:43:30Z,2019-10-07T05:43:30Z,MEMBER,We can persist the `since` value in its own single-row table.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",503234169,Track and use the 'since' value,
https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776180,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776180,MDEyOklzc3VlQ29tbWVudDcwNjc3NjE4MA==,9599,simonw,2020-10-11T22:17:55Z,2020-10-11T22:17:55Z,MEMBER,"We could even do server-side thumbnailing for some of these images, but I'm inclined to serve up the full size ones and set a width on the image element based on the `width` attribute on `<en-media>`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from <en-media> tags inline in Datasette,
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902328760,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57,902328760,IC_kwDODEm0Qs41yHG4,9599,simonw,2021-08-19T23:57:41Z,2021-08-19T23:57:41Z,MEMBER,"Weird, added debug code and got this: `{'screen_name': 'simonw', 'count': 200, 'since_id': 'False', 'tweet_mode': 'extended'}` -  so maybe it's a `twitter-to-sqlite` bug where somehow the string `False` is being passed somewhere.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907645813,"Error: Use either --since or --since_id, not both",
https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-622989874,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12,622989874,MDEyOklzc3VlQ29tbWVudDYyMjk4OTg3NA==,9599,simonw,2020-05-02T17:46:14Z,2020-05-02T17:46:14Z,MEMBER,Without the rowid column facet by topics breaks: https://github-to-sqlite.dogsheep.net/github/recent_releases?_facet=repo&_facet_array=topics&topics__arraycontains=datasette-io,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520756546,Add this view for seeing new releases,
https://github.com/dogsheep/github-to-sqlite/issues/13#issuecomment-602916947,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13,602916947,MDEyOklzc3VlQ29tbWVudDYwMjkxNjk0Nw==,9599,simonw,2020-03-23T23:38:06Z,2020-03-23T23:38:06Z,MEMBER,Woohoo! https://github-to-sqlite-j7hipcg4aq-uc.a.run.app/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",521275281,Set up a live demo Datasette instance,
https://github.com/dogsheep/healthkit-to-sqlite/issues/20#issuecomment-902355471,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20,902355471,IC_kwDOC8tyDs41yNoP,9599,simonw,2021-08-20T01:09:07Z,2021-08-20T01:09:07Z,MEMBER,"Workaround:

    sqlite-utils create-index healthkit.db workout_points -- -date

See https://sqlite-utils.datasette.io/en/stable/cli.html#creating-indexes","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",975166271,Add index on workout_points.date,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-790312268,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,790312268,MDEyOklzc3VlQ29tbWVudDc5MDMxMjI2OA==,9599,simonw,2021-03-04T05:48:16Z,2021-03-04T05:48:16Z,MEMBER,"Wow, my mbox is a 10.35 GB download!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,WIP: Add Gmail takeout mbox import,
https://github.com/dogsheep/twitter-to-sqlite/issues/27#issuecomment-549095217,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/27,549095217,MDEyOklzc3VlQ29tbWVudDU0OTA5NTIxNw==,9599,simonw,2019-11-03T01:06:25Z,2019-11-03T01:06:25Z,MEMBER,"Wow, that `retweets_of_me` endpoint is almost completely useless:
```
$ twitter-to-sqlite fetch https://api.twitter.com/1.1/statuses/retweets_of_me.json
```
It returns my own tweets that have been retweeted, but with no indication at all of who retweeted them.

It looks like this needs to be combined with this API - https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-retweets-id - to fetch the details of up to 100 recent users who actually DID retweet an individual status. But that has a one-every-12-seconds rate limit on it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",514459062,retweets-of-me command,
https://github.com/dogsheep/github-to-sqlite/issues/5#issuecomment-602806530,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/5,602806530,MDEyOklzc3VlQ29tbWVudDYwMjgwNjUzMA==,9599,simonw,2020-03-23T19:23:30Z,2020-03-23T19:23:30Z,MEMBER,Yeah `alter=True` means that this isn't necessary.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493671014,"Add ""incomplete"" boolean to users table for incomplete profiles",
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623863902,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623863902,MDEyOklzc3VlQ29tbWVudDYyMzg2MzkwMg==,9599,simonw,2020-05-05T05:31:53Z,2020-05-05T05:31:53Z,MEMBER,"Yes! Turning those `rowid` values into `id` with this script did the job:
```python
import sqlite3
import sqlite_utils

conn = sqlite3.connect(
    ""/Users/simon/Pictures/Photos Library.photoslibrary/database/search/psi.sqlite""
)


def all_rows(table):
    result = conn.execute(""select rowid as id, * from {}"".format(table))
    cols = [c[0] for c in result.description]
    for row in result.fetchall():
        yield dict(zip(cols, row))


if __name__ == ""__main__"":
    db = sqlite_utils.Database(""psi_copy.db"")
    for table in (""assets"", ""collections"", ""ga"", ""gc"", ""groups""):
        db[table].upsert_all(all_rows(table), pk=""id"", alter=True)
```
Then I ran this query:
```sql
select 
  json_object('img_src', 'https://photos.simonwillison.net/i/' || photos.sha256 || '.' || photos.ext || '?w=400') as photo,
   group_concat(strip_null_chars(groups.content_string), ' ') as words, assets.uuid_0, assets.uuid_1, to_uuid(assets.uuid_0, assets.uuid_1) as uuid
from assets join ga on assets.id = ga.assetid
join groups on ga.groupid = groups.id
join photos on photos.uuid = to_uuid(assets.uuid_0, assets.uuid_1)
where groups.category = 2024
group by assets.id
order by random() limit 10
```
And got these results!
<img width=""1054"" alt=""psi_copy__select_json_object__img_src____https___photos_simonwillison_net_i______photos_sha256___________photos_ext______w_400___as_photo__group_concat_strip_null_chars_groups_content_string________as_words__assets_uuid_0__assets_uuid_1__to"" src=""https://user-images.githubusercontent.com/9599/81037264-f1021a80-8e56-11ea-9924-6f9f55a0fb4b.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,"Import machine-learning detected labels (dog, llama etc) from Apple Photos",
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395781,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626395781,MDEyOklzc3VlQ29tbWVudDYyNjM5NTc4MQ==,9599,simonw,2020-05-10T21:57:09Z,2020-05-10T21:57:09Z,MEMBER,"Yes, I just recreated my virtual environment from scratch and the error went away.

The problem occurred when I ran `pip install datasette-bplist` in the same virtual environment - https://github.com/simonw/datasette-bplist/blob/master/setup.py depends on `bpylist` which is incompatible with `bpylist2`.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,bpylist.archiver.CircularReference: archive has a cycle with uid(13),
https://github.com/dogsheep/github-to-sqlite/issues/54#issuecomment-735464493,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54,735464493,MDEyOklzc3VlQ29tbWVudDczNTQ2NDQ5Mw==,9599,simonw,2020-11-29T21:57:32Z,2020-11-29T21:57:32Z,MEMBER,`$ github-to-sqlite workflows github.db simonw/datasette dogsheep/github-to-sqlite`,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",753026003,github-to-sqlite workflows command,
https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-529239307,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8,529239307,MDEyOklzc3VlQ29tbWVudDUyOTIzOTMwNw==,9599,simonw,2019-09-08T20:36:49Z,2019-09-08T20:36:49Z,MEMBER,"`--attach` can optionally take a name for the database connection alias like this:

    $ twitter-to-sqlite users-lookup users.db --attach foo:attending.db ...

If you omit the `alias:` bit the stem of the database (without the file extension) will be used.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",490803176,--sql and --attach options for feeding commands from SQL queries,
https://github.com/dogsheep/github-to-sqlite/issues/28#issuecomment-615883040,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28,615883040,MDEyOklzc3VlQ29tbWVudDYxNTg4MzA0MA==,9599,simonw,2020-04-18T14:45:38Z,2020-04-18T14:45:38Z,MEMBER,"```
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py"", line 219, in contributors
    utils.save_contributors(db, contributors, repo_full[""id""])
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 354, in save_contributors
    for contributor in contributors:
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 228, in fetch_contributors
    for contributors in paginate(url, headers):
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 286, in paginate
    data = response.json()
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/requests/models.py"", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/json/__init__.py"", line 357, in loads
    return _default_decoder.decode(s)
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/json/decoder.py"", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File ""/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/json/decoder.py"", line 355, in raw_decode
    raise JSONDecodeError(""Expecting value"", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601333634,Pull repository contributors,
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-615519409,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27,615519409,MDEyOklzc3VlQ29tbWVudDYxNTUxOTQwOQ==,9599,simonw,2020-04-18T00:19:16Z,2020-04-18T00:19:16Z,MEMBER,"```
$ github-to-sqlite repos b.db dogsheep
$ sqlite3 b.db '.schema repos'
CREATE TABLE [repos] (
   [id] INTEGER PRIMARY KEY,
...
   [permissions] TEXT,
   [organization] INTEGER REFERENCES [users]([id]),
   FOREIGN KEY(license) REFERENCES licenses(key)
);
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601330277,Repos have a big blob of JSON in the organization column,
https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660547502,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660547502,MDEyOklzc3VlQ29tbWVudDY2MDU0NzUwMg==,9599,simonw,2020-07-18T21:50:37Z,2020-07-18T21:50:37Z,MEMBER,"```
$ github-to-sqlite tags tags.db simonw/datasette dogsheep/github-to-sqlite
$ sqlite-utils tables tags.db --counts
[{""table"": ""users"", ""count"": 2},
 {""table"": ""licenses"", ""count"": 1},
 {""table"": ""repos"", ""count"": 2},
 {""table"": ""tags"", ""count"": 76},
 {""table"": ""licenses_fts"", ""count"": 1},
 {""table"": ""licenses_fts_data"", ""count"": 3},
 {""table"": ""licenses_fts_idx"", ""count"": 1},
 {""table"": ""licenses_fts_docsize"", ""count"": 1},
 {""table"": ""licenses_fts_config"", ""count"": 1},
 {""table"": ""repos_fts"", ""count"": 2},
 {""table"": ""repos_fts_data"", ""count"": 3},
 {""table"": ""repos_fts_idx"", ""count"": 1},
 {""table"": ""repos_fts_docsize"", ""count"": 2},
 {""table"": ""repos_fts_config"", ""count"": 1},
 {""table"": ""users_fts"", ""count"": 2},
 {""table"": ""users_fts_data"", ""count"": 3},
 {""table"": ""users_fts_idx"", ""count"": 1},
 {""table"": ""users_fts_docsize"", ""count"": 2},
 {""table"": ""users_fts_config"", ""count"": 1}]
$ sqlite-utils rows tags.db tags
[{""repo_id"": 107914493, ""name"": ""0.45"", ""sha"": ""f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb""},
 {""repo_id"": 107914493, ""name"": ""0.45a5"", ""sha"": ""676bb64c877d73f8ff496cef4632f5a8a5a9283c""},
 {""repo_id"": 107914493, ""name"": ""0.45a4"", ""sha"": ""265483173bc8341dc02c8b782b9b59d2ce8bbedb""},
 {""repo_id"": 107914493, ""name"": ""0.45a3"", ""sha"": ""1f55a4a2b68fa65e56a28baeb7f44122fdeca7e7""},
 {""repo_id"": 107914493, ""name"": ""0.45a2"", ""sha"": ""1a5b7d318fa923edfcefd3df8f64dae2e9c49d3f""},
 {""repo_id"": 107914493, ""name"": ""0.45a1"", ""sha"": ""b59b92b1b0517cf18fa748ff9d0a0bf86298dd43""},
 {""repo_id"": 107914493, ""name"": ""0.45a0"", ""sha"": ""dda932d818b34ccab11730a76554f0a3748d8348""},
 {""repo_id"": 107914493, ""name"": ""0.44"", ""sha"": ""b906030235efbdff536405d66078f4868ce0d3bd""},
 {""repo_id"": 107914493, ""name"": ""0.43"", ""sha"": ""d56f402822df102f9cf1a9a056449d01a15e3aae""},
 {""repo_id"": 107914493, ""name"": ""0.42"", ""sha"": ""af6c6c5d6f929f951c0e63bfd1c82e37a071b50f""},
 {""repo_id"": 107914493, ""name"": ""0.41"", ""sha"": ""182e5c8745c94576718315f7596ccc81e5e2417b""},
 {""repo_id"": 107914493, ""name"": ""0.40"", ""sha"": ""8da108193b08abf140716f8ac499f32309dfe9cf""},
 {""repo_id"": 107914493, ""name"": ""0.39"", ""sha"": ""dedd775512daee49925882654f252df61a9e3b6d""},
 {""repo_id"": 107914493, ""name"": ""0.38"", ""sha"": ""7e357abbc38dcc9d19a2f1df3252668a48e941e4""},
 {""repo_id"": 107914493, ""name"": ""0.37.1"", ""sha"": ""be20e6991eac2baa9b43e9b26ae209bae805ede5""},
 {""repo_id"": 107914493, ""name"": ""0.37"", ""sha"": ""c9e6841482b299fceadc5ad548c2dbf58a8f1227""},
 {""repo_id"": 107914493, ""name"": ""0.36"", ""sha"": ""b031fe97636b80b05fec409ee1dffb7d044fd4e9""},
 {""repo_id"": 107914493, ""name"": ""0.35"", ""sha"": ""30b6f71b306a43605c99bef79302ed5cb22d1924""},
 {""repo_id"": 107914493, ""name"": ""0.34"", ""sha"": ""e7f60d2a9b59752e20de8412f7b0a3e9a5359a31""},
 {""repo_id"": 107914493, ""name"": ""0.33"", ""sha"": ""59e7014c8a0f4102d7dc79f517540c55c49e1554""},
 {""repo_id"": 107914493, ""name"": ""0.32"", ""sha"": ""a95bedb9c423fa6d772c93ef47bc40f13a5bea50""},
 {""repo_id"": 107914493, ""name"": ""0.31.2"", ""sha"": ""b51f258d00bb3c3b401f15d46a1fbd50394dbe1c""},
 {""repo_id"": 107914493, ""name"": ""0.31.1"", ""sha"": ""a22c7761b61baa61b8e3da7d30887468d61d6b83""},
 {""repo_id"": 107914493, ""name"": ""0.31"", ""sha"": ""7f89928062b1a1fdb2625a946f7cd5161e597401""},
 {""repo_id"": 107914493, ""name"": ""0.30.2"", ""sha"": ""2bf7ce5f517d772a16d7855a35a8a75d4456aad7""},
 {""repo_id"": 107914493, ""name"": ""0.30.1"", ""sha"": ""3ca290e0db03bb4747e24203c445873f74512107""},
 {""repo_id"": 107914493, ""name"": ""0.30"", ""sha"": ""8050f9e1ece9afd0236ad38c6458c12a4ad917e6""},
 {""repo_id"": 107914493, ""name"": ""0.29.3"", ""sha"": ""0fc8afde0eb5ef677f4ac31601540d6168c8208d""},
 {""repo_id"": 107914493, ""name"": ""0.29.2"", ""sha"": ""6abe6faff6b035e9334dd05f8c741ae9b7a47440""},
 {""repo_id"": 107914493, ""name"": ""0.29.1"", ""sha"": ""2a94f3719fb2c4335fcda374fa92f87272b02d34""},
 {""repo_id"": 107914493, ""name"": ""0.29"", ""sha"": ""fb7ee8e0ad59a15083234a48e935525f6e7257dd""},
 {""repo_id"": 107914493, ""name"": ""0.28"", ""sha"": ""e518f76c5f5dd0138032bfb26387f5bb91086a3f""},
 {""repo_id"": 107914493, ""name"": ""0.27.1"", ""sha"": ""3f3f29ac9afe7c41ffc48a3bd2af473a53eecc8a""},
 {""repo_id"": 107914493, ""name"": ""0.27"", ""sha"": ""436b8bc1d17c2ab415800ab209204f94e7f7929e""},
 {""repo_id"": 107914493, ""name"": ""0.26.2"", ""sha"": ""a418c8b44f82d456be523c8690cf7236bb648c22""},
 {""repo_id"": 107914493, ""name"": ""0.26.1"", ""sha"": ""4722acc73ce761556b18f5dcbe36b7fef2ee2c69""},
 {""repo_id"": 107914493, ""name"": ""0.26"", ""sha"": ""424e146697309a54c05d5d1ba1f840849ddbafdc""},
 {""repo_id"": 107914493, ""name"": ""0.25.2"", ""sha"": ""b5128fc53fce6a1bf3b16bad9f318451bc1d1263""},
 {""repo_id"": 107914493, ""name"": ""0.25.1"", ""sha"": ""3dc0b3fa8c9b9bd81540ffe20c8b7e7a72465274""},
 {""repo_id"": 107914493, ""name"": ""0.25"", ""sha"": ""57a71377c992753327a16b417daf79df7f506dd1""},
 {""repo_id"": 107914493, ""name"": ""0.24"", ""sha"": ""28872a1fa789f314b0342f4e6182f1c78d6e2bca""},
 {""repo_id"": 107914493, ""name"": ""0.23.2"", ""sha"": ""6df6f712b36f0fe75694174906e31242427a8d1d""},
 {""repo_id"": 107914493, ""name"": ""0.23.1"", ""sha"": ""dea86b9fba78e032ad09673e884e764387daf209""},
 {""repo_id"": 107914493, ""name"": ""0.23"", ""sha"": ""e04f5b0d348ef7275a0a5ab9eb53527105132885""},
 {""repo_id"": 107914493, ""name"": ""0.22.1"", ""sha"": ""5d6252788230d168ba09f379d1d2af867e3302ab""},
 {""repo_id"": 107914493, ""name"": ""0.22"", ""sha"": ""558d9d7bfef3dd633eb16389281b67d42c9bdeef""},
 {""repo_id"": 107914493, ""name"": ""0.21"", ""sha"": ""403211de632cd15f0820cc9399305fc43c187b47""},
 {""repo_id"": 107914493, ""name"": ""0.20"", ""sha"": ""3a5d7951ce8f35118ffdd7f8d86e09b909e1218c""},
 {""repo_id"": 107914493, ""name"": ""0.19"", ""sha"": ""ba9bfa583179c25aaef94b1f44da7eba74620b9a""},
 {""repo_id"": 107914493, ""name"": ""0.18"", ""sha"": ""43ae15c0d14b3e968e8d5bfef72ac0c39783c3a2""},
 {""repo_id"": 107914493, ""name"": ""0.17"", ""sha"": ""fb988ace7c7e2bee5ac142a0eab22431d0675a77""},
 {""repo_id"": 107914493, ""name"": ""0.16"", ""sha"": ""b6539ff04502536bd1fa96e3b1430bdafc456826""},
 {""repo_id"": 107914493, ""name"": ""0.15"", ""sha"": ""7706fe0c67aba5cfe905c7906cae9e0c43cd75b2""},
 {""repo_id"": 107914493, ""name"": ""0.14"", ""sha"": ""2edc652df6d786e4f2c3f073e3567002d248be09""},
 {""repo_id"": 107914493, ""name"": ""0.13"", ""sha"": ""c160f15c3937f8fbe581276f811e8c58f9137bb1""},
 {""repo_id"": 107914493, ""name"": ""0.12"", ""sha"": ""51bdd67691bd69082ae7690af8b905f06050ee80""},
 {""repo_id"": 107914493, ""name"": ""0.11"", ""sha"": ""b0f3d4e375655f0764f3137dbcede324f9bbc0cb""},
 {""repo_id"": 107914493, ""name"": ""0.10"", ""sha"": ""5928c11ee798a232aa4096706cd47e639d1c9fc2""},
 {""repo_id"": 107914493, ""name"": ""0.9"", ""sha"": ""d75f423b6fcfc074b7c6f8f7679da8876f181edd""},
 {""repo_id"": 107914493, ""name"": ""0.8"", ""sha"": ""fe279ab7b4ae99dab295d5cf4d39ad06d782997e""},
 {""repo_id"": 107914493, ""name"": ""0.7"", ""sha"": ""6b3b05b6db0d2a7b7cec8b8dbb4ddc5e12a376b2""},
 {""repo_id"": 207052882, ""name"": ""2.3"", ""sha"": ""7090e43d804724ef3b31ae5ca9efd6ac05f76cbc""},
 {""repo_id"": 207052882, ""name"": ""2.2"", ""sha"": ""4fe69783b55465e7692a807d3a02a710f69c9c42""},
 {""repo_id"": 207052882, ""name"": ""2.1"", ""sha"": ""9d7aed336c8e62bf372caa800cb4aae3985cbae9""},
 {""repo_id"": 207052882, ""name"": ""2.0"", ""sha"": ""44611df1524a03ce305405e5902c9615e3c73a72""},
 {""repo_id"": 207052882, ""name"": ""1.1"", ""sha"": ""5cd34bd07d704487d48ac741ee5da5317afe88d2""},
 {""repo_id"": 207052882, ""name"": ""1.0.1"", ""sha"": ""3b7ab5685de89fcb6fc92d320c0e24b17be05570""},
 {""repo_id"": 207052882, ""name"": ""1.0"", ""sha"": ""1ea30c8fb1d080bd5e38c577e3ad20bb527a2fe6""},
 {""repo_id"": 207052882, ""name"": ""0.7"", ""sha"": ""e35eec4343aa560c58c1634cc228d0d46c442304""},
 {""repo_id"": 207052882, ""name"": ""0.6"", ""sha"": ""9eb737090fafd0e5a7e314be48402374d99e9828""},
 {""repo_id"": 207052882, ""name"": ""0.5"", ""sha"": ""ae9035f8fe5aff1c54bff4c6b4c2e808a44f0f2a""},
 {""repo_id"": 207052882, ""name"": ""0.4"", ""sha"": ""8c6251c31a05c58c2bfbef114247642d1b3dbb44""},
 {""repo_id"": 207052882, ""name"": ""0.3"", ""sha"": ""f697f247468516aa4ee13b1862b59e0dba18d00f""},
 {""repo_id"": 207052882, ""name"": ""0.2"", ""sha"": ""0fe96bc50fb3d7b1c7e4577db0ddf207eaeebbb9""},
 {""repo_id"": 207052882, ""name"": ""0.1.1"", ""sha"": ""321e0284c64dc48b2143311009886293c05edb07""},
 {""repo_id"": 207052882, ""name"": ""0.1"", ""sha"": ""7387c88a3f84704548e81d43b91615c02b61a957""}]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904,github-to-sqlite tags command for fetching tags,
https://github.com/dogsheep/dogsheep-photos/issues/5#issuecomment-615949574,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/5,615949574,MDEyOklzc3VlQ29tbWVudDYxNTk0OTU3NA==,9599,simonw,2020-04-18T21:06:07Z,2020-04-18T21:06:07Z,MEMBER,"```
$ photos-to-sqlite s3-auth
Create S3 credentials and paste them here:

Access key ID: xxx
Secret access key: yyy
$ cat auth.json 
{
    ""access_key_id"": ""xxx"",
    ""secret_access_key"": ""yyy""
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602551638,photos-to-sqlite s3-auth command,
https://github.com/dogsheep/dogsheep-photos/issues/6#issuecomment-615983393,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6,615983393,MDEyOklzc3VlQ29tbWVudDYxNTk4MzM5Mw==,9599,simonw,2020-04-18T23:53:10Z,2020-04-18T23:53:10Z,MEMBER,"```
$ photos-to-sqlite upload photos3.db ~/Pictures/Photos\ Library.photoslibrary/Masters/2020
Uploading 2.09 GB  [##----------------------------------]    6%  00:36:37
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602575575,Add progress bar to upload command,
https://github.com/dogsheep/twitter-to-sqlite/issues/37#issuecomment-615509578,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/37,615509578,MDEyOklzc3VlQ29tbWVudDYxNTUwOTU3OA==,9599,simonw,2020-04-17T23:36:00Z,2020-04-17T23:36:00Z,MEMBER,"```
$ twitter-to-sqlite user-timeline doggo.db doggoenthuonetuh
Traceback (most recent call last):
...
  File ""/Users/simonw/Dropbox/Development/twitter-to-sqlite/twitter_to_sqlite/utils.py"", line 272, in transform_user
    user[""created_at""] = parser.parse(user[""created_at""])
KeyError: 'created_at'
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585353598,"Handle ""User not found"" error",
https://github.com/dogsheep/github-to-sqlite/issues/47#issuecomment-675523053,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/47,675523053,MDEyOklzc3VlQ29tbWVudDY3NTUyMzA1Mw==,9599,simonw,2020-08-18T14:45:53Z,2020-08-18T14:45:53Z,MEMBER,"```
% github-to-sqlite emojis emojis.db --fetch
  [########----------------------------]  397/1682   23%  00:03:43
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681086659,emojis command,
https://github.com/dogsheep/dogsheep-photos/issues/9#issuecomment-618724149,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/9,618724149,MDEyOklzc3VlQ29tbWVudDYxODcyNDE0OQ==,9599,simonw,2020-04-23T23:35:29Z,2020-04-23T23:35:29Z,MEMBER,"```
% photos-to-sqlite upload photos.db ~/Pictures/Photos\ Library.photoslibrary/originals
Fetching existing keys from S3...
Got 22,446 existing keys
Calculating hashes  [####--------------------------------]   13%  00:04:14
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",605938063,"upload command should be resumable, should only upload photos not already uploaded",
https://github.com/dogsheep/dogsheep-photos/issues/13#issuecomment-620273692,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/13,620273692,MDEyOklzc3VlQ29tbWVudDYyMDI3MzY5Mg==,9599,simonw,2020-04-27T22:42:50Z,2020-04-27T22:42:50Z,MEMBER,"```
>>> def ext_counts(directory):
...     counts = {}
...     for path in pathlib.Path(directory).glob(""**/*""):
...         ext = path.suffix
...         counts[ext] = counts.get(ext, 0) + 1
...     return counts
... 
>>> 
>>> ext_counts(""/Users/simon/Pictures/Photos Library.photoslibrary/originals"")
{'': 16, '.heic': 15478, '.jpeg': 21691, '.mov': 946, '.png': 2262, '.gif': 38, '.mp4': 116, '.aae': 2}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",607888367,Also upload movie files,
https://github.com/dogsheep/twitter-to-sqlite/issues/8#issuecomment-529240286,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/8,529240286,MDEyOklzc3VlQ29tbWVudDUyOTI0MDI4Ng==,9599,simonw,2019-09-08T20:48:33Z,2019-09-08T20:48:33Z,MEMBER,"```ATTACH DATABASE ""file:blah.db?mode=ro"" AS foo```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",490803176,--sql and --attach options for feeding commands from SQL queries,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-602811785,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,602811785,MDEyOklzc3VlQ29tbWVudDYwMjgxMTc4NQ==,9599,simonw,2020-03-23T19:33:56Z,2020-03-23T19:34:06Z,MEMBER,"```json
[
  {
    ""sha"": ""a8dc914089d399d9b522ebb51b67f9ac2e8aa6b0"",
    ""node_id"": ""MDY6Q29tbWl0OTMyMDk6YThkYzkxNDA4OWQzOTlkOWI1MjJlYmI1MWI2N2Y5YWMyZThhYTZiMA=="",
    ""commit"": {
      ""author"": {
        ""name"": ""Simon Willison"",
        ""email"": ""simon@..."",
        ""date"": ""2008-12-18T23:17:12Z""
      },
      ""committer"": {
        ""name"": ""Simon Willison"",
        ""email"": ""simon@..."",
        ""date"": ""2008-12-18T23:17:12Z""
      },
      ""message"": ""First commit"",
      ""tree"": {
        ""sha"": ""ac2dfb75e2592c59165c2880f3f7a16dafd452a1"",
        ""url"": ""https://api.github.com/repos/simonw/simonw.github.com/git/trees/ac2dfb75e2592c59165c2880f3f7a16dafd452a1""
      },
      ""url"": ""https://api.github.com/repos/simonw/simonw.github.com/git/commits/a8dc914089d399d9b522ebb51b67f9ac2e8aa6b0"",
      ""comment_count"": 0,
      ""verification"": {
        ""verified"": false,
        ""reason"": ""unsigned"",
        ""signature"": null,
        ""payload"": null
      }
    },
    ""url"": ""https://api.github.com/repos/simonw/simonw.github.com/commits/a8dc914089d399d9b522ebb51b67f9ac2e8aa6b0"",
    ""html_url"": ""https://github.com/simonw/simonw.github.com/commit/a8dc914089d399d9b522ebb51b67f9ac2e8aa6b0"",
    ""comments_url"": ""https://api.github.com/repos/simonw/simonw.github.com/commits/a8dc914089d399d9b522ebb51b67f9ac2e8aa6b0/comments"",
    ""author"": null,
    ""committer"": null,
    ""parents"": []
  }
]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/healthkit-to-sqlite/issues/2#issuecomment-513439411,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/2,513439411,MDEyOklzc3VlQ29tbWVudDUxMzQzOTQxMQ==,9599,simonw,2019-07-20T05:58:57Z,2019-07-20T05:58:57Z,MEMBER,"```python
 'Workout': {'attr_counts': {'creationDate': 663,
                             'device': 230,
                             'duration': 663,
                             'durationUnit': 663,
                             'endDate': 663,
                             'sourceName': 663,
                             'sourceVersion': 663,
                             'startDate': 663,
                             'totalDistance': 663,
                             'totalDistanceUnit': 663,
                             'totalEnergyBurned': 663,
                             'totalEnergyBurnedUnit': 663,
                             'workoutActivityType': 663},
             'child_counts': {'MetadataEntry': 1928,
                              'WorkoutEvent': 2094,
                              'WorkoutRoute': 340},
             'count': 663,
             'parent_counts': {'HealthData': 663}},
 'WorkoutEvent': {'attr_counts': {'date': 2094,
                                  'duration': 837,
                                  'durationUnit': 837,
                                  'type': 2094},
                  'child_counts': {},
                  'count': 2094,
                  'parent_counts': {'Workout': 2094}},
 'WorkoutRoute': {'attr_counts': {'creationDate': 340,
                                  'endDate': 340,
                                  'sourceName': 340,
                                  'sourceVersion': 340,
                                  'startDate': 340},
                  'child_counts': {'Location': 398683, 'MetadataEntry': 546},
                  'count': 340,
                  'parent_counts': {'Workout': 340}}}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",470637152,Import workouts,
https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660551397,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660551397,MDEyOklzc3VlQ29tbWVudDY2MDU1MTM5Nw==,9599,simonw,2020-07-18T22:27:32Z,2020-07-18T23:05:45Z,MEMBER,"```sql
with most_recent_releases as (
  with ranked as (
    select
      repo,
      tag_name,
      published_at,
      row_number() OVER (
        partition BY repo
        ORDER BY
          published_at DESC
      ) rank
    FROM
      releases
  )
  select
    *
  from
    ranked
  where
    rank = 1
)
select
  repos.full_name as repo,
  most_recent_releases.tag_name as release,
  commits.committer_date as release_commit_date,
  (
    select
      count(*)
    from
      commits c2
    where
      c2.repo = repos.id
      and c2.committer_date > commits.committer_date
  ) as commits_since_release,
  'https://github.com/' || repos.full_name || '/compare/' || most_recent_releases.tag_name || '...' || repos.default_branch as view_commits
from
  most_recent_releases
  join repos on most_recent_releases.repo = repos.id
  join tags on tags.repo = repos.id
  and tags.name = most_recent_releases.tag_name
  join commits on tags.sha = commits.sha
order by
  commits_since_release desc
```


repo | release | release_commit_date | commits_since_release | view_commits
-- | -- | -- | -- | --
simonw/datasette | 0.45 | 2020-07-01T21:43:07Z | 9 | https://github.com/simonw/datasette/compare/0.45...master
dogsheep/twitter-to-sqlite | 0.21.1 | 2020-04-30T18:20:43Z | 2 | https://github.com/dogsheep/twitter-to-sqlite/compare/0.21.1...master
dogsheep/github-to-sqlite | 2.3 | 2020-07-09T23:26:34Z | 2 | https://github.com/dogsheep/github-to-sqlite/compare/2.3...master
dogsheep/dogsheep-photos | 0.4.1 | 2020-05-25T20:11:20Z | 2 | https://github.com/dogsheep/dogsheep-photos/compare/0.4.1...master
dogsheep/swarm-to-sqlite | 0.3.1 | 2020-03-28T02:29:41Z | 1 | https://github.com/dogsheep/swarm-to-sqlite/compare/0.3.1...master
dogsheep/hacker-news-to-sqlite | 0.3.1 | 2020-03-21T22:39:34Z | 1 | https://github.com/dogsheep/hacker-news-to-sqlite/compare/0.3.1...master
simonw/sqlite-utils | 2.11 | 2020-07-08T17:36:07Z | 0 | https://github.com/simonw/sqlite-utils/compare/2.11...master
dogsheep/healthkit-to-sqlite | 0.5 | 2020-03-28T01:50:51Z | 0 | https://github.com/dogsheep/healthkit-to-sqlite/compare/0.5...master
dogsheep/inaturalist-to-sqlite | 0.2 | 2020-03-24T00:35:44Z | 0 | https://github.com/dogsheep/inaturalist-to-sqlite/compare/0.2...master
dogsheep/genome-to-sqlite | 0.1 | 2019-09-19T15:38:10Z | 0 | https://github.com/dogsheep/genome-to-sqlite/compare/0.1...master
dogsheep/pocket-to-sqlite | 0.2 | 2020-03-27T22:23:16Z | 0 | https://github.com/dogsheep/pocket-to-sqlite/compare/0.2...master

https://github-to-sqlite.dogsheep.net/github?sql=with+most_recent_releases+as+%28%0D%0A++with+ranked+as+%28%0D%0A++++select%0D%0A++++++repo%2C%0D%0A++++++tag_name%2C%0D%0A++++++published_at%2C%0D%0A++++++row_number%28%29+OVER+%28%0D%0A++++++++partition+BY+repo%0D%0A++++++++ORDER+BY%0D%0A++++++++++published_at+DESC%0D%0A++++++%29+rank%0D%0A++++FROM%0D%0A++++++releases%0D%0A++%29%0D%0A++select%0D%0A++++*%0D%0A++from%0D%0A++++ranked%0D%0A++where%0D%0A++++rank+%3D+1%0D%0A%29%0D%0Aselect%0D%0A++repos.full_name+as+repo%2C%0D%0A++most_recent_releases.tag_name+as+release%2C%0D%0A++commits.committer_date+as+release_commit_date%2C%0D%0A++%28%0D%0A++++select%0D%0A++++++count%28*%29%0D%0A++++from%0D%0A++++++commits+c2%0D%0A++++where%0D%0A++++++c2.repo+%3D+repos.id%0D%0A++++++and+c2.committer_date+%3E+commits.committer_date%0D%0A++%29+as+commits_since_release%2C%0D%0A++%27https%3A%2F%2Fgithub.com%2F%27+%7C%7C+repos.full_name+%7C%7C+%27%2Fcompare%2F%27+%7C%7C+most_recent_releases.tag_name+%7C%7C+%27...%27+%7C%7C+repos.default_branch+as+view_commits%0D%0Afrom%0D%0A++most_recent_releases%0D%0A++join+repos+on+most_recent_releases.repo+%3D+repos.id%0D%0A++join+tags+on+tags.repo+%3D+repos.id%0D%0A++and+tags.name+%3D+most_recent_releases.tag_name%0D%0A++join+commits+on+tags.sha+%3D+commits.sha%0D%0Aorder+by%0D%0A++commits_since_release+desc","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904,github-to-sqlite tags command for fetching tags,
https://github.com/dogsheep/github-to-sqlite/issues/30#issuecomment-616879753,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/30,616879753,MDEyOklzc3VlQ29tbWVudDYxNjg3OTc1Mw==,9599,simonw,2020-04-21T00:29:29Z,2020-04-21T00:29:29Z,MEMBER,`assignee` looks like it's the incorrect type too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",603618244,Issues milestone column is the wrong type,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-625947133,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,625947133,MDEyOklzc3VlQ29tbWVudDYyNTk0NzEzMw==,9599,simonw,2020-05-08T18:13:06Z,2020-05-08T18:13:06Z,MEMBER,`datasette-media` will be able to handle this once I implement https://github.com/simonw/datasette-media/issues/3,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/dogsheep-beta/issues/9#issuecomment-686154486,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/9,686154486,MDEyOklzc3VlQ29tbWVudDY4NjE1NDQ4Ng==,9599,simonw,2020-09-03T00:18:54Z,2020-09-03T00:18:54Z,MEMBER,"`display_sql` could be optional. If it's not defined, a `row` object is passed to the template which is the row that's stored in `search_index`. If `display_sql` IS defined then it's executed and the result is made available as a `display` object in addition to the `row` object.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",691521965,Mechanism for defining custom display of results,
https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188228964,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188228964,IC_kwDODFdgUs5G0u9k,9599,simonw,2022-07-18T19:45:30Z,2022-07-18T19:47:35Z,MEMBER,"`pycmarkgfm` doesn't implement the Markdown plugin extension I was using.

I'm going to drop the GFM rendering from the demo, and just treat it as regular markdown.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063,500 error in github-to-sqlite demo,
https://github.com/dogsheep/twitter-to-sqlite/issues/19#issuecomment-541248629,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/19,541248629,MDEyOklzc3VlQ29tbWVudDU0MTI0ODYyOQ==,9599,simonw,2019-10-11T22:48:56Z,2019-10-11T22:48:56Z,MEMBER,"`since_id` documented here: https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-home_timeline

> Returns results with an ID greater than (that is, more recent than) the specified ID. There are limits to the number of Tweets which can be accessed through the API. If the limit of Tweets has occured since the since_id, the since_id will be forced to the oldest ID available.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506087267,since_id support for home-timeline,
https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-594154644,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8,594154644,MDEyOklzc3VlQ29tbWVudDU5NDE1NDY0NA==,9599,simonw,2020-03-03T20:33:57Z,2020-03-03T20:33:57Z,MEMBER,`sqlite-utils` supports proper upserts now so this problem should be easy to fix.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",516763727,"stargazers command, refs #4",
https://github.com/dogsheep/github-to-sqlite/issues/12#issuecomment-594155249,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12,594155249,MDEyOklzc3VlQ29tbWVudDU5NDE1NTI0OQ==,9599,simonw,2020-03-03T20:35:17Z,2020-03-03T20:35:17Z,MEMBER,"`swarm-to-sqlite` has an example of adding views here: https://github.com/dogsheep/swarm-to-sqlite/blob/f2c89dd613fb8a7f14e5267ccc2145463b996190/swarm_to_sqlite/utils.py#L141

I think that approach can be approved by first checking if the view exists, then dropping it, then recreating it. Could even try to see if the view exists and matches what we were going to set it to and do nothing if that is the case.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520756546,Add this view for seeing new releases,
https://github.com/dogsheep/github-to-sqlite/issues/3#issuecomment-531516956,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/3,531516956,MDEyOklzc3VlQ29tbWVudDUzMTUxNjk1Ng==,9599,simonw,2019-09-14T21:56:31Z,2019-09-14T21:56:31Z,MEMBER,"https://api.github.com/users/simonw/repos

It would be useful to be able to fetch stargazers, forks etc as well. Not sure if that should be a separate command or a `--stargazers` option to this command.

Probably a separate command since `issues` is a separate command already.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",493670426,Command to fetch all repos belonging to a user or organization,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615933273,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615933273,MDEyOklzc3VlQ29tbWVudDYxNTkzMzI3Mw==,9599,simonw,2020-04-18T19:37:33Z,2020-04-18T19:37:33Z,MEMBER,"https://console.aws.amazon.com/s3/bucket/create?region=us-west-1

![S3_Management_Console](https://user-images.githubusercontent.com/9599/79669552-33e2a380-8171-11ea-9ab5-5785d34f652a.png)

I created it with no public read-write access. I plan to use signed URLs via a transforming proxy to access images for display on the web.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,Upload all my photos to a secure S3 bucket,
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962418,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962418,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjQxOA==,9599,simonw,2020-07-06T00:41:38Z,2020-07-06T00:41:38Z,MEMBER,"https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/logs?project=datasette-222320 has some clues.

<img width=""818"" alt=""github-to-sqlite_–_Service_details_–_Cloud_Run_–_datasette_–_Google_Cloud_Platform"" src=""https://user-images.githubusercontent.com/9599/86545887-92086280-bee6-11ea-8787-b3b85a7bb4b9.png"">
<img width=""1451"" alt=""github-to-sqlite_–_Service_details_–_Cloud_Run_–_datasette_–_Google_Cloud_Platform"" src=""https://user-images.githubusercontent.com/9599/86545917-c11ed400-bee6-11ea-9cf8-957911ca9f6a.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727,Demo is failing to deploy,
https://github.com/dogsheep/swarm-to-sqlite/issues/3#issuecomment-542875885,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3,542875885,MDEyOklzc3VlQ29tbWVudDU0Mjg3NTg4NQ==,9599,simonw,2019-10-16T20:23:08Z,2019-10-16T20:23:08Z,MEMBER,"https://developer.foursquare.com/docs/api/users/checkins documents `afterTimestamp`:
> Retrieve the first results to follow these seconds since epoch. This should be useful for paging forward in time, or when polling for changes. To avoid missing results when polling, we recommend subtracting several seconds from the last poll time and then de-duplicating.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",487600595,Option to fetch only checkins more recent than the current max checkin,
https://github.com/dogsheep/twitter-to-sqlite/issues/3#issuecomment-541493242,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/3,541493242,MDEyOklzc3VlQ29tbWVudDU0MTQ5MzI0Mg==,9599,simonw,2019-10-14T03:35:36Z,2019-10-14T03:35:36Z,MEMBER,"https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",488833975,Command for running a search and saving tweets for that search,
https://github.com/dogsheep/twitter-to-sqlite/issues/18#issuecomment-541118773,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/18,541118773,MDEyOklzc3VlQ29tbWVudDU0MTExODc3Mw==,9599,simonw,2019-10-11T15:48:31Z,2019-10-11T15:48:31Z,MEMBER,https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-home_timeline,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",505928530,Command to import home-timeline,
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-902328369,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57,902328369,IC_kwDODEm0Qs41yHAx,9599,simonw,2021-08-19T23:56:26Z,2021-08-19T23:56:26Z,MEMBER,"https://developer.twitter.com/en/docs/twitter-api/v1/tweets/timelines/api-reference/get-statuses-user_timeline says the API has been replaced by the new v2 one, but it should still work - and the `since_id` parameter is still documented on that page.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907645813,"Error: Use either --since or --since_id, not both",
https://github.com/dogsheep/dogsheep-photos/issues/25#issuecomment-631253248,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25,631253248,MDEyOklzc3VlQ29tbWVudDYzMTI1MzI0OA==,9599,simonw,2020-05-20T05:54:18Z,2020-05-20T05:54:18Z,MEMBER,https://dogsheep-photos.dogsheep.net/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621332242,Create a public demo,
https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962669,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41,653962669,MDEyOklzc3VlQ29tbWVudDY1Mzk2MjY2OQ==,9599,simonw,2020-07-06T00:42:57Z,2020-07-06T00:42:57Z,MEMBER,"https://github-to-sqlite.dogsheep.net/-/plugins

```json
[
    {
        ""name"": ""datasette-json-html"",
        ""static"": false,
        ""templates"": false,
        ""version"": ""0.6"",
        ""hooks"": [
            ""prepare_connection"",
            ""render_cell""
        ]
    },
    {
        ""name"": ""datasette-render-markdown"",
        ""static"": false,
        ""templates"": false,
        ""version"": ""1.1.2"",
        ""hooks"": [
            ""extra_template_vars"",
            ""render_cell""
        ]
    },
    {
        ""name"": ""datasette-pretty-json"",
        ""static"": false,
        ""templates"": false,
        ""version"": ""0.2"",
        ""hooks"": [
            ""render_cell""
        ]
    },
    {
        ""name"": ""datasette-search-all"",
        ""static"": false,
        ""templates"": true,
        ""version"": ""0.2.1"",
        ""hooks"": [
            ""asgi_wrapper"",
            ""extra_template_vars""
        ]
    },
    {
        ""name"": ""datasette-vega"",
        ""static"": true,
        ""templates"": false,
        ""version"": ""0.6.2"",
        ""hooks"": [
            ""extra_css_urls"",
            ""extra_js_urls""
        ]
    }
]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",651159727,Demo is failing to deploy,
https://github.com/dogsheep/github-to-sqlite/issues/31#issuecomment-617491607,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31,617491607,MDEyOklzc3VlQ29tbWVudDYxNzQ5MTYwNw==,9599,simonw,2020-04-22T01:20:19Z,2020-04-22T01:20:19Z,MEMBER,"https://github-to-sqlite.dogsheep.net/github/milestones now link to repo:

<img width=""1095"" alt=""Screen Shot 2020-04-21 at 6 19 03 PM"" src=""https://user-images.githubusercontent.com/9599/79930145-956a7280-83fc-11ea-9b47-b0c5674a0d4f.png"">

And so do issues: https://github-to-sqlite.dogsheep.net/github/issues

<img width=""1095"" alt=""Screen Shot 2020-04-21 at 6 19 53 PM"" src=""https://user-images.githubusercontent.com/9599/79930200-bd59d600-83fc-11ea-8fb8-b40772140409.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",603624862,Issue and milestone should have foreign key to repo,
https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554811,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45,660554811,MDEyOklzc3VlQ29tbWVudDY2MDU1NDgxMQ==,9599,simonw,2020-07-18T23:03:13Z,2020-07-18T23:03:13Z,MEMBER,https://github-to-sqlite.dogsheep.net/github/tags now shows a `repo` column instead of a `repo_id` column.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660429601,Fix the demo - it breaks because of the tags table change,
https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660548780,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43,660548780,MDEyOklzc3VlQ29tbWVudDY2MDU0ODc4MA==,9599,simonw,2020-07-18T22:02:37Z,2020-07-18T23:05:56Z,MEMBER,https://github-to-sqlite.dogsheep.net/github/tags?_facet=repo,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",660355904,github-to-sqlite tags command for fetching tags,
https://github.com/dogsheep/github-to-sqlite/issues/32#issuecomment-617490914,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/32,617490914,MDEyOklzc3VlQ29tbWVudDYxNzQ5MDkxNA==,9599,simonw,2020-04-22T01:17:44Z,2020-04-22T01:17:44Z,MEMBER,"https://github-to-sqlite.dogsheep.net/github?sql=select+html_url%2C+id%2C+issue+from+issue_comments+order+by+updated_at+desc+limit+101 now shows issues.

And https://github-to-sqlite.dogsheep.net/github/issue_comments links to them:

<img width=""395"" alt=""Screen Shot 2020-04-21 at 6 17 33 PM"" src=""https://user-images.githubusercontent.com/9599/79930066-6227e380-83fc-11ea-8416-8b9d55660cb5.png"">
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",604222295,Issue comments don't appear to populate issues foreign key,
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626394989,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626394989,MDEyOklzc3VlQ29tbWVudDYyNjM5NDk4OQ==,9599,simonw,2020-05-10T21:50:36Z,2020-05-10T21:50:36Z,MEMBER,https://github.com/Marketcircle/bpylist/pull/2 looks relevant here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,bpylist.archiver.CircularReference: archive has a cycle with uid(13),
https://github.com/dogsheep/github-to-sqlite/issues/74#issuecomment-1188225625,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74,1188225625,IC_kwDODFdgUs5G0uJZ,9599,simonw,2022-07-18T19:41:52Z,2022-07-18T19:41:52Z,MEMBER,"https://github.com/Zopieux/py-gfm says that library is no longer maintained, and suggests https://github.com/Zopieux/pycmarkgfm as an alternative.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1308461063,500 error in github-to-sqlite demo,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633704127,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,633704127,MDEyOklzc3VlQ29tbWVudDYzMzcwNDEyNw==,9599,simonw,2020-05-25T20:14:22Z,2020-05-25T20:14:22Z,MEMBER,https://github.com/dogsheep/dogsheep-photos/blob/0.4.1/README.md#serving-photos-locally-with-datasette-media,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633643921,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,633643921,MDEyOklzc3VlQ29tbWVudDYzMzY0MzkyMQ==,9599,simonw,2020-05-25T16:29:44Z,2020-05-25T16:29:44Z,MEMBER,https://github.com/dogsheep/dogsheep-photos/blob/dc43fa8653cb9c7238a36f52239b91d1ec916d5c/README.md#serving-photos-locally-with-datasette-media,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/evernote-to-sqlite/issues/13#issuecomment-905206234,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13,905206234,IC_kwDOEhK-wc419Fna,9599,simonw,2021-08-25T05:58:42Z,2021-08-25T05:58:42Z,MEMBER,"https://github.com/dogsheep/evernote-to-sqlite/blob/36a466f142e5bad52719851c2fbda0c05cd35b99/evernote_to_sqlite/utils.py#L34-L42

Not sure why I was round-tripping the `content_xml` like that - I will try not doing that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",978743426,xml.etree.ElementTree.ParseError: not well-formed (invalid token),
https://github.com/dogsheep/github-to-sqlite/issues/23#issuecomment-602928533,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/23,602928533,MDEyOklzc3VlQ29tbWVudDYwMjkyODUzMw==,9599,simonw,2020-03-24T00:15:49Z,2020-03-24T00:15:49Z,MEMBER,https://github.com/dogsheep/github-to-sqlite/releases/tag/1.0,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",586595839,Release 1.0,
https://github.com/dogsheep/pocket-to-sqlite/issues/12#issuecomment-1627564127,https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12,1627564127,IC_kwDODLZ_YM5hAqhf,9599,simonw,2023-07-09T01:19:42Z,2023-07-09T01:19:42Z,MEMBER,https://github.com/dogsheep/pocket-to-sqlite/tree/0.2.3 and https://pypi.org/project/pocket-to-sqlite/0.2.3/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1795187493,Switch to pyproject.toml,
https://github.com/dogsheep/twitter-to-sqlite/issues/10#issuecomment-543270714,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10,543270714,MDEyOklzc3VlQ29tbWVudDU0MzI3MDcxNA==,9599,simonw,2019-10-17T17:05:16Z,2019-10-17T17:05:16Z,MEMBER,"https://github.com/pallets/click/blob/716a5be90f56ce6cd506bb53d5739d09374b1636/click/_termui_impl.py#L93 is how Click does this:
```
        self.is_hidden = not isatty(self.file)
```
Where `isatty` is a Click utility function: `from ._compat import isatty`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",492297930,Rethink progress bars for various commands,
https://github.com/dogsheep/github-to-sqlite/issues/27#issuecomment-615518606,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27,615518606,MDEyOklzc3VlQ29tbWVudDYxNTUxODYwNg==,9599,simonw,2020-04-18T00:14:32Z,2020-04-18T00:14:32Z,MEMBER,https://github.com/simonw/sqlite-utils/issues/100 is done and released in sqlite-utils 2.7.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",601330277,Repos have a big blob of JSON in the organization column,
https://github.com/dogsheep/github-to-sqlite/issues/18#issuecomment-601980957,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18,601980957,MDEyOklzc3VlQ29tbWVudDYwMTk4MDk1Nw==,9599,simonw,2020-03-21T02:30:03Z,2020-03-21T02:30:03Z,MEMBER,"https://github.community/t5/GitHub-API-Development-and/Request-for-commits-quot-author-quot-null-and-quot-committer/m-p/35842/highlight/true#M3372
> Commits aren't always associated with a GitHub user. For example, perhaps a friend of mine and I were working on a project together. I have a GitHub account and my friend doesn't. If we both add commits to the repository using our own email addresses and names and then I push the repository to GitHub, my commits will be associated with my GitHub user account but my friends' commits will show up with `author` and `committer` as `null`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",585411547,Commits in GitHub API can have null author,
https://github.com/dogsheep/hacker-news-to-sqlite/issues/2#issuecomment-886135922,https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/2,886135922,IC_kwDODtX3eM400Vxy,9599,simonw,2021-07-25T02:06:20Z,2021-07-25T02:06:20Z,MEMBER,"https://hn.algolia.com/api/v1/search_by_date?query=simonwillison.net&restrictSearchableAttributes=url looks like it does what I want.

https://hn.algolia.com/api/v1/search_by_date?query=simonwillison.net&restrictSearchableAttributes=url&hitsPerPage=1000 - returns 1000 at once.

Otherwise you have to paginate using `&page=2` etc - up to `nbPages` pages.

https://www.algolia.com/doc/api-reference/api-parameters/hitsPerPage/ says 1000 is the maximum.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",952179830,Command for fetching Hacker News threads from the search API,
https://github.com/dogsheep/dogsheep-photos/issues/7#issuecomment-615993178,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/7,615993178,MDEyOklzc3VlQ29tbWVudDYxNTk5MzE3OA==,9599,simonw,2020-04-19T00:37:08Z,2020-04-19T00:37:08Z,MEMBER,https://pypi.org/project/ImageHash/ Is one option.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602585497,Integrate image content hashing,
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631229485,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631229485,MDEyOklzc3VlQ29tbWVudDYzMTIyOTQ4NQ==,9599,simonw,2020-05-20T04:31:02Z,2020-05-20T04:31:02Z,MEMBER,https://pypi.org/project/dogsheep-photos/ is live.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/dogsheep-photos/issues/26#issuecomment-631229409,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26,631229409,MDEyOklzc3VlQ29tbWVudDYzMTIyOTQwOQ==,9599,simonw,2020-05-20T04:30:40Z,2020-05-20T04:30:40Z,MEMBER,https://pypi.org/project/photos-to-sqlite/ now links to dogsheep-photos.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",621444763,Rename project to dogsheep-photos,
https://github.com/dogsheep/dogsheep-photos/issues/17#issuecomment-624278090,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17,624278090,MDEyOklzc3VlQ29tbWVudDYyNDI3ODA5MA==,9599,simonw,2020-05-05T20:06:01Z,2020-05-05T20:06:01Z,MEMBER,https://www.python.org/dev/peps/pep-0508/#environment-markers I think I want `sys_platform` of `darwin`.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612860531,Only install osxphotos if running on macOS,
https://github.com/dogsheep/swarm-to-sqlite/issues/4#issuecomment-526853542,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/4,526853542,MDEyOklzc3VlQ29tbWVudDUyNjg1MzU0Mg==,9599,simonw,2019-08-31T18:06:32Z,2019-08-31T18:06:32Z,MEMBER,"https://your-foursquare-oauth-token.glitch.me/

Source code: https://glitch.com/~your-foursquare-oauth-token","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",487601121,Online tool for getting a Foursquare OAuth token,