![](\"/-/media/photo/{{)
\r\n{% endfor %}\r\n![](\"https://photos.simonwillison.net/i/{{)
\r\n Taken in {{ display.place_city }} on {{ display.date }}
\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691521965, "label": "Mechanism for defining custom display of results"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685970384", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7", "id": 685970384, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTk3MDM4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T20:11:41Z", "updated_at": "2020-09-02T20:11:59Z", "author_association": "MEMBER", "body": "Default categories:\r\n\r\n- 1 = created\r\n- 2 = saved", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691265198, "label": "Mechanism for differentiating between \"by me\" and \"liked by me\""}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685966707", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7", "id": 685966707, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTk2NjcwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T20:04:08Z", "updated_at": "2020-09-02T20:04:08Z", "author_association": "MEMBER", "body": "I'll make `category` a foreign key to a `categories` table so Datasette can automatically show the `name` column.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691265198, "label": "Mechanism for differentiating between \"by me\" and \"liked by me\""}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685966361", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7", "id": 685966361, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTk2NjM2MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T20:03:29Z", "updated_at": "2020-09-02T20:03:41Z", "author_association": "MEMBER", "body": "I'm going to implement the first version of this as an indexed integer `category` column which has 1 for \"about me\" and 2 for \"liked by me\" - and space for other category numerals in the future, albeit a row can only belong to one category.\r\n\r\nI'll think about a full tagging system separately.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691265198, "label": "Mechanism for differentiating between \"by me\" and \"liked by me\""}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685965516", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7", "id": 685965516, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTk2NTUxNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T20:01:54Z", "updated_at": "2020-09-02T20:01:54Z", "author_association": "MEMBER", "body": "Relevant post: https://sqlite.org/forum/forumpost/9f06fedaa5 - drh says:\r\n\r\n> Indexes are one-to-one. There is one entry in the index for each row in the table.\r\n>\r\n> You are asking for an index that is many-to-one - multiple index entries for each table row.\r\n>\r\n> A Full-Text Index is basically a many-to-one index. So if all of your array entries really are words, you could probably get this to work using a Full-Text Index.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691265198, "label": "Mechanism for differentiating between \"by me\" and \"liked by me\""}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685962280", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7", "id": 685962280, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTk2MjI4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T19:55:26Z", "updated_at": "2020-09-02T19:59:58Z", "author_association": "MEMBER", "body": "Relevant: https://charlesleifer.com/blog/a-tour-of-tagging-schemas-many-to-many-bitmaps-and-more/\r\n\r\nSQLite supports bitwise operators Binary AND (&) and Binary OR (|) - I could try those. Not sure how they interact with indexes though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691265198, "label": "Mechanism for differentiating between \"by me\" and \"liked by me\""}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/3#issuecomment-685961809", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3", "id": 685961809, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTk2MTgwOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T19:54:24Z", "updated_at": "2020-09-02T19:54:24Z", "author_association": "MEMBER", "body": "This should implement search highlighting too, as seen on https://til.simonwillison.net/til/search?q=cloud\r\n\r\n![\"TIL_search__cloud\"](\"https://user-images.githubusercontent.com/9599/92029959-32c6a300-ed1b-11ea-8b5e-971950980c38.png\")
\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 689810340, "label": "Datasette plugin to provide custom page for running faceted, ranked searches"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/8#issuecomment-685960072", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/8", "id": 685960072, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTk2MDA3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T19:50:47Z", "updated_at": "2020-09-02T19:50:47Z", "author_association": "MEMBER", "body": "This doesn't actually help, because the Datasette table view page doesn't then support adding the `where search_index_fts match :query` bit.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691369691, "label": "Create a view for running faceted searches"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/7#issuecomment-685895540", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/7", "id": 685895540, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTg5NTU0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-02T17:46:44Z", "updated_at": "2020-09-02T17:46:44Z", "author_association": "MEMBER", "body": "Some opet questions about this:\r\n\r\n- Should I restrict to two exclusive categories here, or should I have a generic category mechanism that can be expanded to more than two?\r\n- Should an item be able to exist in more than one category? Do I want to be able to mark an indexed item as both by-me and liked-by-me for example? This question is more interesting if the number of categories is greater than two.\r\n- How should this be modeled? Single column, multiple boolean columns, JSON array, m2m against separate table?\r\n- What's the best way to make this performant", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 691265198, "label": "Mechanism for differentiating between \"by me\" and \"liked by me\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/2#issuecomment-685121074", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/2", "id": 685121074, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTEyMTA3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-01T20:42:00Z", "updated_at": "2020-09-01T20:42:00Z", "author_association": "MEMBER", "body": "Documentation at the bottom of the Usage section here: https://github.com/dogsheep/dogsheep-beta/blob/0.2/README.md#usage", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 689809225, "label": "Apply porter stemming"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/2#issuecomment-685115519", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/2", "id": 685115519, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NTExNTUxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-01T20:31:57Z", "updated_at": "2020-09-01T20:31:57Z", "author_association": "MEMBER", "body": "Actually this doesn't work: you can't turn on stemming for specific tables, because all of the content goes into a single `search_index` table which is configured the same way.\r\n\r\nSo stemming needs to be a global option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 689809225, "label": "Apply porter stemming"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/pocket-to-sqlite/issues/5#issuecomment-684425714", "issue_url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/5", "id": 684425714, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NDQyNTcxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-01T06:18:32Z", "updated_at": "2020-09-01T06:18:32Z", "author_association": "MEMBER", "body": "Good suggestion, I'll setup a demo somewhere.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 629473827, "label": "Set up a demo"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/pocket-to-sqlite/issues/3#issuecomment-684424396", "issue_url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3", "id": 684424396, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NDQyNDM5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-01T06:17:45Z", "updated_at": "2020-09-01T06:17:45Z", "author_association": "MEMBER", "body": "It looks like I could ignore the `image` column and synthesize a unique key from the data in the `images` column using `$item_id/$image_id`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 503243784, "label": "Extract images into separate tables"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/4#issuecomment-684395444", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/4", "id": 684395444, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NDM5NTQ0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-01T06:00:03Z", "updated_at": "2020-09-01T06:00:03Z", "author_association": "MEMBER", "body": "I ran `sqlite-utils optimize beta.db` against my test DB and the size reduced from 183M to 176M - and a 450ms search ran in 359ms. So not a huge improvement but still worthwhile.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 689839399, "label": "Optimize the FTS table"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-beta/issues/3#issuecomment-684250044", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/3", "id": 684250044, "node_id": "MDEyOklzc3VlQ29tbWVudDY4NDI1MDA0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-01T05:01:09Z", "updated_at": "2020-09-01T05:01:23Z", "author_association": "MEMBER", "body": "Maybe this starts out as a custom templated canned query.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 689810340, "label": "Datasette plugin to provide custom page for running faceted, ranked searches"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/47#issuecomment-675523053", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/47", "id": 675523053, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTUyMzA1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T14:45:53Z", "updated_at": "2020-08-18T14:45:53Z", "author_association": "MEMBER", "body": "```\r\n% github-to-sqlite emojis emojis.db --fetch\r\n [########----------------------------] 397/1682 23% 00:03:43\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681086659, "label": "emojis command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/39#issuecomment-675509550", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/39", "id": 675509550, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTUwOTU1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T14:23:56Z", "updated_at": "2020-08-18T14:23:56Z", "author_association": "MEMBER", "body": "I think this is fixed: https://github-to-sqlite.dogsheep.net/github/issues?_facet=repo", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613777056, "label": "issues foreign key to repo isn't working"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/46#issuecomment-675259273", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/46", "id": 675259273, "node_id": "MDEyOklzc3VlQ29tbWVudDY3NTI1OTI3Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-18T05:28:32Z", "updated_at": "2020-08-18T05:28:32Z", "author_association": "MEMBER", "body": "Oh that's interesting - i didn't realize \"reviews\" were a separate concept. I'd definitely accept a pull request adding those!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 664485022, "label": "Feature: pull request reviews and comments"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/swarm-to-sqlite/issues/8#issuecomment-669241341", "issue_url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/8", "id": 669241341, "node_id": "MDEyOklzc3VlQ29tbWVudDY2OTI0MTM0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-08-05T14:55:14Z", "updated_at": "2020-08-05T14:55:14Z", "author_association": "MEMBER", "body": "Looks like there's a column that's not consistently there, so the table got created without it.\r\n\r\nEasiest fix is to add `alter=True` to this line:\r\n\r\nhttps://github.com/dogsheep/swarm-to-sqlite/blob/f4a82633da927cde672c9d9af92930bfca2e3ddf/swarm_to_sqlite/utils.py#L94\r\n\r\nThat will cause `sqlite-utils` to notice if there's a missing column and add it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 648245071, "label": "Error thrown: table photos has no column named hasSticker"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-663143160", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48", "id": 663143160, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MzE0MzE2MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-23T17:46:07Z", "updated_at": "2020-07-23T17:46:07Z", "author_association": "MEMBER", "body": "Frustratingly, these links don't work on PyPI: https://pypi.org/project/twitter-to-sqlite/\r\n\r\nThere's an issue about that here: https://github.com/pypa/readme_renderer/issues/169", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 663976976, "label": "Add a table of contents to the README"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-662630868", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48", "id": 662630868, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MjYzMDg2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-22T19:03:02Z", "updated_at": "2020-07-22T19:03:02Z", "author_association": "MEMBER", "body": "Done!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 663976976, "label": "Add a table of contents to the README"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/48#issuecomment-662626901", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/48", "id": 662626901, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MjYyNjkwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-22T18:54:53Z", "updated_at": "2020-07-22T18:54:53Z", "author_association": "MEMBER", "body": "I'm going to use a GitHub Action to run `npx markdown-toc README.md -i`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 663976976, "label": "Add a table of contents to the README"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660548780", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43", "id": 660548780, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU0ODc4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T22:02:37Z", "updated_at": "2020-07-18T23:05:56Z", "author_association": "MEMBER", "body": "https://github-to-sqlite.dogsheep.net/github/tags?_facet=repo", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660355904, "label": "github-to-sqlite tags command for fetching tags"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660551397", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43", "id": 660551397, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU1MTM5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T22:27:32Z", "updated_at": "2020-07-18T23:05:45Z", "author_association": "MEMBER", "body": "```sql\r\nwith most_recent_releases as (\r\n with ranked as (\r\n select\r\n repo,\r\n tag_name,\r\n published_at,\r\n row_number() OVER (\r\n partition BY repo\r\n ORDER BY\r\n published_at DESC\r\n ) rank\r\n FROM\r\n releases\r\n )\r\n select\r\n *\r\n from\r\n ranked\r\n where\r\n rank = 1\r\n)\r\nselect\r\n repos.full_name as repo,\r\n most_recent_releases.tag_name as release,\r\n commits.committer_date as release_commit_date,\r\n (\r\n select\r\n count(*)\r\n from\r\n commits c2\r\n where\r\n c2.repo = repos.id\r\n and c2.committer_date > commits.committer_date\r\n ) as commits_since_release,\r\n 'https://github.com/' || repos.full_name || '/compare/' || most_recent_releases.tag_name || '...' || repos.default_branch as view_commits\r\nfrom\r\n most_recent_releases\r\n join repos on most_recent_releases.repo = repos.id\r\n join tags on tags.repo = repos.id\r\n and tags.name = most_recent_releases.tag_name\r\n join commits on tags.sha = commits.sha\r\norder by\r\n commits_since_release desc\r\n```\r\n\r\n\r\nrepo | release | release_commit_date | commits_since_release | view_commits\r\n-- | -- | -- | -- | --\r\nsimonw/datasette | 0.45 | 2020-07-01T21:43:07Z | 9 | https://github.com/simonw/datasette/compare/0.45...master\r\ndogsheep/twitter-to-sqlite | 0.21.1 | 2020-04-30T18:20:43Z | 2 | https://github.com/dogsheep/twitter-to-sqlite/compare/0.21.1...master\r\ndogsheep/github-to-sqlite | 2.3 | 2020-07-09T23:26:34Z | 2 | https://github.com/dogsheep/github-to-sqlite/compare/2.3...master\r\ndogsheep/dogsheep-photos | 0.4.1 | 2020-05-25T20:11:20Z | 2 | https://github.com/dogsheep/dogsheep-photos/compare/0.4.1...master\r\ndogsheep/swarm-to-sqlite | 0.3.1 | 2020-03-28T02:29:41Z | 1 | https://github.com/dogsheep/swarm-to-sqlite/compare/0.3.1...master\r\ndogsheep/hacker-news-to-sqlite | 0.3.1 | 2020-03-21T22:39:34Z | 1 | https://github.com/dogsheep/hacker-news-to-sqlite/compare/0.3.1...master\r\nsimonw/sqlite-utils | 2.11 | 2020-07-08T17:36:07Z | 0 | https://github.com/simonw/sqlite-utils/compare/2.11...master\r\ndogsheep/healthkit-to-sqlite | 0.5 | 2020-03-28T01:50:51Z | 0 | https://github.com/dogsheep/healthkit-to-sqlite/compare/0.5...master\r\ndogsheep/inaturalist-to-sqlite | 0.2 | 2020-03-24T00:35:44Z | 0 | https://github.com/dogsheep/inaturalist-to-sqlite/compare/0.2...master\r\ndogsheep/genome-to-sqlite | 0.1 | 2019-09-19T15:38:10Z | 0 | https://github.com/dogsheep/genome-to-sqlite/compare/0.1...master\r\ndogsheep/pocket-to-sqlite | 0.2 | 2020-03-27T22:23:16Z | 0 | https://github.com/dogsheep/pocket-to-sqlite/compare/0.2...master\r\n\r\nhttps://github-to-sqlite.dogsheep.net/github?sql=with+most_recent_releases+as+%28%0D%0A++with+ranked+as+%28%0D%0A++++select%0D%0A++++++repo%2C%0D%0A++++++tag_name%2C%0D%0A++++++published_at%2C%0D%0A++++++row_number%28%29+OVER+%28%0D%0A++++++++partition+BY+repo%0D%0A++++++++ORDER+BY%0D%0A++++++++++published_at+DESC%0D%0A++++++%29+rank%0D%0A++++FROM%0D%0A++++++releases%0D%0A++%29%0D%0A++select%0D%0A++++*%0D%0A++from%0D%0A++++ranked%0D%0A++where%0D%0A++++rank+%3D+1%0D%0A%29%0D%0Aselect%0D%0A++repos.full_name+as+repo%2C%0D%0A++most_recent_releases.tag_name+as+release%2C%0D%0A++commits.committer_date+as+release_commit_date%2C%0D%0A++%28%0D%0A++++select%0D%0A++++++count%28*%29%0D%0A++++from%0D%0A++++++commits+c2%0D%0A++++where%0D%0A++++++c2.repo+%3D+repos.id%0D%0A++++++and+c2.committer_date+%3E+commits.committer_date%0D%0A++%29+as+commits_since_release%2C%0D%0A++%27https%3A%2F%2Fgithub.com%2F%27+%7C%7C+repos.full_name+%7C%7C+%27%2Fcompare%2F%27+%7C%7C+most_recent_releases.tag_name+%7C%7C+%27...%27+%7C%7C+repos.default_branch+as+view_commits%0D%0Afrom%0D%0A++most_recent_releases%0D%0A++join+repos+on+most_recent_releases.repo+%3D+repos.id%0D%0A++join+tags+on+tags.repo+%3D+repos.id%0D%0A++and+tags.name+%3D+most_recent_releases.tag_name%0D%0A++join+commits+on+tags.sha+%3D+commits.sha%0D%0Aorder+by%0D%0A++commits_since_release+desc", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660355904, "label": "github-to-sqlite tags command for fetching tags"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554811", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45", "id": 660554811, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU1NDgxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T23:03:13Z", "updated_at": "2020-07-18T23:03:13Z", "author_association": "MEMBER", "body": "https://github-to-sqlite.dogsheep.net/github/tags now shows a `repo` column instead of a `repo_id` column.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660429601, "label": "Fix the demo - it breaks because of the tags table change"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554299", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45", "id": 660554299, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU1NDI5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T22:58:24Z", "updated_at": "2020-07-18T23:02:52Z", "author_association": "MEMBER", "body": "Deploying the fixed version like this:\r\n```\r\n$ gcloud config set run/region us-central1\r\n$ gcloud config set project datasette-222320\r\n$ datasette publish cloudrun /tmp/github.db \\\r\n -m demo-metadata.json \\\r\n --service github-to-sqlite \\\r\n --install=py-gfm \\\r\n --install='datasette-search-all>=0.3' \\\r\n --install='datasette-render-markdown>=1.1.2' \\\r\n --install=datasette-pretty-json \\\r\n --install=datasette-json-html \\\r\n --install=datasette-vega\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660429601, "label": "Fix the demo - it breaks because of the tags table change"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660554162", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45", "id": 660554162, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU1NDE2Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T22:56:58Z", "updated_at": "2020-07-18T22:56:58Z", "author_association": "MEMBER", "body": "Manually fixing the database:\r\n```\r\n$ wget 'https://github-to-sqlite.dogsheep.net/github.db'\r\n--2020-07-18 15:52:33-- https://github-to-sqlite.dogsheep.net/github.db\r\nResolving github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)... 172.217.5.115\r\nConnecting to github-to-sqlite.dogsheep.net (github-to-sqlite.dogsheep.net)|172.217.5.115|:443... connected.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: 14626816 (14M) [application/octet-stream]\r\nSaving to: \u2018github.db\u2019\r\n\r\ngithub.db 100%[============================================================================>] 13.95M 1.22MB/s in 18s \r\n\r\n2020-07-18 15:52:53 (773 KB/s) - \u2018github.db\u2019 saved [14626816/14626816]\r\n\r\n$ sqlite3 github.db \r\nSQLite version 3.28.0 2019-04-15 14:49:49\r\nEnter \".help\" for usage hints.\r\nsqlite> drop table tags;\r\nsqlite> ^D\r\n$ github-to-sqlite tags github.db simonw/datasette simonw/sqlite-utils dogsheep/healthkit-to-sqlite dogsheep/swarm-to-sqlite dogsheep/twitter-to-sqlite dogsheep/inaturalist-to-sqlite dogsheep/google-takeout-to-sqlite dogsheep/github-to-sqlite dogsheep/genome-to-sqlite dogsheep/pocket-to-sqlite dogsheep/hacker-news-to-sqlite dogsheep/dogsheep-photos \r\n$ sqlite-utils tables github.db --counts\r\n[{\"table\": \"users\", \"count\": 4048},\r\n {\"table\": \"repos\", \"count\": 210},\r\n ...\r\n {\"table\": \"stars\", \"count\": 4140},\r\n {\"table\": \"tags\", \"count\": 188}]\r\n$ sqlite-utils rows github.db tags \r\n[{\"repo\": 107914493, \"name\": \"0.45\", \"sha\": \"f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb\"},\r\n {\"repo\": 107914493, \"name\": \"0.45a5\", \"sha\": \"676bb64c877d73f8ff496cef4632f5a8a5a9283c\"},\r\n ...\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660429601, "label": "Fix the demo - it breaks because of the tags table change"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660553711", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45", "id": 660553711, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU1MzcxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T22:52:16Z", "updated_at": "2020-07-18T22:52:16Z", "author_association": "MEMBER", "body": "I think the best fix is to download the `github.db` database, manually fix it and then manually deploy it to Cloud Run from my laptop.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660429601, "label": "Fix the demo - it breaks because of the tags table change"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/45#issuecomment-660553646", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45", "id": 660553646, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU1MzY0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T22:51:41Z", "updated_at": "2020-07-18T22:51:41Z", "author_association": "MEMBER", "body": "I could fix this by putting `REFRESH_DB` in a commit message:\r\n\r\nhttps://github.com/dogsheep/github-to-sqlite/blob/4ae4aa6f172344b19ff3513707195ee6d2654bd4/.github/workflows/deploy-demo.yml#L41-L46\r\n\r\nBut... doing so would lose the data I've collected in https://github-to-sqlite.dogsheep.net/github/dependents?_sort_desc=first_seen_utc concerning the first time each dependent repo was spotted.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660429601, "label": "Fix the demo - it breaks because of the tags table change"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660547502", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43", "id": 660547502, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDU0NzUwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T21:50:37Z", "updated_at": "2020-07-18T21:50:37Z", "author_association": "MEMBER", "body": "```\r\n$ github-to-sqlite tags tags.db simonw/datasette dogsheep/github-to-sqlite\r\n$ sqlite-utils tables tags.db --counts\r\n[{\"table\": \"users\", \"count\": 2},\r\n {\"table\": \"licenses\", \"count\": 1},\r\n {\"table\": \"repos\", \"count\": 2},\r\n {\"table\": \"tags\", \"count\": 76},\r\n {\"table\": \"licenses_fts\", \"count\": 1},\r\n {\"table\": \"licenses_fts_data\", \"count\": 3},\r\n {\"table\": \"licenses_fts_idx\", \"count\": 1},\r\n {\"table\": \"licenses_fts_docsize\", \"count\": 1},\r\n {\"table\": \"licenses_fts_config\", \"count\": 1},\r\n {\"table\": \"repos_fts\", \"count\": 2},\r\n {\"table\": \"repos_fts_data\", \"count\": 3},\r\n {\"table\": \"repos_fts_idx\", \"count\": 1},\r\n {\"table\": \"repos_fts_docsize\", \"count\": 2},\r\n {\"table\": \"repos_fts_config\", \"count\": 1},\r\n {\"table\": \"users_fts\", \"count\": 2},\r\n {\"table\": \"users_fts_data\", \"count\": 3},\r\n {\"table\": \"users_fts_idx\", \"count\": 1},\r\n {\"table\": \"users_fts_docsize\", \"count\": 2},\r\n {\"table\": \"users_fts_config\", \"count\": 1}]\r\n$ sqlite-utils rows tags.db tags\r\n[{\"repo_id\": 107914493, \"name\": \"0.45\", \"sha\": \"f1f581b7ffcd5d8f3ae6c1c654d813a6641410eb\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.45a5\", \"sha\": \"676bb64c877d73f8ff496cef4632f5a8a5a9283c\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.45a4\", \"sha\": \"265483173bc8341dc02c8b782b9b59d2ce8bbedb\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.45a3\", \"sha\": \"1f55a4a2b68fa65e56a28baeb7f44122fdeca7e7\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.45a2\", \"sha\": \"1a5b7d318fa923edfcefd3df8f64dae2e9c49d3f\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.45a1\", \"sha\": \"b59b92b1b0517cf18fa748ff9d0a0bf86298dd43\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.45a0\", \"sha\": \"dda932d818b34ccab11730a76554f0a3748d8348\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.44\", \"sha\": \"b906030235efbdff536405d66078f4868ce0d3bd\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.43\", \"sha\": \"d56f402822df102f9cf1a9a056449d01a15e3aae\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.42\", \"sha\": \"af6c6c5d6f929f951c0e63bfd1c82e37a071b50f\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.41\", \"sha\": \"182e5c8745c94576718315f7596ccc81e5e2417b\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.40\", \"sha\": \"8da108193b08abf140716f8ac499f32309dfe9cf\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.39\", \"sha\": \"dedd775512daee49925882654f252df61a9e3b6d\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.38\", \"sha\": \"7e357abbc38dcc9d19a2f1df3252668a48e941e4\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.37.1\", \"sha\": \"be20e6991eac2baa9b43e9b26ae209bae805ede5\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.37\", \"sha\": \"c9e6841482b299fceadc5ad548c2dbf58a8f1227\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.36\", \"sha\": \"b031fe97636b80b05fec409ee1dffb7d044fd4e9\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.35\", \"sha\": \"30b6f71b306a43605c99bef79302ed5cb22d1924\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.34\", \"sha\": \"e7f60d2a9b59752e20de8412f7b0a3e9a5359a31\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.33\", \"sha\": \"59e7014c8a0f4102d7dc79f517540c55c49e1554\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.32\", \"sha\": \"a95bedb9c423fa6d772c93ef47bc40f13a5bea50\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.31.2\", \"sha\": \"b51f258d00bb3c3b401f15d46a1fbd50394dbe1c\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.31.1\", \"sha\": \"a22c7761b61baa61b8e3da7d30887468d61d6b83\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.31\", \"sha\": \"7f89928062b1a1fdb2625a946f7cd5161e597401\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.30.2\", \"sha\": \"2bf7ce5f517d772a16d7855a35a8a75d4456aad7\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.30.1\", \"sha\": \"3ca290e0db03bb4747e24203c445873f74512107\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.30\", \"sha\": \"8050f9e1ece9afd0236ad38c6458c12a4ad917e6\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.29.3\", \"sha\": \"0fc8afde0eb5ef677f4ac31601540d6168c8208d\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.29.2\", \"sha\": \"6abe6faff6b035e9334dd05f8c741ae9b7a47440\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.29.1\", \"sha\": \"2a94f3719fb2c4335fcda374fa92f87272b02d34\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.29\", \"sha\": \"fb7ee8e0ad59a15083234a48e935525f6e7257dd\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.28\", \"sha\": \"e518f76c5f5dd0138032bfb26387f5bb91086a3f\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.27.1\", \"sha\": \"3f3f29ac9afe7c41ffc48a3bd2af473a53eecc8a\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.27\", \"sha\": \"436b8bc1d17c2ab415800ab209204f94e7f7929e\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.26.2\", \"sha\": \"a418c8b44f82d456be523c8690cf7236bb648c22\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.26.1\", \"sha\": \"4722acc73ce761556b18f5dcbe36b7fef2ee2c69\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.26\", \"sha\": \"424e146697309a54c05d5d1ba1f840849ddbafdc\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.25.2\", \"sha\": \"b5128fc53fce6a1bf3b16bad9f318451bc1d1263\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.25.1\", \"sha\": \"3dc0b3fa8c9b9bd81540ffe20c8b7e7a72465274\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.25\", \"sha\": \"57a71377c992753327a16b417daf79df7f506dd1\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.24\", \"sha\": \"28872a1fa789f314b0342f4e6182f1c78d6e2bca\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.23.2\", \"sha\": \"6df6f712b36f0fe75694174906e31242427a8d1d\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.23.1\", \"sha\": \"dea86b9fba78e032ad09673e884e764387daf209\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.23\", \"sha\": \"e04f5b0d348ef7275a0a5ab9eb53527105132885\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.22.1\", \"sha\": \"5d6252788230d168ba09f379d1d2af867e3302ab\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.22\", \"sha\": \"558d9d7bfef3dd633eb16389281b67d42c9bdeef\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.21\", \"sha\": \"403211de632cd15f0820cc9399305fc43c187b47\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.20\", \"sha\": \"3a5d7951ce8f35118ffdd7f8d86e09b909e1218c\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.19\", \"sha\": \"ba9bfa583179c25aaef94b1f44da7eba74620b9a\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.18\", \"sha\": \"43ae15c0d14b3e968e8d5bfef72ac0c39783c3a2\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.17\", \"sha\": \"fb988ace7c7e2bee5ac142a0eab22431d0675a77\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.16\", \"sha\": \"b6539ff04502536bd1fa96e3b1430bdafc456826\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.15\", \"sha\": \"7706fe0c67aba5cfe905c7906cae9e0c43cd75b2\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.14\", \"sha\": \"2edc652df6d786e4f2c3f073e3567002d248be09\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.13\", \"sha\": \"c160f15c3937f8fbe581276f811e8c58f9137bb1\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.12\", \"sha\": \"51bdd67691bd69082ae7690af8b905f06050ee80\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.11\", \"sha\": \"b0f3d4e375655f0764f3137dbcede324f9bbc0cb\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.10\", \"sha\": \"5928c11ee798a232aa4096706cd47e639d1c9fc2\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.9\", \"sha\": \"d75f423b6fcfc074b7c6f8f7679da8876f181edd\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.8\", \"sha\": \"fe279ab7b4ae99dab295d5cf4d39ad06d782997e\"},\r\n {\"repo_id\": 107914493, \"name\": \"0.7\", \"sha\": \"6b3b05b6db0d2a7b7cec8b8dbb4ddc5e12a376b2\"},\r\n {\"repo_id\": 207052882, \"name\": \"2.3\", \"sha\": \"7090e43d804724ef3b31ae5ca9efd6ac05f76cbc\"},\r\n {\"repo_id\": 207052882, \"name\": \"2.2\", \"sha\": \"4fe69783b55465e7692a807d3a02a710f69c9c42\"},\r\n {\"repo_id\": 207052882, \"name\": \"2.1\", \"sha\": \"9d7aed336c8e62bf372caa800cb4aae3985cbae9\"},\r\n {\"repo_id\": 207052882, \"name\": \"2.0\", \"sha\": \"44611df1524a03ce305405e5902c9615e3c73a72\"},\r\n {\"repo_id\": 207052882, \"name\": \"1.1\", \"sha\": \"5cd34bd07d704487d48ac741ee5da5317afe88d2\"},\r\n {\"repo_id\": 207052882, \"name\": \"1.0.1\", \"sha\": \"3b7ab5685de89fcb6fc92d320c0e24b17be05570\"},\r\n {\"repo_id\": 207052882, \"name\": \"1.0\", \"sha\": \"1ea30c8fb1d080bd5e38c577e3ad20bb527a2fe6\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.7\", \"sha\": \"e35eec4343aa560c58c1634cc228d0d46c442304\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.6\", \"sha\": \"9eb737090fafd0e5a7e314be48402374d99e9828\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.5\", \"sha\": \"ae9035f8fe5aff1c54bff4c6b4c2e808a44f0f2a\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.4\", \"sha\": \"8c6251c31a05c58c2bfbef114247642d1b3dbb44\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.3\", \"sha\": \"f697f247468516aa4ee13b1862b59e0dba18d00f\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.2\", \"sha\": \"0fe96bc50fb3d7b1c7e4577db0ddf207eaeebbb9\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.1.1\", \"sha\": \"321e0284c64dc48b2143311009886293c05edb07\"},\r\n {\"repo_id\": 207052882, \"name\": \"0.1\", \"sha\": \"7387c88a3f84704548e81d43b91615c02b61a957\"}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660355904, "label": "github-to-sqlite tags command for fetching tags"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/43#issuecomment-660536265", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43", "id": 660536265, "node_id": "MDEyOklzc3VlQ29tbWVudDY2MDUzNjI2NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-18T20:15:12Z", "updated_at": "2020-07-18T20:15:12Z", "author_association": "MEMBER", "body": "I want to create a SQL query which shows me all of my repositories that have commits that are NOT in the most recent release.\r\n\r\nThe releases table doesn't have enough information for this because it doesn't tell you the commit hash associated with each release, just the tag: https://github-to-sqlite.dogsheep.net/github/releases", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 660355904, "label": "github-to-sqlite tags command for fetching tags"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653966670", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41", "id": 653966670, "node_id": "MDEyOklzc3VlQ29tbWVudDY1Mzk2NjY3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-06T01:07:02Z", "updated_at": "2020-07-06T01:07:02Z", "author_association": "MEMBER", "body": "OK that fix worked.https://github.com/dogsheep/github-to-sqlite/runs/839764768?check_suite_focus=true", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 651159727, "label": "Demo is failing to deploy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962708", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41", "id": 653962708, "node_id": "MDEyOklzc3VlQ29tbWVudDY1Mzk2MjcwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-06T00:43:10Z", "updated_at": "2020-07-06T00:43:10Z", "author_association": "MEMBER", "body": "I bet it's datasette-search-all.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 651159727, "label": "Demo is failing to deploy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962669", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41", "id": 653962669, "node_id": "MDEyOklzc3VlQ29tbWVudDY1Mzk2MjY2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-06T00:42:57Z", "updated_at": "2020-07-06T00:42:57Z", "author_association": "MEMBER", "body": "https://github-to-sqlite.dogsheep.net/-/plugins\r\n\r\n```json\r\n[\r\n {\r\n \"name\": \"datasette-json-html\",\r\n \"static\": false,\r\n \"templates\": false,\r\n \"version\": \"0.6\",\r\n \"hooks\": [\r\n \"prepare_connection\",\r\n \"render_cell\"\r\n ]\r\n },\r\n {\r\n \"name\": \"datasette-render-markdown\",\r\n \"static\": false,\r\n \"templates\": false,\r\n \"version\": \"1.1.2\",\r\n \"hooks\": [\r\n \"extra_template_vars\",\r\n \"render_cell\"\r\n ]\r\n },\r\n {\r\n \"name\": \"datasette-pretty-json\",\r\n \"static\": false,\r\n \"templates\": false,\r\n \"version\": \"0.2\",\r\n \"hooks\": [\r\n \"render_cell\"\r\n ]\r\n },\r\n {\r\n \"name\": \"datasette-search-all\",\r\n \"static\": false,\r\n \"templates\": true,\r\n \"version\": \"0.2.1\",\r\n \"hooks\": [\r\n \"asgi_wrapper\",\r\n \"extra_template_vars\"\r\n ]\r\n },\r\n {\r\n \"name\": \"datasette-vega\",\r\n \"static\": true,\r\n \"templates\": false,\r\n \"version\": \"0.6.2\",\r\n \"hooks\": [\r\n \"extra_css_urls\",\r\n \"extra_js_urls\"\r\n ]\r\n }\r\n]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 651159727, "label": "Demo is failing to deploy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962530", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41", "id": 653962530, "node_id": "MDEyOklzc3VlQ29tbWVudDY1Mzk2MjUzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-06T00:42:13Z", "updated_at": "2020-07-06T00:42:13Z", "author_association": "MEMBER", "body": "So it looks like it's the ASGI lifespan change I made in https://github.com/simonw/datasette/commit/16f592247a2a0e140ada487e9972645406dcae69 - It must be incompatible with one of the plugins.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 651159727, "label": "Demo is failing to deploy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653962418", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41", "id": 653962418, "node_id": "MDEyOklzc3VlQ29tbWVudDY1Mzk2MjQxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-06T00:41:38Z", "updated_at": "2020-07-06T00:41:38Z", "author_association": "MEMBER", "body": "https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/logs?project=datasette-222320 has some clues.\r\n\r\n![\"github-to-sqlite_\u2013_Service_details_\u2013_Cloud_Run_\u2013_datasette_\u2013_Google_Cloud_Platform\"](\"https://user-images.githubusercontent.com/9599/86545887-92086280-bee6-11ea-8787-b3b85a7bb4b9.png\")
\r\n![\"github-to-sqlite_\u2013_Service_details_\u2013_Cloud_Run_\u2013_datasette_\u2013_Google_Cloud_Platform\"](\"https://user-images.githubusercontent.com/9599/86545917-c11ed400-bee6-11ea-9cf8-957911ca9f6a.png\")
\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 651159727, "label": "Demo is failing to deploy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653960989", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41", "id": 653960989, "node_id": "MDEyOklzc3VlQ29tbWVudDY1Mzk2MDk4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-06T00:32:34Z", "updated_at": "2020-07-06T00:32:34Z", "author_association": "MEMBER", "body": "Same error.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 651159727, "label": "Demo is failing to deploy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/41#issuecomment-653947916", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41", "id": 653947916, "node_id": "MDEyOklzc3VlQ29tbWVudDY1Mzk0NzkxNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-07-05T22:40:47Z", "updated_at": "2020-07-05T22:40:47Z", "author_association": "MEMBER", "body": "Might be that it's not got enough RAM. I'll try deploying to a larger instance.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 651159727, "label": "Demo is failing to deploy"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645599881", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47", "id": 645599881, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTU5OTg4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-17T20:13:48Z", "updated_at": "2020-06-17T20:13:48Z", "author_association": "MEMBER", "body": "I've now figured out how to compile specific SQLite versions to help replicate this problem: https://github.com/simonw/til/blob/master/sqlite/ld-preload.md\r\n\r\nNext step: replicate the problem!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639542974, "label": "Fall back to FTS4 if FTS5 is not available"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/47#issuecomment-645512127", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/47", "id": 645512127, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTUxMjEyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-17T17:24:22Z", "updated_at": "2020-06-17T17:24:22Z", "author_association": "MEMBER", "body": "That means your version of SQLite is old enough that it doesn't support the FTS5 extension.\r\n\r\nCould you share what operating system you're running, and what the output is that you get from running this?\r\n\r\n python -c 'import sqlite3; print(sqlite3.connect(\":memory:\").execute(\"select sqlite_version()\").fetchone()[0])'\r\n\r\nI can teach this tool to fall back on FTS4 if FTS5 isn't available.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639542974, "label": "Fall back to FTS4 if FTS5 is not available"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643414646", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40", "id": 643414646, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzQxNDY0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T18:06:48Z", "updated_at": "2020-06-12T18:06:48Z", "author_association": "MEMBER", "body": "That fixed it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637899539, "label": "Demo deploy is broken"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643393506", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40", "id": 643393506, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzM5MzUwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T17:21:14Z", "updated_at": "2020-06-12T17:21:14Z", "author_association": "MEMBER", "body": "I only install SQLite for this:\r\n\r\nhttps://github.com/dogsheep/github-to-sqlite/blob/c0d54e0260468be38152293df5abd775c068495d/.github/workflows/deploy-demo.yml#L77-L78\r\n\r\nI'm going to remove the need to install sqlite3 by making this possible with sqlite-utils: https://github.com/simonw/sqlite-utils/issues/115", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637899539, "label": "Demo deploy is broken"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633704127", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633704127, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzcwNDEyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T20:14:22Z", "updated_at": "2020-05-25T20:14:22Z", "author_association": "MEMBER", "body": "https://github.com/dogsheep/dogsheep-photos/blob/0.4.1/README.md#serving-photos-locally-with-datasette-media", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633644225", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633644225, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzY0NDIyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T16:30:44Z", "updated_at": "2020-05-25T16:30:44Z", "author_association": "MEMBER", "body": "I'll add docs on using `datasette-json-html` too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633643921", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633643921, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzY0MzkyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T16:29:44Z", "updated_at": "2020-05-25T16:29:44Z", "author_association": "MEMBER", "body": "https://github.com/dogsheep/dogsheep-photos/blob/dc43fa8653cb9c7238a36f52239b91d1ec916d5c/README.md#serving-photos-locally-with-datasette-media", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 613006393, "label": "Ability to serve thumbnailed Apple Photo from its place on disk"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-633629944", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20", "id": 633629944, "node_id": "MDEyOklzc3VlQ29tbWVudDYzMzYyOTk0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-25T15:47:42Z", "updated_at": "2020-05-25T15:47:42Z", "author_association": "MEMBER", "body": "I'll add a proper section to the README, but for the moment here's how I do this.\r\n\r\nFirst, install `datasette` and the `datasette-media` plugin.\r\n\r\nCreate a `metadata.yaml` file with the following content:\r\n\r\n```yaml\r\nplugins:\r\n datasette-media:\r\n photo:\r\n sql: |-\r\n select path as filepath, 200 as resize_height from apple_photos where uuid = :key\r\n photo-big:\r\n sql: |-\r\n select path as filepath, 1024 as resize_height from apple_photos where uuid = :key\r\n```\r\nNow run `datasette -m metadata.yaml photos.db` - thumbnails will be served at http://127.0.0.1:8001/-/media/photo/F4469918-13F3-43D8-9EC1-734C0E6B60AD and larger sizes of the image at http://127.0.0.1:8001/-/media/photo-big/A8B02C7D-365E-448B-9510-69F80C26304D\r\n\r\nI also made myself two custom pages, one showing recent images and one showing random images.\r\n\r\nTo do this, install the `datasette-template-sql` plugin and then create a `templates/pages` directory and add these files:\r\n\r\n`recent-photos.html`\r\n```html\r\n\r\n{% endfor %}\r\n\r\n{% endfor %}\r\n![\"psi_copy__select_json_object__img_src____https___photos_simonwillison_net_i______photos_sha256___________photos_ext______w_400___as_photo__group_concat_strip_null_chars_groups_content_string________as_words__assets_uuid_0__assets_uuid_1__to\"](\"https://user-images.githubusercontent.com/9599/81037264-f1021a80-8e56-11ea-9924-6f9f55a0fb4b.png\")
\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623857417", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623857417, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzg1NzQxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T05:01:47Z", "updated_at": "2020-05-05T05:01:47Z", "author_association": "MEMBER", "body": "Even that didn't work - it didn't copy across the rowid values. I'm pretty sure that's what's wrong here:\r\n```\r\nsqlite3 /Users/simon/Pictures/Photos\\ Library.photoslibrary/database/search/psi.sqlite 'select rowid, uuid_0, uuid_1 from assets limit 10' \r\n1619605|-9205353363298198838|4814875488794983828\r\n1641378|-9205348195631362269|390804289838822030\r\n1634974|-9205331524553603243|-3834026796261633148\r\n1619083|-9205326176986145401|7563404215614709654\r\n22131|-9205315724827218763|8370531509591906734\r\n1645633|-9205247376092758131|-1311540150497601346\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623855885", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623855885, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzg1NTg4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T04:54:39Z", "updated_at": "2020-05-05T04:54:53Z", "author_association": "MEMBER", "body": "Trying this import mechanism instead:\r\n`sqlite3 /Users/simon/Pictures/Photos\\ Library.photoslibrary/database/search/psi.sqlite .dump | grep -v 'CREATE INDEX' | grep -v 'CREATE TRIGGER' | grep -v 'CREATE VIRTUAL TABLE' | sqlite3 search.db`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623855841", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623855841, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzg1NTg0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T04:54:28Z", "updated_at": "2020-05-05T04:54:28Z", "author_association": "MEMBER", "body": "Things were not matching up for me correctly:\r\n\r\n![\"search__select_json_object__img_src____https___photos_simonwillison_net_i______photos_sha256___________photos_ext______w_400___as_photo__groups_content_string__assets_uuid_0__assets_uuid_1__to_uuid_assets_uuid_0__assets_uuid_1__as_uuid__pho\"](\"https://user-images.githubusercontent.com/9599/81035923-ca8db080-8e51-11ea-95a7-6ee60bae7502.png\")
\r\n\r\nI think that's because my import script didn't correctly import the existing `rowid` values.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623846880", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623846880, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzg0Njg4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T04:06:08Z", "updated_at": "2020-05-05T04:06:08Z", "author_association": "MEMBER", "body": "This function seems to convert them into UUIDs that match my photos:\r\n```python\r\ndef to_uuid(uuid_0, uuid_1):\r\n b = uuid_0.to_bytes(8, 'little', signed=True) + uuid_1.to_bytes(8, 'little', signed=True)\r\n return str(uuid.UUID(bytes=b)).upper()\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623811131", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623811131, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzgxMTEzMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T03:16:18Z", "updated_at": "2020-05-05T03:16:18Z", "author_association": "MEMBER", "body": "Here's how to convert two integers unto a UUID using Java. Not sure if it's the solution I need though (or how to do the same thing in Python):\r\n\r\nhttps://repl.it/repls/EuphoricSomberClasslibrary\r\n\r\n![\"Repl_it_-_EuphoricSomberClasslibrary\"](\"https://user-images.githubusercontent.com/9599/81032267-0d488c00-8e44-11ea-9be7-680eaccd1611.png\")
\r\n\r\n```java\r\nimport java.util.UUID;\r\n\r\nclass Main {\r\n public static void main(String[] args) {\r\n java.util.UUID uuid = new java.util.UUID(\r\n 2544182952487526660L,\r\n -3640314103732024685L\r\n );\r\n System.out.println(\r\n uuid\r\n );\r\n }\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623807568", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623807568, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzgwNzU2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T02:56:06Z", "updated_at": "2020-05-05T02:56:06Z", "author_association": "MEMBER", "body": "I'm pretty sure this is what I'm after. The `groups` table has what looks like identified labels in the rows with category = 2025:\r\n\r\n![\"words__groups__2_528_rows_where_where_category___2025\"](\"https://user-images.githubusercontent.com/9599/81031361-e0df4080-8e40-11ea-9060-6d850aa52140.png\")
\r\n\r\nThen there's a `ga` table that maps groups to assets:\r\n\r\n![\"words__ga__633_653_rows\"](\"https://user-images.githubusercontent.com/9599/81031387-f48aa700-8e40-11ea-9a3d-da23903be928.png\")
\r\n\r\nAnd an `assets` table which looks like it has one row for every one of my photos:\r\n\r\n![\"words__assets__40_419_rows\"](\"https://user-images.githubusercontent.com/9599/81031402-04a28680-8e41-11ea-8047-e9199d068563.png\")
\r\n\r\nOne major challenge: these UUIDs are split into two integer numbers, `uuid_0` and `uuid_1` - but the main photos database uses regular UUIDs like this:\r\n\r\n\r\n\r\nI need to figure out how to match up these two different UUID representations. I asked on Twitter if anyone has any ideas: https://twitter.com/simonw/status/1257500689019703296", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623806687", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623806687, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzgwNjY4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T02:51:16Z", "updated_at": "2020-05-05T02:51:16Z", "author_association": "MEMBER", "body": "Running datasette against it directly doesn't work:\r\n```\r\nsimon@Simons-MacBook-Pro search % datasette psi.sqlite\r\nServe! files=('psi.sqlite',) (immutables=()) on port 8001\r\nUsage: datasette serve [OPTIONS] [FILES]...\r\n\r\nError: Connection to psi.sqlite failed check: no such tokenizer: PSITokenizer\r\n```\r\nInstead, I created a new SQLite database with a copy of some of the key tables, like this:\r\n```\r\nsqlite-utils rows psi.sqlite groups | sqlite-utils insert /tmp/search.db groups -\r\nsqlite-utils rows psi.sqlite assets | sqlite-utils insert /tmp/search.db assets -\r\nsqlite-utils rows psi.sqlite ga | sqlite-utils insert /tmp/search.db ga -\r\nsqlite-utils rows psi.sqlite collections | sqlite-utils insert /tmp/search.db collections -\r\nsqlite-utils rows psi.sqlite gc | sqlite-utils insert /tmp/search.db gc -\r\nsqlite-utils rows psi.sqlite lookup | sqlite-utils insert /tmp/search.db lookup -\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623806533", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623806533, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzgwNjUzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T02:50:16Z", "updated_at": "2020-05-05T02:50:16Z", "author_association": "MEMBER", "body": "I figured there must be a separate database that Photos uses to store the text of the identified labels.\r\n\r\nI used \"Open Files and Ports\" in Activity Monitor against the Photos app to try and spot candidates... and found `/Users/simon/Pictures/Photos Library.photoslibrary/database/search/psi.sqlite` - a 53MB SQLite database file.\r\n\r\n![\"Item-0_and_Item-0_and_Item-0_and_Item-0\"](\"https://user-images.githubusercontent.com/9599/81031213-61ea0800-8e40-11ea-8237-cfce4a5128e0.png\")
\r\n\r\nHere's the schema of that file:\r\n```\r\n$ sqlite3 psi.sqlite .schema\r\nCREATE TABLE word_embedding(word TEXT, extended_word TEXT, score DOUBLE);\r\nCREATE INDEX word_embedding_index ON word_embedding(word);\r\nCREATE VIRTUAL TABLE word_embedding_prefix USING fts5(extended_word)\r\n/* word_embedding_prefix(extended_word) */;\r\nCREATE TABLE IF NOT EXISTS 'word_embedding_prefix_data'(id INTEGER PRIMARY KEY, block BLOB);\r\nCREATE TABLE IF NOT EXISTS 'word_embedding_prefix_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;\r\nCREATE TABLE IF NOT EXISTS 'word_embedding_prefix_content'(id INTEGER PRIMARY KEY, c0);\r\nCREATE TABLE IF NOT EXISTS 'word_embedding_prefix_docsize'(id INTEGER PRIMARY KEY, sz BLOB);\r\nCREATE TABLE IF NOT EXISTS 'word_embedding_prefix_config'(k PRIMARY KEY, v) WITHOUT ROWID;\r\nCREATE TABLE groups(category INT2, owning_groupid INT, content_string TEXT, normalized_string TEXT, lookup_identifier TEXT, token_ranges_0 INT8, token_ranges_1 INT8, UNIQUE(category, owning_groupid, content_string, lookup_identifier, token_ranges_0, token_ranges_1));\r\nCREATE TABLE assets(uuid_0 INT, uuid_1 INT, creationDate INT, UNIQUE(uuid_0, uuid_1));\r\nCREATE TABLE ga(groupid INT, assetid INT, PRIMARY KEY(groupid, assetid));\r\nCREATE TABLE collections(uuid_0 INT, uuid_1 INT, startDate INT, endDate INT, title TEXT, subtitle TEXT, keyAssetUUID_0 INT, keyAssetUUID_1 INT, typeAndNumberOfAssets INT32, sortDate DOUBLE, UNIQUE(uuid_0, uuid_1));\r\nCREATE TABLE gc(groupid INT, collectionid INT, PRIMARY KEY(groupid, collectionid));\r\nCREATE VIRTUAL TABLE prefix USING fts5(content='groups', normalized_string, category UNINDEXED, tokenize = 'PSITokenizer');\r\nCREATE TABLE IF NOT EXISTS 'prefix_data'(id INTEGER PRIMARY KEY, block BLOB);\r\nCREATE TABLE IF NOT EXISTS 'prefix_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;\r\nCREATE TABLE IF NOT EXISTS 'prefix_docsize'(id INTEGER PRIMARY KEY, sz BLOB);\r\nCREATE TABLE IF NOT EXISTS 'prefix_config'(k PRIMARY KEY, v) WITHOUT ROWID;\r\nCREATE TABLE lookup(identifier TEXT PRIMARY KEY, category INT2);\r\nCREATE TRIGGER trigger_groups_insert AFTER INSERT ON groups BEGIN INSERT INTO prefix(rowid, normalized_string, category) VALUES (new.rowid, new.normalized_string, new.category); END;\r\nCREATE TRIGGER trigger_groups_delete AFTER DELETE ON groups BEGIN INSERT INTO prefix(prefix, rowid, normalized_string, category) VALUES('delete', old.rowid, old.normalized_string, old.category); END;\r\nCREATE INDEX group_pk ON groups(category, content_string, normalized_string, lookup_identifier);\r\nCREATE INDEX asset_pk ON assets(uuid_0, uuid_1);\r\nCREATE INDEX ga_assetid ON ga(assetid, groupid);\r\nCREATE INDEX collection_pk ON collections(uuid_0, uuid_1);\r\nCREATE INDEX gc_collectionid ON gc(collectionid);\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623806085", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623806085, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzgwNjA4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T02:47:18Z", "updated_at": "2020-05-05T02:47:18Z", "author_association": "MEMBER", "body": "In https://github.com/RhetTbull/osxphotos/issues/121#issuecomment-623249263 Rhet Turnbull spotted a table called `ZSCENEIDENTIFIER` which looked like it might have the right data, but the columns in it aren't particularly helpful:\r\n```\r\nZ_PK,Z_ENT,Z_OPT,ZSCENEIDENTIFIER,ZASSETATTRIBUTES,ZCONFIDENCE\r\n8,49,1,731,5,0.11834716796875\r\n9,49,1,684,6,0.0233648251742125\r\n10,49,1,1702,1,0.026153564453125\r\n```\r\nI love the look of those confidence scores, but what do the numbers mean?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623805823", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16", "id": 623805823, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzgwNTgyMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-05T02:45:56Z", "updated_at": "2020-05-05T02:45:56Z", "author_association": "MEMBER", "body": "I filed an issue with `osxphotos` about this here: https://github.com/RhetTbull/osxphotos/issues/121", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612287234, "label": "Import machine-learning detected labels (dog, llama etc) from Apple Photos"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/dogsheep-photos/issues/15#issuecomment-623739934", "issue_url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15", "id": 623739934, "node_id": "MDEyOklzc3VlQ29tbWVudDYyMzczOTkzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-05-04T22:24:26Z", "updated_at": "2020-05-04T22:24:26Z", "author_association": "MEMBER", "body": "Twitter thread with some examples of photos that are coming up from queries against these scores: https://twitter.com/simonw/status/1257434670750408705", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 612151767, "label": "Expose scores from ZCOMPUTEDASSETATTRIBUTES"}, "performed_via_github_app": null}