html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/dogsheep/dogsheep-beta/issues/29#issuecomment-712266834,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29,712266834,MDEyOklzc3VlQ29tbWVudDcxMjI2NjgzNA==,9599,simonw,2020-10-19T16:01:23Z,2020-10-19T16:01:23Z,MEMBER,Might just be a documented pattern for how to configure this in YAML templates.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",724759588,Add search highlighting snippets, https://github.com/dogsheep/github-to-sqlite/issues/50#issuecomment-711569063,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/50,711569063,MDEyOklzc3VlQ29tbWVudDcxMTU2OTA2Mw==,9599,simonw,2020-10-19T05:01:29Z,2020-10-19T05:01:29Z,MEMBER,"Demo of `--accept`: github-to-sqlite get /repos/simonw/datasette/readme --accept 'application/vnd.github.VERSION.html' ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703218756,Commands for making authenticated API calls, https://github.com/dogsheep/dogsheep-beta/issues/28#issuecomment-711089647,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/28,711089647,MDEyOklzc3VlQ29tbWVudDcxMTA4OTY0Nw==,9599,simonw,2020-10-17T22:43:13Z,2020-10-17T22:43:13Z,MEMBER,"Since my personal Dogsheep uses Datasette authentication, I'm going to need to pass through cookies. https://github.com/simonw/datasette/issues/1020 will solve that in the future but for now I need to solve it explicitly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723861683,Switch to using datasette.client, https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711081703,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711081703,MDEyOklzc3VlQ29tbWVudDcxMTA4MTcwMw==,9599,simonw,2020-10-17T21:18:35Z,2020-10-17T21:18:35Z,MEMBER,"OK, if you upgrade to the just-released 1.0 this should work (it worked against my Spanish export).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",723838331,export.xml file name varies with different language settings, https://github.com/dogsheep/healthkit-to-sqlite/issues/11#issuecomment-711079760,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/11,711079760,MDEyOklzc3VlQ29tbWVudDcxMTA3OTc2MA==,9599,simonw,2020-10-17T21:00:05Z,2020-10-17T21:00:05Z,MEMBER,Checking for either ` "" TEMPLATE = """"""
{}
"""""".strip() EN_MEDIA_SCRIPT = """""" Array.from(document.querySelectorAll('en-media')).forEach(el => { let hash = el.getAttribute('hash'); let type = el.getAttribute('type'); let path = `/evernote/resources_data/${hash}.json?_shape=array`; fetch(path).then(r => r.json()).then(rows => { let b64 = rows[0].data.encoded; let data = `data:${type};base64,${b64}`; el.innerHTML = ``; }); }); """""" @hookimpl def render_cell(value, table): if not table: # Don't render content from arbitrary SQL queries, could be XSS hole return if not value or not isinstance(value, str): return value = value.strip() if value.startswith(START) and value.endswith(END): trimmed = value[len(START) : -len(END)] trimmed = trimmed.split("">"", 1)[1] # Replace those horrible double newlines trimmed = trimmed.replace(""

"", ""
"") return jinja2.Markup(TEMPLATE.format(trimmed)) @hookimpl def extra_body_script(): return EN_MEDIA_SCRIPT ``` It works! It does however demonstrate that Evernote's ""clip this webpage"" feature means there is a LOT of weird HTML that can get into a note. It looks like they've filtered out the scripts but I wouldn't bet on it - they certainly don't filter out many of the inline styles. So running Bleach is almost certainly a good idea.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from tags inline in Datasette, https://github.com/dogsheep/swarm-to-sqlite/issues/8#issuecomment-707332912,https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/8,707332912,MDEyOklzc3VlQ29tbWVudDcwNzMzMjkxMg==,9599,simonw,2020-10-12T20:35:06Z,2020-10-12T20:35:06Z,MEMBER,Shipped a fix for this in [swarm-to-sqlite 0.3.2](https://github.com/dogsheep/swarm-to-sqlite/releases/tag/0.3.2).,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",648245071,Error thrown: table photos has no column named hasSticker, https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706786548,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4,706786548,MDEyOklzc3VlQ29tbWVudDcwNjc4NjU0OA==,9599,simonw,2020-10-11T23:39:46Z,2020-10-11T23:39:46Z,MEMBER,Should have used porter stemming for this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938508,Configure FTS + add an index on the date columns, https://github.com/dogsheep/evernote-to-sqlite/issues/6#issuecomment-706785201,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6,706785201,MDEyOklzc3VlQ29tbWVudDcwNjc4NTIwMQ==,9599,simonw,2020-10-11T23:29:39Z,2020-10-11T23:29:39Z,MEMBER,"It looks to me like each of those `` blocks has a number of guesses in order of confidence: ```xml wonders, wanders, wonders ? wonders wonders. ``` So maybe the best approach here is to just take the first `t` element within each `item`. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718949182,Better handling of OCR data, https://github.com/dogsheep/evernote-to-sqlite/issues/6#issuecomment-706785086,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6,706785086,MDEyOklzc3VlQ29tbWVudDcwNjc4NTA4Ng==,9599,simonw,2020-10-11T23:28:50Z,2020-10-11T23:28:50Z,MEMBER,"The XML for the OCR stuff is a bit weird. Currently I'm doing this to it: https://github.com/dogsheep/evernote-to-sqlite/blob/c33d7b043a45eb3e88676e5fa3ce31755199d9f8/evernote_to_sqlite/utils.py#L70-L78 This can produce some odd results, for example: > Sure 'Sure, 'Sure. Sure, Sure. sure sure. sure ? If you If Yau [you live jive In m 1n an area devoid of natural wonders, wanders, wonders ? wonders wonders. your mind will be blown, blown' blown. blown ? -e i ? ,1 IL it ? at ? KY ? fl ft bat at Which came from this image: ![image](https://user-images.githubusercontent.com/9599/95692952-5dd7c880-0bde-11eb-939a-d10b800a4105.png) The XML for that is: ```xml Sure 'Sure, 'Sure. Sure, Sure. sure sure. sure ? If you If Yau [you live jive In m 1n an area devoid of natural wonders, wanders, wonders ? wonders wonders. your mind will be blown, blown' blown. blown ? -e i ? ,1 IL it ? at ? KY ? fl ft bat at ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718949182,Better handling of OCR data, https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706784028,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4,706784028,MDEyOklzc3VlQ29tbWVudDcwNjc4NDAyOA==,9599,simonw,2020-10-11T23:20:32Z,2020-10-11T23:20:32Z,MEMBER,I haven't done the FTS on OCR yet. I'm going to move that to another ticket because it requires more thought.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938508,Configure FTS + add an index on the date columns, https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776808,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776808,MDEyOklzc3VlQ29tbWVudDcwNjc3NjgwOA==,9599,simonw,2020-10-11T22:23:14Z,2020-10-11T22:23:14Z,MEMBER,... but it's still important to be able to get to the rendered note directly from the browse notes `/evernote/notes` page. Maybe use a simple `render_cell()` hook that just knows how to generate the link to the rendered note page?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from tags inline in Datasette, https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776680,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776680,MDEyOklzc3VlQ29tbWVudDcwNjc3NjY4MA==,9599,simonw,2020-10-11T22:22:16Z,2020-10-11T22:22:16Z,MEMBER,"Maybe the best way do this is with a custom route, `/-/evernote/note-id` - that way I can clean the HTML and resolve the other things in the `` structure without using `render_cell()` and the like. My concern about using `render_cell()` is that it could lead to weird security problems when combined with `?sql=` queries.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from tags inline in Datasette, https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776447,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776447,MDEyOklzc3VlQ29tbWVudDcwNjc3NjQ0Nw==,9599,simonw,2020-10-11T22:20:32Z,2020-10-11T22:20:32Z,MEMBER,Or... I could do this client-side. JavaScript that looks for `` tags and fetches the data using `fetch()` wouldn't be too hard to write.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from tags inline in Datasette, https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776242,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776242,MDEyOklzc3VlQ29tbWVudDcwNjc3NjI0Mg==,9599,simonw,2020-10-11T22:18:30Z,2020-10-11T22:19:48Z,MEMBER,"Alternatively, rather than relying on `datasette-media` this could base64-embed the images. `evernote-to-sqlite` could register itself as a Datasette plugin that knows how to do this. Maybe rename the column to `evernote_content` and register a render cell hook that knows how to rewrite those note bodies so that they are visible? Might need to feed them through Bleach too, just in case any nasty code can get into them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from tags inline in Datasette, https://github.com/dogsheep/evernote-to-sqlite/issues/5#issuecomment-706776180,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/5,706776180,MDEyOklzc3VlQ29tbWVudDcwNjc3NjE4MA==,9599,simonw,2020-10-11T22:17:55Z,2020-10-11T22:17:55Z,MEMBER,"We could even do server-side thumbnailing for some of these images, but I'm inclined to serve up the full size ones and set a width on the image element based on the `width` attribute on ``.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718938889,Figure out how to display images from tags inline in Datasette, https://github.com/dogsheep/evernote-to-sqlite/issues/1#issuecomment-706775706,https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/1,706775706,MDEyOklzc3VlQ29tbWVudDcwNjc3NTcwNg==,9599,simonw,2020-10-11T22:14:00Z,2020-10-11T22:14:00Z,MEMBER,A live demo would be good too.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",718934942,Documentation on how to use this with Datasette, https://github.com/dogsheep/github-to-sqlite/pull/48#issuecomment-704553385,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/48,704553385,MDEyOklzc3VlQ29tbWVudDcwNDU1MzM4NQ==,9599,simonw,2020-10-06T21:07:44Z,2020-10-06T21:07:44Z,MEMBER,"Sorry for not looking at this sooner, trying it out now - pull request looks great!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",681228542,Add pull requests, https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695879531,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695879531,MDEyOklzc3VlQ29tbWVudDY5NTg3OTUzMQ==,9599,simonw,2020-09-21T02:55:28Z,2020-09-21T02:55:54Z,MEMBER,"Actually for the tie-breaker it should be something like https://latest.datasette.io/fixtures?sql=select+pk%2C+created%2C+planet_int%2C+on_earth%2C+state%2C+city_id%2C+neighborhood%2C+tags%2C+complex_array%2C+distinct_some_null+from+facetable+where+%28created+%3E+%3Ap1+or+%28created+%3D+%3Ap1+and+%28%28pk+%3E+%3Ap0%29%29%29%29+order+by+created%2C+pk+limit+11&p0=10&p1=2019-01-16+08%3A00%3A00 ```sql where ( created > :p1 or ( created = :p1 and ((pk > :p0)) ) ) ``` But with `rowid` and `timestamp` in place of `pk` and `created`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination, https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695879237,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695879237,MDEyOklzc3VlQ29tbWVudDY5NTg3OTIzNw==,9599,simonw,2020-09-21T02:53:29Z,2020-09-21T02:53:29Z,MEMBER,"If previous page ended at `2018-02-11T16:32:53+00:00`: ```sql select search_index.rowid, search_index.type, search_index.key, search_index.title, search_index.category, search_index.timestamp, search_index.search_1 from search_index where date(""timestamp"") = '2018-02-11' and timestamp < '2018-02-11T16:32:53+00:00' order by search_index.timestamp desc, rowid limit 41 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination, https://github.com/dogsheep/dogsheep-beta/issues/16#issuecomment-695877627,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16,695877627,MDEyOklzc3VlQ29tbWVudDY5NTg3NzYyNw==,9599,simonw,2020-09-21T02:42:29Z,2020-09-21T02:42:29Z,MEMBER,"Fun twist: assuming `timestamp` is always stored as UTC, I need the interface to be timezone aware so I can see e.g. everything from 4th July 2020 in the San Francisco timezone definition of 4th July 2020.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694493566,Timeline view, https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695875274,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695875274,MDEyOklzc3VlQ29tbWVudDY5NTg3NTI3NA==,9599,simonw,2020-09-21T02:28:58Z,2020-09-21T02:28:58Z,MEMBER,Datasette's implementation is complex because it has to support compound primary keys: https://github.com/simonw/datasette/blob/a258339a935d8d29a95940ef1db01e98bb85ae63/datasette/utils/__init__.py#L88-L114 - but that's not something that's needed for dogsheep-beta.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination, https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695856967,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695856967,MDEyOklzc3VlQ29tbWVudDY5NTg1Njk2Nw==,9599,simonw,2020-09-21T00:26:59Z,2020-09-21T00:26:59Z,MEMBER,It's a shame Datasette doesn't currently have an easy way to implement sorted-by-rank keyset-paginated using a TableView or QueryView. I'll have to do this using the custom SQL query constructed in the plugin: https://github.com/dogsheep/dogsheep-beta/blob/bed9df2b3ef68189e2e445427721a28f4e9b4887/dogsheep_beta/__init__.py#L8-L43,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination, https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695856398,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695856398,MDEyOklzc3VlQ29tbWVudDY5NTg1NjM5OA==,9599,simonw,2020-09-21T00:22:20Z,2020-09-21T00:22:20Z,MEMBER,I'm going to try for keyset pagination sorted by relevance just as a learning exercise.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination, https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695855723,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695855723,MDEyOklzc3VlQ29tbWVudDY5NTg1NTcyMw==,9599,simonw,2020-09-21T00:16:52Z,2020-09-21T00:17:53Z,MEMBER,"It feels a bit weird to implement keyset pagination against results sorted by `rank` because the ranks could change substantially if the search index gets updated while the user is paginating. I may just ignore that though. If you want reliable pagination you can get it by sorting by date. Maybe it doesn't even make sense to offer pagination if you sort by relevance?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination, https://github.com/dogsheep/dogsheep-beta/issues/26#issuecomment-695855646,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/26,695855646,MDEyOklzc3VlQ29tbWVudDY5NTg1NTY0Ng==,9599,simonw,2020-09-21T00:16:11Z,2020-09-21T00:16:11Z,MEMBER,"Should I do this with offset/limit or should I do proper keyset pagination? I think keyset because then it will work well for the full search interface with no filters or search string.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",705215230,Pagination, https://github.com/dogsheep/dogsheep-beta/issues/16#issuecomment-695851036,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/16,695851036,MDEyOklzc3VlQ29tbWVudDY5NTg1MTAzNg==,9599,simonw,2020-09-20T23:34:57Z,2020-09-20T23:34:57Z,MEMBER,Really basic starting point is to add facet by date.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694493566,Timeline view, https://github.com/dogsheep/dogsheep-beta/issues/15#issuecomment-695124698,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/15,695124698,MDEyOklzc3VlQ29tbWVudDY5NTEyNDY5OA==,9599,simonw,2020-09-18T23:17:38Z,2020-09-18T23:17:38Z,MEMBER,This can be part of the demo instance in #6.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",694136490,Add a bunch of config examples, https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-695113871,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,695113871,MDEyOklzc3VlQ29tbWVudDY5NTExMzg3MQ==,9599,simonw,2020-09-18T22:30:17Z,2020-09-18T22:30:17Z,MEMBER,"I think I know what's going on here: https://github.com/dogsheep/dogsheep-beta/blob/0f1b951c5131d16f3c8559a8e4d79ed5c559e3cb/dogsheep_beta/__init__.py#L166-L171 This is a logic bug - the `compiled` variable could be the template from the previous loop!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'", https://github.com/dogsheep/dogsheep-beta/issues/25#issuecomment-695109140,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25,695109140,MDEyOklzc3VlQ29tbWVudDY5NTEwOTE0MA==,9599,simonw,2020-09-18T22:12:20Z,2020-09-18T22:12:20Z,MEMBER,Documented here: https://github.com/dogsheep/dogsheep-beta/blob/534fc9689227eba70e69a45da0cee5820bbda9e1/README.md#datasette-plugin,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",704685890,template_debug mechanism, https://github.com/dogsheep/dogsheep-beta/issues/25#issuecomment-695108895,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/25,695108895,MDEyOklzc3VlQ29tbWVudDY5NTEwODg5NQ==,9599,simonw,2020-09-18T22:11:32Z,2020-09-18T22:11:32Z,MEMBER,"I'm going to make this a new plugin configuration setting, `template_debug`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",704685890,template_debug mechanism, https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694557425,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694557425,MDEyOklzc3VlQ29tbWVudDY5NDU1NzQyNQ==,9599,simonw,2020-09-17T23:41:01Z,2020-09-17T23:41:01Z,MEMBER,I removed all of the `json.loads()` calls and I'm still getting that `Undefined` error.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'", https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694554584,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694554584,MDEyOklzc3VlQ29tbWVudDY5NDU1NDU4NA==,9599,simonw,2020-09-17T23:31:25Z,2020-09-17T23:31:25Z,MEMBER,"I'd prefer it if errors in these template fragments were displayed as errors inline where the fragment should have been inserted, rather than 500ing the whole page - especially since the template fragments are user-provided and could have all kinds of odd errors in them which should be as easy to debug as possible.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'", https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694553579,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694553579,MDEyOklzc3VlQ29tbWVudDY5NDU1MzU3OQ==,9599,simonw,2020-09-17T23:28:37Z,2020-09-17T23:28:37Z,MEMBER,"More investigation in pdb: ``` (dogsheep-beta) dogsheep-beta % datasette . --get '/-/beta?q=pycon&sort=oldest' --pdb > /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py(341)loads() -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' (Pdb) list 336 if s.startswith('\ufeff'): 337 raise JSONDecodeError(""Unexpected UTF-8 BOM (decode using utf-8-sig)"", 338 s, 0) 339 else: 340 if not isinstance(s, (bytes, bytearray)): 341 -> raise TypeError(f'the JSON object must be str, bytes or bytearray, ' 342 f'not {s.__class__.__name__}') 343 s = s.decode(detect_encoding(s), 'surrogatepass') 344 345 if ""encoding"" in kw: 346 import warnings (Pdb) bytes (Pdb) locals()['s'] Undefined (Pdb) type(locals()['s']) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'", https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694552681,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694552681,MDEyOklzc3VlQ29tbWVudDY5NDU1MjY4MQ==,9599,simonw,2020-09-17T23:25:54Z,2020-09-17T23:25:54Z,MEMBER,"This is the template fragment it's rendering: ```html+jinja

Tweet by @{{ display.screen_name }} ({{ display.user_name }}, {{ ""{:,}"".format(display.followers_count or 0) }} followers) on {{ display.created_at }}

{{ display.full_text }}
{% if display.media_urls and json.loads(display.media_urls) %} {% for url in json.loads(display.media_urls) %} {% endfor %} {% endif %}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'", https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694552393,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694552393,MDEyOklzc3VlQ29tbWVudDY5NDU1MjM5Mw==,9599,simonw,2020-09-17T23:25:01Z,2020-09-17T23:25:17Z,MEMBER,"Ran `locals()` In the debugger: `{'range': , 'dict': , 'lipsum': , 'cycler': , 'joiner': , 'namespace': , 'rank': -9.383801886431414, 'rowid': 14297, 'type': 'twitter.db/tweets', 'key': '312658917933076480', 'title': 'Tweet by @chrisstreeter', 'category': 2, 'timestamp': '2013-03-15T20:17:49+00:00', 'search_1': '@simonw are you at pycon? Would love to meet you.', 'display': {'avatar_url': 'https://pbs.twimg.com/profile_images/806275088597204993/38yLHfJi_normal.jpg', 'user_name': 'Chris Streeter', 'screen_name': 'chrisstreeter', 'followers_count': 280, 'tweet_id': 312658917933076480, 'created_at': '2013-03-15T20:17:49+00:00', 'full_text': '@simonw are you at pycon? Would love to meet you.', 'media_urls_2': '[]', 'media_urls': '[]'}, 'json': }`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",703970814,"the JSON object must be str, bytes or bytearray, not 'Undefined'", https://github.com/dogsheep/dogsheep-beta/issues/24#issuecomment-694551646,https://api.github.com/repos/dogsheep/dogsheep-beta/issues/24,694551646,MDEyOklzc3VlQ29tbWVudDY5NDU1MTY0Ng==,9599,simonw,2020-09-17T23:22:48Z,2020-09-17T23:22:48Z,MEMBER,"Looks like its happening in a Jinja fragment template for one of the results: ``` /Users/simon/Dropbox/Development/dogsheep-beta/dogsheep_beta/__init__.py(169)process_results() -> output = compiled.render({**result, **{""json"": json}}) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/asyncsupport.py(71)render() -> return original_render(self, *args, **kwargs) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(1090)render() -> self.environment.handle_exception() /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/environment.py(832)handle_exception() -> reraise(*rewrite_traceback_stack(source=source)) /Users/simon/.local/share/virtualenvs/dogsheep-beta-u_po4Rpj/lib/python3.8/site-packages/jinja2/_compat.py(28)reraise() -> raise value.with_traceback(tb)