4,581 rows sorted by updated_at descending

View and edit SQL

author_association

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
720654925 https://github.com/simonw/datasette/issues/1077#issuecomment-720654925 https://api.github.com/repos/simonw/datasette/issues/1077 MDEyOklzc3VlQ29tbWVudDcyMDY1NDkyNQ== simonw 9599 2020-11-02T18:43:25Z 2020-11-02T18:43:25Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
database_actions plugin hook 733829385  
720637322 https://github.com/simonw/datasette/issues/1077#issuecomment-720637322 https://api.github.com/repos/simonw/datasette/issues/1077 MDEyOklzc3VlQ29tbWVudDcyMDYzNzMyMg== simonw 9599 2020-11-02T18:09:17Z 2020-11-02T18:09:17Z OWNER

Here's the table_actions implementation: 2f7731e9e5ff9b324beb5039fbe2be55d704a184

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
database_actions plugin hook 733829385  
720354227 https://github.com/simonw/datasette/issues/838#issuecomment-720354227 https://api.github.com/repos/simonw/datasette/issues/838 MDEyOklzc3VlQ29tbWVudDcyMDM1NDIyNw== psychemedia 82988 2020-11-02T09:33:58Z 2020-11-02T09:33:58Z CONTRIBUTOR

Thanks; just a note that the datasette.urls.static(path) and datasette.urls.static_plugins(plugin_name, path) items both seem to be repeated and appear in the docs twice?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Incorrect URLs when served behind a proxy with base_url set 637395097  
720110298 https://github.com/simonw/datasette/issues/1079#issuecomment-720110298 https://api.github.com/repos/simonw/datasette/issues/1079 MDEyOklzc3VlQ29tbWVudDcyMDExMDI5OA== simonw 9599 2020-11-01T15:58:22Z 2020-11-01T15:58:22Z OWNER

Might try a drop shadow on that menu too.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Handle long breadcrumbs better with new menu 733999615  
720028476 https://github.com/simonw/datasette/issues/782#issuecomment-720028476 https://api.github.com/repos/simonw/datasette/issues/782 MDEyOklzc3VlQ29tbWVudDcyMDAyODQ3Ng== simonw 9599 2020-11-01T05:00:05Z 2020-11-01T05:00:05Z OWNER

This should be the key focus for Datasette 0.52.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Redesign default JSON format in preparation for Datasette 1.0 627794879  
720021029 https://github.com/simonw/datasette/issues/949#issuecomment-720021029 https://api.github.com/repos/simonw/datasette/issues/949 MDEyOklzc3VlQ29tbWVudDcyMDAyMTAyOQ== simonw 9599 2020-11-01T03:29:48Z 2020-11-01T03:29:48Z OWNER

I'm not going to do any more work on this - SQL isn't an auto-complete friendly enough language.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Try out CodeMirror SQL hints 684961449  
720003026 https://github.com/simonw/datasette/issues/1077#issuecomment-720003026 https://api.github.com/repos/simonw/datasette/issues/1077 MDEyOklzc3VlQ29tbWVudDcyMDAwMzAyNg== simonw 9599 2020-10-31T23:48:21Z 2020-10-31T23:50:07Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
database_actions plugin hook 733829385  
719996693 https://github.com/simonw/datasette/issues/1048#issuecomment-719996693 https://api.github.com/repos/simonw/datasette/issues/1048 MDEyOklzc3VlQ29tbWVudDcxOTk5NjY5Mw== simonw 9599 2020-10-31T22:32:09Z 2020-10-31T22:32:09Z OWNER

The row_path part of these really isn't very user friendly, since you need to properly URL-encode the values. The safest way to do so is by calling this obscure, undocumented utility function: https://github.com/simonw/datasette/blob/f0bd2d05f5f7832df4879822afb99d2096c00d48/datasette/utils/__init__.py#L84-L98

This feels like it should be improved before I turn it into a documented API.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Documentation and unit tests for urls.row() urls.row_blob() methods 728905098  
719994676 https://github.com/simonw/datasette/issues/1047#issuecomment-719994676 https://api.github.com/repos/simonw/datasette/issues/1047 MDEyOklzc3VlQ29tbWVudDcxOTk5NDY3Ng== simonw 9599 2020-10-31T22:11:25Z 2020-10-31T22:11:25Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
A new section in the docs about how Datasette handles BLOB columns 728895233  
719988113 https://github.com/simonw/datasette/issues/1027#issuecomment-719988113 https://api.github.com/repos/simonw/datasette/issues/1027 MDEyOklzc3VlQ29tbWVudDcxOTk4ODExMw== simonw 9599 2020-10-31T21:03:37Z 2020-10-31T21:03:37Z OWNER

On my Mac, I run:

datasette . -p 8009 --config base_url:/datasette-prefix/

Then I edited /usr/local/etc/httpd/httpd.conf and add this section:

LoadModule proxy_module lib/httpd/modules/mod_proxy.so
LoadModule proxy_http_module lib/httpd/modules/mod_proxy_http.so
ProxyPass          /datasette-prefix/    http://localhost:8009/datasette-prefix/

I ran Apache in the foreground like so:

apachectl -X

Now hitting http://localhost:8081/datasette-prefix/fixtures/compound_three_primary_keys worked!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add documentation on serving Datasette behind a proxy using base_url 722758132  
719986922 https://github.com/simonw/datasette/issues/1023#issuecomment-719986922 https://api.github.com/repos/simonw/datasette/issues/1023 MDEyOklzc3VlQ29tbWVudDcxOTk4NjkyMg== simonw 9599 2020-10-31T20:51:01Z 2020-10-31T20:51:01Z OWNER

This should all be working correctly now.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Fix issues relating to base_url 722673818  
719986904 https://github.com/simonw/datasette/issues/838#issuecomment-719986904 https://api.github.com/repos/simonw/datasette/issues/838 MDEyOklzc3VlQ29tbWVudDcxOTk4NjkwNA== simonw 9599 2020-10-31T20:50:41Z 2020-10-31T20:50:41Z OWNER

OK, this should be working now. You can use the datasette.urls.static_plugins() method to generate the correct URLs in the extra_css_urls plugin hook: https://docs.datasette.io/en/latest/internals.html#datasette-urls

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Incorrect URLs when served behind a proxy with base_url set 637395097  
719986800 https://github.com/simonw/datasette/issues/1041#issuecomment-719986800 https://api.github.com/repos/simonw/datasette/issues/1041 MDEyOklzc3VlQ29tbWVudDcxOTk4NjgwMA== simonw 9599 2020-10-31T20:49:28Z 2020-10-31T20:49:28Z OWNER

Implemented in a4ca26a2659d21779adf625183061d8879954c15

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
extra_js_urls and extra_css_urls should respect base_url setting 727627923  
719986698 https://github.com/simonw/datasette/issues/1072#issuecomment-719986698 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTk4NjY5OA== simonw 9599 2020-10-31T20:48:17Z 2020-10-31T20:48:17Z OWNER

Here's the datasette-edit-templates plugin WIP I had before removing the hook: https://github.com/simonw/datasette-edit-templates/tree/82855c2612b84bc09c48fca885f831633a0d1552

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719983750 https://github.com/simonw/datasette/issues/1075#issuecomment-719983750 https://api.github.com/repos/simonw/datasette/issues/1075 MDEyOklzc3VlQ29tbWVudDcxOTk4Mzc1MA== simonw 9599 2020-10-31T20:22:29Z 2020-10-31T20:22:29Z OWNER

I bet this is because I'm mucking around with one of those __ methods. I'll try just doing the non-underscore methods instead.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
PrefixedUrlString mechanism broke everything 733796942  
719983565 https://github.com/simonw/datasette/issues/1075#issuecomment-719983565 https://api.github.com/repos/simonw/datasette/issues/1075 MDEyOklzc3VlQ29tbWVudDcxOTk4MzU2NQ== simonw 9599 2020-10-31T20:21:03Z 2020-10-31T20:21:03Z OWNER

Here's the output of dir(str):

['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isascii', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
PrefixedUrlString mechanism broke everything 733796942  
719983484 https://github.com/simonw/datasette/issues/1075#issuecomment-719983484 https://api.github.com/repos/simonw/datasette/issues/1075 MDEyOklzc3VlQ29tbWVudDcxOTk4MzQ4NA== simonw 9599 2020-10-31T20:20:28Z 2020-10-31T20:20:28Z OWNER

It looks like this is specific to the way PrefixedUrlString is built.

(Pdb) class Weird(str): pass
(Pdb) isinstance(Weird('bob'), collections.abc.Awaitable)
False

So subclassing strings doesn't trigger this bug, but something about PrefixedUrlString causes the problem.

Here's the current PrefixedUrlString implementation:

https://github.com/simonw/datasette/blob/bf18b9ba175a7b25fb8b765847397dd6efb8bb7b/datasette/utils/__init__.py#L1015-L1035

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
PrefixedUrlString mechanism broke everything 733796942  
719983240 https://github.com/simonw/datasette/issues/1075#issuecomment-719983240 https://api.github.com/repos/simonw/datasette/issues/1075 MDEyOklzc3VlQ29tbWVudDcxOTk4MzI0MA== simonw 9599 2020-10-31T20:18:49Z 2020-10-31T20:18:49Z OWNER

Here's the core problem:

(Pdb) isinstance('bob', collections.abc.Awaitable)
False
(Pdb) isinstance(PrefixedUrlString('bob'), collections.abc.Awaitable)
*** TypeError: issubclass() arg 1 must be a class

For some reason isinstance() does not like being handed an instance of PrefixedUrlString.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
PrefixedUrlString mechanism broke everything 733796942  
719981173 https://github.com/simonw/datasette/issues/1075#issuecomment-719981173 https://api.github.com/repos/simonw/datasette/issues/1075 MDEyOklzc3VlQ29tbWVudDcxOTk4MTE3Mw== simonw 9599 2020-10-31T20:02:30Z 2020-10-31T20:03:45Z OWNER

I wonder how Jinja's Markup() class works? It uses https://pypi.org/project/MarkupSafe/

It's a subclass of str, defined here: https://github.com/pallets/markupsafe/blob/master/src/markupsafe/__init__.py

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
PrefixedUrlString mechanism broke everything 733796942  
719980742 https://github.com/simonw/datasette/issues/1075#issuecomment-719980742 https://api.github.com/repos/simonw/datasette/issues/1075 MDEyOklzc3VlQ29tbWVudDcxOTk4MDc0Mg== simonw 9599 2020-10-31T19:58:57Z 2020-10-31T19:58:57Z OWNER

Sample traceback:

    <link rel="stylesheet" href="{{ urls.static('app.css') }}?{{ app_css_hash }}">
/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/jinja2/asyncsupport.py:173: in auto_await
    if inspect.isawaitable(value):
/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/inspect.py:226: in isawaitable
    isinstance(object, collections.abc.Awaitable))
/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/abc.py:139: in __instancecheck__
    return _abc_instancecheck(cls, instance)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'collections.abc.Awaitable'>
subclass = <bound method PrefixedUrlString.__getattribute__.<locals>.method of '/-/static/app.css'>

    def __subclasscheck__(cls, subclass):
        """Override for issubclass(subclass, cls)."""
>       return _abc_subclasscheck(cls, subclass)
E       TypeError: issubclass() arg 1 must be a class

This is within Jinja. It looks like Jinja really doesn't like methods that return non-string objects like PrefixedUrlString.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
PrefixedUrlString mechanism broke everything 733796942  
719977864 https://github.com/simonw/datasette/issues/1074#issuecomment-719977864 https://api.github.com/repos/simonw/datasette/issues/1074 MDEyOklzc3VlQ29tbWVudDcxOTk3Nzg2NA== simonw 9599 2020-10-31T19:35:01Z 2020-10-31T19:35:01Z OWNER

These plugins were not designed to be actually hosted online, so they do some nasty things like linking to the made-up plugin-example.com domain. I should fix that.

https://user-images.githubusercontent.com/9599/97788348-7b60d800-1b75-11eb-8519-a58e52108841.png">

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
latest.datasette.io should include plugins from fixtures 733768037  
719966176 https://github.com/simonw/datasette/issues/1067#issuecomment-719966176 https://api.github.com/repos/simonw/datasette/issues/1067 MDEyOklzc3VlQ29tbWVudDcxOTk2NjE3Ng== simonw 9599 2020-10-31T17:51:31Z 2020-10-31T17:51:31Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu on view pages, not on query pages 732905360  
719965426 https://github.com/simonw/datasette/issues/1074#issuecomment-719965426 https://api.github.com/repos/simonw/datasette/issues/1074 MDEyOklzc3VlQ29tbWVudDcxOTk2NTQyNg== simonw 9599 2020-10-31T17:45:00Z 2020-10-31T17:45:00Z OWNER

This is working. Go to https://latest.datasette.io/login-as-root and click the button, then visit this page to see extra content added by plugins: https://latest.datasette.io/fixtures/compound_three_primary_keys

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
latest.datasette.io should include plugins from fixtures 733768037  
719963074 https://github.com/simonw/datasette/issues/1074#issuecomment-719963074 https://api.github.com/repos/simonw/datasette/issues/1074 MDEyOklzc3VlQ29tbWVudDcxOTk2MzA3NA== simonw 9599 2020-10-31T17:23:48Z 2020-10-31T17:23:48Z OWNER

Needs a way to login as root, seeing as several plugins only show extra content if the user is logged in as root.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
latest.datasette.io should include plugins from fixtures 733768037  
719961701 https://github.com/simonw/datasette/issues/1067#issuecomment-719961701 https://api.github.com/repos/simonw/datasette/issues/1067 MDEyOklzc3VlQ29tbWVudDcxOTk2MTcwMQ== simonw 9599 2020-10-31T17:11:59Z 2020-10-31T17:11:59Z OWNER

It bothers me that these aren't visible in any public demos. Maybe latest.datasette.io should include the my_plugins.py and my_plugins2.py plugins?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu on view pages, not on query pages 732905360  
719959754 https://github.com/simonw/datasette/issues/1026#issuecomment-719959754 https://api.github.com/repos/simonw/datasette/issues/1026 MDEyOklzc3VlQ29tbWVudDcxOTk1OTc1NA== simonw 9599 2020-10-31T16:56:35Z 2020-10-31T16:56:35Z OWNER

1041 can also benefit from the string subclass that shows that base_url has been added.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
How should datasette.client interact with base_url 722738988  
719959419 https://github.com/simonw/datasette/issues/1067#issuecomment-719959419 https://api.github.com/repos/simonw/datasette/issues/1067 MDEyOklzc3VlQ29tbWVudDcxOTk1OTQxOQ== simonw 9599 2020-10-31T16:53:42Z 2020-10-31T16:53:42Z OWNER

For the 0.51 release I'm going to add tests that show this works on view pages. I won't implement it for query pages.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu on view pages, not on query pages 732905360  
719956184 https://github.com/simonw/datasette/issues/1067#issuecomment-719956184 https://api.github.com/repos/simonw/datasette/issues/1067 MDEyOklzc3VlQ29tbWVudDcxOTk1NjE4NA== simonw 9599 2020-10-31T16:26:09Z 2020-10-31T16:26:09Z OWNER

Should the hook provide an indication that it's running on a different type of page? I think yes for queries. Not sure about views - they behave very much like tables, and the plugin can always introspect to see if something is a view if it needs to.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu on view pages, not on query pages 732905360  
719955724 https://github.com/simonw/datasette/issues/1070#issuecomment-719955724 https://api.github.com/repos/simonw/datasette/issues/1070 MDEyOklzc3VlQ29tbWVudDcxOTk1NTcyNA== simonw 9599 2020-10-31T16:22:45Z 2020-10-31T16:22:45Z OWNER

I've removed this plugin hook in #1073.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template() example in documentation showing loading from a database 733390884  
719955491 https://github.com/simonw/datasette/issues/1072#issuecomment-719955491 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTk1NTQ5MQ== simonw 9599 2020-10-31T16:20:58Z 2020-10-31T16:20:58Z OWNER

Here's the proof of concept FunctionLoader that showed me that this wasn't going to work:
```diff
diff --git a/datasette/app.py b/datasette/app.py
index 4b28e71..b076be7 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -21,7 +21,7 @@ from pathlib import Path
from markupsafe import Markup
from itsdangerous import URLSafeSerializer
import jinja2
-from jinja2 import ChoiceLoader, Environment, FileSystemLoader, PrefixLoader
+from jinja2 import ChoiceLoader, Environment, FileSystemLoader, FunctionLoader, PrefixLoader
from jinja2.environment import Template
from jinja2.exceptions import TemplateNotFound
import uvicorn
@@ -300,6 +300,7 @@ class Datasette:
template_paths.append(default_templates)
template_loader = ChoiceLoader(
[
+ FunctionLoader(self._load_template_from_plugins),
FileSystemLoader(template_paths),
# Support {% extends "default:table.html" %}:
PrefixLoader(
@@ -322,6 +323,17 @@ class Datasette:
self._root_token = secrets.token_hex(32)
self.client = DatasetteClient(self)

  • def _load_template_from_plugins(self, template):
  • "If auto reloading is enabled it’s called to check if the template changed"

  • uptodatefunc = lambda: True
  • source = pm.hook.load_template(
  • template=template,
  • datasette=self,
  • )
  • if source is None:
  • return None
  • return source, template, uptodatefunc
    +
    @property
    def urls(self):
    return Urls(self)
    @@ -719,35 +731,7 @@ class Datasette:
    else:
    if isinstance(templates, str):
    templates = [templates]
    -
  • Give plugins first chance at loading the template

  • break_outer = False
  • plugin_template_source = None
  • plugin_template_name = None
  • template_name = None
  • for template_name in templates:
  • if break_outer:
  • break
  • plugin_template_source = pm.hook.load_template(
  • template=template_name,
  • request=request,
  • datasette=self,
  • )
  • plugin_template_source = await await_me_maybe(plugin_template_source)
  • if plugin_template_source:
  • break_outer = True
  • plugin_template_name = template_name
  • break
  • if plugin_template_source is not None:
  • template = self.jinja_env.from_string(plugin_template_source)
  • else:
  • template = self.jinja_env.select_template(templates)
  • for template_name in templates:
  • from_plugin = template_name == plugin_template_name
  • used = from_plugin or template_name == template.name
  • templates_considered.append(
  • {"name": template_name, "used": used, "from_plugin": from_plugin}
  • )
  • template = self.jinja_env.select_template(templates)
    body_scripts = []
    # pylint: disable=no-member
    for extra_script in pm.hook.extra_body_script(
    diff --git a/datasette/hookspecs.py b/datasette/hookspecs.py
    index ca84b35..7804def 100644
    --- a/datasette/hookspecs.py
    +++ b/datasette/hookspecs.py
    @@ -50,7 +50,7 @@ def extra_template_vars(

@hookspec(firstresult=True)
-def load_template(template, request, datasette):
+def load_template(template, datasette):
"Load the specified template, returning the template code as a string"

diff --git a/docs/plugin_hooks.rst b/docs/plugin_hooks.rst
index 3c57b6a..8f2704e 100644
--- a/docs/plugin_hooks.rst
+++ b/docs/plugin_hooks.rst
@@ -273,15 +273,12 @@ Example: `datasette-cluster-map <https://github.com/simonw/datasette-cluster-map

.. _plugin_hook_load_template:

-load_template(template, request, datasette)

+load_template(template, datasette)
+----------------------------------

template - string
The template that is being rendered, e.g. database.html

-request - object or None
- The current HTTP :ref:internals_request. This can be None if the request object is not available.
-
datasette - :ref:internals_datasette
You can use this to access plugin configuration options via datasette.plugin_config(your_plugin_name)
```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719834200 https://github.com/simonw/datasette/issues/1073#issuecomment-719834200 https://api.github.com/repos/simonw/datasette/issues/1073 MDEyOklzc3VlQ29tbWVudDcxOTgzNDIwMA== simonw 9599 2020-10-30T22:52:48Z 2020-10-30T22:52:48Z OWNER

Should mostly be a case of backing out the changes from this commit: 81dea4b07ab2b6f4eaaf248307d2b588472054a1

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Remove load_template plugin hook 733560417  
719833744 https://github.com/simonw/datasette/issues/1072#issuecomment-719833744 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgzMzc0NA== simonw 9599 2020-10-30T22:50:57Z 2020-10-30T22:50:57Z OWNER

Yeah I'm going to remove the load_template plugin hook and see if it's possible to build the edit templates extension against prepare_jinja2_environment instead.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719833070 https://github.com/simonw/datasette/issues/1072#issuecomment-719833070 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgzMzA3MA== simonw 9599 2020-10-30T22:48:04Z 2020-10-30T22:48:04Z OWNER

https://github.com/simonw/datasette/blob/a2a709072059c6b3da365df9a332ca744c2079e9/datasette/app.py#L310-L318

So yeah that plugin hook can probably modify the list of loaders available to the Environment.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719832853 https://github.com/simonw/datasette/issues/1072#issuecomment-719832853 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgzMjg1Mw== simonw 9599 2020-10-30T22:47:12Z 2020-10-30T22:47:12Z OWNER

Maybe I should ditch this hook entirely in favour of the existing prepare_jinja2_environment hook. Could that add new template loaders?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719832651 https://github.com/simonw/datasette/issues/1072#issuecomment-719832651 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgzMjY1MQ== simonw 9599 2020-10-30T22:46:25Z 2020-10-30T22:46:25Z OWNER

I tried using a FunctionLoader and got this error on startup:

  File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 989, in __init__
    for filepath in self.ds.jinja_env.list_templates()
  File "/Users/simon/.local/share/virtualenvs/datasette-edit-templates-agoZyE3x/lib/python3.8/site-packages/jinja2/environment.py", line 810, in list_templates
    names = self.loader.list_templates()
  File "/Users/simon/.local/share/virtualenvs/datasette-edit-templates-agoZyE3x/lib/python3.8/site-packages/jinja2/loaders.py", line 434, in list_templates
    found.update(loader.list_templates())
  File "/Users/simon/.local/share/virtualenvs/datasette-edit-templates-agoZyE3x/lib/python3.8/site-packages/jinja2/loaders.py", line 99, in list_templates
    raise TypeError("this loader cannot iterate over all templates")
TypeError: this loader cannot iterate over all templates

So if I'm going to define a custom Jinja loader I'll need to teach plugins to answer the "list templates" query.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719819331 https://github.com/simonw/datasette/issues/1072#issuecomment-719819331 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxOTMzMQ== simonw 9599 2020-10-30T22:00:43Z 2020-10-30T22:00:43Z OWNER

I'll try getting that to work. If I can't get it to work I'll drop the plugin hook for the moment.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719819234 https://github.com/simonw/datasette/issues/1072#issuecomment-719819234 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxOTIzNA== simonw 9599 2020-10-30T22:00:21Z 2020-10-30T22:00:21Z OWNER

There might be a way to save this. Async template loading can't be supported, but what if you could define a load_template() hook which returned a sync function that returned templates...

Then the datasette-edit-templates plugin could reply to load_template by loading all DB templates into memory and returning a load_template sync function that looked up the values in those already-loaded templates.

It could even maintain an in-memory cache that gets updated when a template is edited.

If I do this, I could remove the ability to return an async function from load_template() but add that in the future should Jinja implement a mechanism for async template loading.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719814279 https://github.com/simonw/datasette/issues/1072#issuecomment-719814279 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxNDI3OQ== simonw 9599 2020-10-30T21:45:33Z 2020-10-30T21:45:33Z OWNER

Sadly I'm going to bump load_template from Datasette 0.51 - I don't think I should block the release on resolving this issue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719813970 https://github.com/simonw/datasette/issues/1072#issuecomment-719813970 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxMzk3MA== simonw 9599 2020-10-30T21:44:40Z 2020-10-30T21:44:40Z OWNER

I'm pretty sure that run_in_executor() workaround won't work. https://github.com/django/asgiref/blob/7becc9daca2628c46af1cb7e46b4c47c1ea27adf/asgiref/sync.py#L83 for example says "You cannot use AsyncToSync in the same thread as an async event loop".

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719813212 https://github.com/simonw/datasette/issues/1072#issuecomment-719813212 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxMzIxMg== simonw 9599 2020-10-30T21:42:35Z 2020-10-30T21:42:35Z OWNER

Filed a feature request here: https://github.com/pallets/jinja/issues/1304

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719811312 https://github.com/simonw/datasette/issues/1072#issuecomment-719811312 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxMTMxMg== simonw 9599 2020-10-30T21:36:49Z 2020-10-30T21:36:49Z OWNER

There's one other option: in datasette-edit-templates I could maybe use asyncio.get_event_loop().run_in_executor(...) to load the templates asynchronously within the Jinja template loader mechanism.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719810533 https://github.com/simonw/datasette/issues/1072#issuecomment-719810533 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxMDUzMw== simonw 9599 2020-10-30T21:34:38Z 2020-10-30T21:34:38Z OWNER

... no wait, my comments above assume that I'm just building the datasette-edit-templates plugin. Does this work as a general solution for all of Datasette? I don't think it does.

This may mean I need to delay the whole feature.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719810023 https://github.com/simonw/datasette/issues/1072#issuecomment-719810023 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgxMDAyMw== simonw 9599 2020-10-30T21:33:06Z 2020-10-30T21:33:06Z OWNER

The ideal solution is for Jinja to offer async template loading. I'll file a feature request, then I'll implement the second option above (async load all templates from the DB before each render).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719809780 https://github.com/simonw/datasette/issues/1072#issuecomment-719809780 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgwOTc4MA== simonw 9599 2020-10-30T21:32:28Z 2020-10-30T21:32:28Z OWNER

Here's an alternative that would definitely work and would be a lot simpler, at the cost of a fair amount of RAM:

  1. Before rendering the template, load ALL of the most-recent-versions of the templates that are stored in the DB. Use those to populate a DictLoader.
  2. Render the template.

This does mean loading template bodies that we won't use. Provided an instance has less than 100 templates I imagine this will work just fine.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719809259 https://github.com/simonw/datasette/issues/1072#issuecomment-719809259 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgwOTI1OQ== simonw 9599 2020-10-30T21:31:10Z 2020-10-30T21:31:10Z OWNER

How can we tell what template Jinja will need to render?

One approach that could work:

  1. Set up a dummy template loader which records the name of the template that was requested
  2. Load the template
  3. Now we know the list of templates that were requested. Async load those
  4. The dummy template loader can now return the ones we have loaded. Load the template again.
  5. Did it request any more templates? If so, load those, and repeat.
  6. Keep on with this loop until a template load (which might even have to be a render) fails to request any templates that we have not yet loaded.
  7. Render the template.

This is GROSS. It feels like a huge waste of CPU, and it could lead to very weird behaviour if any template variables have side effects.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719807502 https://github.com/simonw/datasette/issues/1072#issuecomment-719807502 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgwNzUwMg== simonw 9599 2020-10-30T21:26:49Z 2020-10-30T21:26:49Z OWNER

It looks like Jinja does not have a mechanism for asynchronous template loading - the loader API is synchronous.

One option may be to figure out which templates are needed (including inherited templates and includes) before rendering the template. Then async load those templates from the database into a DictLoader, then pass that DictLoader to Jinja.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719803880 https://github.com/simonw/datasette/issues/1072#issuecomment-719803880 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTgwMzg4MA== simonw 9599 2020-10-30T21:17:11Z 2020-10-30T21:17:11Z OWNER

Example from the Jinja docs: https://jinja.palletsprojects.com/en/2.11.x/api/#jinja2.BaseLoader

from jinja2 import BaseLoader, TemplateNotFound
from os.path import join, exists, getmtime

class MyLoader(BaseLoader):

    def __init__(self, path):
        self.path = path

    def get_source(self, environment, template):
        path = join(self.path, template)
        if not exists(path):
            raise TemplateNotFound(template)
        mtime = getmtime(path)
        with file(path) as f:
            source = f.read().decode('utf-8')
        return source, path, lambda: mtime == getmtime(path)

Also available: jinja2.FunctionLoader(load_func) which lets me pass it a function like this one:

>>> def load_template(name):
...     if name == 'index.html':
...         return '...'
...
>>> loader = FunctionLoader(load_template)

Just one catch: I need to be able to load templates asynchronously, because they live in the database. Let's hope Jinja has a mechanism for that!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719785005 https://github.com/simonw/datasette/issues/1072#issuecomment-719785005 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTc4NTAwNQ== simonw 9599 2020-10-30T20:36:22Z 2020-10-30T20:36:22Z OWNER

It should be easy enough to show a comment that says which original template names were considered, but I may not be able to show which one was actually used (or which ones came from plugins).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719784606 https://github.com/simonw/datasette/issues/1072#issuecomment-719784606 https://api.github.com/repos/simonw/datasette/issues/1072 MDEyOklzc3VlQ29tbWVudDcxOTc4NDYwNg== simonw 9599 2020-10-30T20:35:33Z 2020-10-30T20:35:33Z OWNER

To fix this I think I need to move the load_template implementation into a Jinja template loader.

I'm not sure I'll be able to keep the Templates considered comment working though:

https://github.com/simonw/datasette/blob/a2a709072059c6b3da365df9a332ca744c2079e9/datasette/app.py#L745-L750

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template hook doesn't work for include/extends 733499930  
719777499 https://github.com/simonw/datasette/issues/1071#issuecomment-719777499 https://api.github.com/repos/simonw/datasette/issues/1071 MDEyOklzc3VlQ29tbWVudDcxOTc3NzQ5OQ== simonw 9599 2020-10-30T20:20:01Z 2020-10-30T20:20:01Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Messages should be displayed full width 733485423  
719657478 https://github.com/simonw/datasette/pull/1069#issuecomment-719657478 https://api.github.com/repos/simonw/datasette/issues/1069 MDEyOklzc3VlQ29tbWVudDcxOTY1NzQ3OA== codecov[bot] 22429695 2020-10-30T16:31:21Z 2020-10-30T17:46:36Z NONE

Codecov Report

Merging #1069 into main will increase coverage by 0.01%.
The diff coverage is 95.83%.

@@            Coverage Diff             @@
##             main    #1069      +/-   ##
==========================================
+ Coverage   91.30%   91.32%   +0.01%     
==========================================
  Files          29       29              
  Lines        3736     3756      +20     
==========================================
+ Hits         3411     3430      +19     
- Misses        325      326       +1     
<table> <thead> <tr> <th>Impacted Files</th> <th>Coverage Δ</th> <th></th> </tr> </thead> <tbody> <tr> <td>datasette/views/base.py</td> <td>93.94% <ø> (-0.04%)</td> <td>:arrow_down:</td> </tr> <tr> <td>datasette/app.py</td> <td>96.38% <95.45%> (-0.05%)</td> <td>:arrow_down:</td> </tr> <tr> <td>datasette/hookspecs.py</td> <td>100.00% <100.00%> (ø)</td> <td></td> </tr> </tbody> </table>

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 222f79b...92f3840. Read the comment docs.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template() plugin hook 733303548  
719672967 https://github.com/simonw/datasette/pull/1069#issuecomment-719672967 https://api.github.com/repos/simonw/datasette/issues/1069 MDEyOklzc3VlQ29tbWVudDcxOTY3Mjk2Nw== simonw 9599 2020-10-30T16:58:01Z 2020-10-30T16:58:01Z OWNER

OK, new hook specification is:

@hookspec(firstresult=True)
def load_template(template, request, datasette):
    "Load the specified template, returning the template code as a string"
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template() plugin hook 733303548  
719670714 https://github.com/simonw/datasette/pull/1069#issuecomment-719670714 https://api.github.com/repos/simonw/datasette/issues/1069 MDEyOklzc3VlQ29tbWVudDcxOTY3MDcxNA== simonw 9599 2020-10-30T16:53:56Z 2020-10-30T16:53:56Z OWNER

I'm having second thoughts about the design of the plugin hook.

Consider the following:

            plugin_template_source = pm.hook.load_template(
                template=template_name,
                database=context.get("database"),
                table=context.get("table"),
                columns=context.get("columns"),
                view_name=self.name,
                request=request,
                datasette=self.ds,
            )

It's a bit gross that database, table and columns are pulled out of the context like that. This doesn't make sense for pages that are rendered by plugins, for example.

So maybe for the first release of this plugin hook I should cut it down to just seeing template, request and datasette. I can add the table/view/etc stuff back in later if it turns out to be necessary.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template() plugin hook 733303548  
719666912 https://github.com/simonw/datasette/pull/1069#issuecomment-719666912 https://api.github.com/repos/simonw/datasette/issues/1069 MDEyOklzc3VlQ29tbWVudDcxOTY2NjkxMg== simonw 9599 2020-10-30T16:47:44Z 2020-10-30T16:47:44Z OWNER

Bringing over a comment from #1042:

I'd like to do this all in the datasette.render_template() method to ensure it's available to plugins as well, not just core code that uses the BaseView class.

This code is the problem:

https://github.com/simonw/datasette/blob/d3e9b0aecb6f8e9b2befd9c654ccb7ce852db3e7/datasette/views/base.py#L114-L133

I think I'll fix this by moving the select_templates mechanism into datasette.render_templates().

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template() plugin hook 733303548  
719664530 https://github.com/simonw/datasette/pull/1069#issuecomment-719664530 https://api.github.com/repos/simonw/datasette/issues/1069 MDEyOklzc3VlQ29tbWVudDcxOTY2NDUzMA== simonw 9599 2020-10-30T16:43:40Z 2020-10-30T16:43:40Z OWNER

I should include an example in the documentation that shows loading templates from a database table.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template() plugin hook 733303548  
719640430 https://github.com/simonw/datasette/pull/1069#issuecomment-719640430 https://api.github.com/repos/simonw/datasette/issues/1069 MDEyOklzc3VlQ29tbWVudDcxOTY0MDQzMA== simonw 9599 2020-10-30T16:01:13Z 2020-10-30T16:01:13Z OWNER

Next steps: build a demonstration plugin against this.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
load_template() plugin hook 733303548  
719630745 https://github.com/simonw/datasette/issues/1068#issuecomment-719630745 https://api.github.com/repos/simonw/datasette/issues/1068 MDEyOklzc3VlQ29tbWVudDcxOTYzMDc0NQ== simonw 9599 2020-10-30T15:44:13Z 2020-10-30T15:44:13Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Default menu links should check a real permission  732939921  
719332460 https://github.com/simonw/datasette/issues/1068#issuecomment-719332460 https://api.github.com/repos/simonw/datasette/issues/1068 MDEyOklzc3VlQ29tbWVudDcxOTMzMjQ2MA== simonw 9599 2020-10-30T07:13:10Z 2020-10-30T07:13:10Z OWNER

I mainly want this so I can add that debug menu to my Dogsheep.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Default menu links should check a real permission  732939921  
719331236 https://github.com/simonw/datasette/issues/1068#issuecomment-719331236 https://api.github.com/repos/simonw/datasette/issues/1068 MDEyOklzc3VlQ29tbWVudDcxOTMzMTIzNg== simonw 9599 2020-10-30T07:11:58Z 2020-10-30T07:11:58Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Default menu links should check a real permission  732939921  
719329219 https://github.com/simonw/datasette/issues/1068#issuecomment-719329219 https://api.github.com/repos/simonw/datasette/issues/1068 MDEyOklzc3VlQ29tbWVudDcxOTMyOTIxOQ== simonw 9599 2020-10-30T07:09:59Z 2020-10-30T07:09:59Z OWNER

Permission idea: debug-menu

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Default menu links should check a real permission  732939921  
719328661 https://github.com/simonw/datasette/issues/1068#issuecomment-719328661 https://api.github.com/repos/simonw/datasette/issues/1068 MDEyOklzc3VlQ29tbWVudDcxOTMyODY2MQ== simonw 9599 2020-10-30T07:09:30Z 2020-10-30T07:09:30Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Default menu links should check a real permission  732939921  
719322666 https://github.com/simonw/datasette/issues/1067#issuecomment-719322666 https://api.github.com/repos/simonw/datasette/issues/1067 MDEyOklzc3VlQ29tbWVudDcxOTMyMjY2Ng== simonw 9599 2020-10-30T07:04:02Z 2020-10-30T07:04:02Z OWNER

Maybe rename it to actions_menu and have it work for database, view, table and query pages using different arguments on each.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu on view pages, not on query pages 732905360  
719320948 https://github.com/simonw/datasette/issues/1067#issuecomment-719320948 https://api.github.com/repos/simonw/datasette/issues/1067 MDEyOklzc3VlQ29tbWVudDcxOTMyMDk0OA== simonw 9599 2020-10-30T07:02:37Z 2020-10-30T07:02:37Z OWNER

Yes, this should be possible - no point restricting what plugin authors can do with the feature. Will need to add some extra arguments to the plugin hook for this.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu on view pages, not on query pages 732905360  
719195346 https://github.com/simonw/datasette/issues/690#issuecomment-719195346 https://api.github.com/repos/simonw/datasette/issues/690 MDEyOklzc3VlQ29tbWVudDcxOTE5NTM0Ng== simonw 9599 2020-10-30T05:20:42Z 2020-10-30T05:20:42Z OWNER

I've now added two new plugin hooks: menu_links() and table_actions().

I'm going to close this issue. Further work (on column actions and and database actions) can happen in separate tickets, but I won't include them in Datasette 0.51 since they're much less interesting than table and instance actions.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Mechanism for plugins to add action menu items for various things 573755726  
719194756 https://github.com/simonw/datasette/issues/1066#issuecomment-719194756 https://api.github.com/repos/simonw/datasette/issues/1066 MDEyOklzc3VlQ29tbWVudDcxOTE5NDc1Ng== simonw 9599 2020-10-30T05:18:35Z 2020-10-30T05:18:35Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu plus plugin hook 732859030  
719194619 https://github.com/simonw/datasette/issues/1066#issuecomment-719194619 https://api.github.com/repos/simonw/datasette/issues/1066 MDEyOklzc3VlQ29tbWVudDcxOTE5NDYxOQ== simonw 9599 2020-10-30T05:18:04Z 2020-10-30T05:18:04Z OWNER

The cog only appears if at least one table action has been registered by a plugin. It looks like this:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu plus plugin hook 732859030  
719154646 https://github.com/simonw/datasette/issues/1066#issuecomment-719154646 https://api.github.com/repos/simonw/datasette/issues/1066 MDEyOklzc3VlQ29tbWVudDcxOTE1NDY0Ng== simonw 9599 2020-10-30T03:48:15Z 2020-10-30T03:48:15Z OWNER

This will use a very similar implementation to the navigation menu in #1064 - similar plugin hook and I'll use a <details><summary> to implement it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Table actions menu plus plugin hook 732859030  
719153773 https://github.com/simonw/datasette/pull/1065#issuecomment-719153773 https://api.github.com/repos/simonw/datasette/issues/1065 MDEyOklzc3VlQ29tbWVudDcxOTE1Mzc3Mw== codecov[bot] 22429695 2020-10-30T03:44:57Z 2020-10-30T03:44:57Z NONE

Codecov Report

Merging #1065 into main will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1065      +/-   ##
==========================================
+ Coverage   91.23%   91.27%   +0.03%     
==========================================
  Files          28       29       +1     
  Lines        3710     3724      +14     
==========================================
+ Hits         3385     3399      +14     
  Misses        325      325              
<table> <thead> <tr> <th>Impacted Files</th> <th>Coverage Δ</th> <th></th> </tr> </thead> <tbody> <tr> <td>datasette/plugins.py</td> <td>82.35% <ø> (ø)</td> <td></td> </tr> <tr> <td>datasette/app.py</td> <td>96.42% <100.00%> (+0.03%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/default_menu_links.py</td> <td>100.00% <100.00%> (ø)</td> <td></td> </tr> <tr> <td>datasette/hookspecs.py</td> <td>100.00% <100.00%> (ø)</td> <td></td> </tr> </tbody> </table>

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a861be...5f118b5. Read the comment docs.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Nav menu plus menu_links() hook 732856937  
719117185 https://github.com/simonw/datasette/issues/1064#issuecomment-719117185 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTExNzE4NQ== simonw 9599 2020-10-30T01:35:17Z 2020-10-30T01:35:17Z OWNER

I'm going to go with a list of {"label": ..., "href": ...} as the first iteration of this. The logout link will not be returned as part of the plugin output. A default plugin will provide the debug tools if the user is logged in as root.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719111597 https://github.com/simonw/datasette/issues/1064#issuecomment-719111597 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTExMTU5Nw== simonw 9599 2020-10-30T01:15:05Z 2020-10-30T01:15:05Z OWNER

I'm torn on this one. I think I have a very slight preference for plugins returning structured objects as opposed to HTML. Less likely to regret that choice in the future?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719111373 https://github.com/simonw/datasette/issues/1064#issuecomment-719111373 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTExMTM3Mw== simonw 9599 2020-10-30T01:14:13Z 2020-10-30T01:14:13Z OWNER

Plugins returning HTML makes more sense for some of the other areas that plugins will be able to inject content - e.g. injecting content on the table or row page above the table.

If I'm going to have that as a pattern though it may make sense to use HTML here, since that will be consistent with other places that plugins can inject additional content.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719110808 https://github.com/simonw/datasette/issues/1064#issuecomment-719110808 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTExMDgwOA== simonw 9599 2020-10-30T01:12:09Z 2020-10-30T01:12:19Z OWNER

Or... plugins could return HTML - maybe optionally using helper functions to generate common HTML such that plugins which use the helpers can have their HTML modified in the future.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719110582 https://github.com/simonw/datasette/issues/1064#issuecomment-719110582 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTExMDU4Mg== simonw 9599 2020-10-30T01:11:13Z 2020-10-30T01:11:13Z OWNER

Should plugins be able to add forms like the logout form here, or should they be restricted to adding navigation links?

I can't think of a reason a plugin would need to add a form. The logout form is a special case to protect against logout-csrf attacks.

So I think plugins get to return a list of dictionaries, each with a label and an href:

return [{
  "label": "Upload CSVs",
  "href": datasette.urls.path("/-/upload-csvs")
}]

But... is there an argument for returning headings, to divide up the menu?

I think so. I also like the idea that a default plugin checks for the root user and outputs links to the different debugging tools - maybe those should be wrapped in a section heading.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719109770 https://github.com/simonw/datasette/issues/1064#issuecomment-719109770 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTEwOTc3MA== simonw 9599 2020-10-30T01:08:14Z 2020-10-30T01:08:14Z OWNER

How should the plugin hook work?

Here's the first version of the HTML:

        <div class="nav-menu-inner">
            <ul>
                <li><a href="{{ urls.instance() }}">Home</a></li>
                <li><a href="{{ urls.path('/-/plugins') }}">Installed plugins</a></li>
                <li><a href="{{ urls.path('/-/versions') }}">Software versions</a></li>
                <li><a href="{{ urls.path('/-/metadata') }}">Metadata</a></li>
                {% if show_logout %}
                <form action="{{ urls.logout() }}" method="post">
                    <input type="hidden" name="csrftoken" value="{{ csrftoken() }}">
                    <button class="button-as-link">Log out</button>
                </form>{% endif %}
            </ul>
        </div>
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719106174 https://github.com/simonw/datasette/issues/1064#issuecomment-719106174 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTEwNjE3NA== simonw 9599 2020-10-30T00:55:12Z 2020-10-30T00:55:12Z OWNER

So what should go in this menu?

If the user is logged in as root, I'll link to the various debug pages.

If they're not logged in at all I don't think the menu should appear.

If they are logged in as anyone, it should display to give them access to the "log out" button.

Plugins can add links to it. If those plugins add links, the menu will display.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719105641 https://github.com/simonw/datasette/issues/1064#issuecomment-719105641 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTEwNTY0MQ== simonw 9599 2020-10-30T00:53:00Z 2020-10-30T00:53:00Z OWNER

Tips for making this accessible: https://css-tricks.com/accessible-svgs/

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719104883 https://github.com/simonw/datasette/issues/1064#issuecomment-719104883 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTEwNDg4Mw== simonw 9599 2020-10-30T00:50:01Z 2020-10-30T00:52:29Z OWNER

Here's what the prototype looks like so far:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719105197 https://github.com/simonw/datasette/issues/1064#issuecomment-719105197 https://api.github.com/repos/simonw/datasette/issues/1064 MDEyOklzc3VlQ29tbWVudDcxOTEwNTE5Nw== simonw 9599 2020-10-30T00:51:16Z 2020-10-30T00:51:16Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Navigation menu plus plugin hook 732798913  
719094027 https://github.com/simonw/datasette/issues/1034#issuecomment-719094027 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxOTA5NDAyNw== simonw 9599 2020-10-30T00:11:17Z 2020-10-30T00:11:17Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
719066706 https://github.com/simonw/datasette/issues/1063#issuecomment-719066706 https://api.github.com/repos/simonw/datasette/issues/1063 MDEyOklzc3VlQ29tbWVudDcxOTA2NjcwNg== simonw 9599 2020-10-29T22:46:28Z 2020-10-29T22:46:28Z OWNER

I'm not going to do the base64 thing unless someone asks for it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.csv should link to .blob downloads 732685643  
719053669 https://github.com/simonw/datasette/issues/1051#issuecomment-719053669 https://api.github.com/repos/simonw/datasette/issues/1051 MDEyOklzc3VlQ29tbWVudDcxOTA1MzY2OQ== simonw 9599 2020-10-29T22:12:16Z 2020-10-29T22:12:16Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better display of binary data on arbitrary query results page 729096595  
719050754 https://github.com/simonw/datasette/issues/1034#issuecomment-719050754 https://api.github.com/repos/simonw/datasette/issues/1034 MDEyOklzc3VlQ29tbWVudDcxOTA1MDc1NA== simonw 9599 2020-10-29T22:04:52Z 2020-10-29T22:04:52Z OWNER

I'm going to link to. the new .blob representation using the new ?_blob_hash=xxx argument to ensure that the content served is the expected binary blob.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better way of representing binary data in .csv output 725184645  
719050390 https://github.com/simonw/datasette/issues/1063#issuecomment-719050390 https://api.github.com/repos/simonw/datasette/issues/1063 MDEyOklzc3VlQ29tbWVudDcxOTA1MDM5MA== simonw 9599 2020-10-29T22:04:00Z 2020-10-29T22:04:00Z OWNER

This will close #1034.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.csv should link to .blob downloads 732685643  
719049115 https://github.com/simonw/datasette/pull/1061#issuecomment-719049115 https://api.github.com/repos/simonw/datasette/issues/1061 MDEyOklzc3VlQ29tbWVudDcxOTA0OTExNQ== codecov[bot] 22429695 2020-10-29T22:00:57Z 2020-10-29T22:00:57Z NONE

Codecov Report

Merging #1061 into main will increase coverage by 0.07%.
The diff coverage is 96.87%.

@@            Coverage Diff             @@
##             main    #1061      +/-   ##
==========================================
+ Coverage   91.13%   91.20%   +0.07%     
==========================================
  Files          27       28       +1     
  Lines        3677     3697      +20     
==========================================
+ Hits         3351     3372      +21     
+ Misses        326      325       -1     
<table> <thead> <tr> <th>Impacted Files</th> <th>Coverage Δ</th> <th></th> </tr> </thead> <tbody> <tr> <td>datasette/plugins.py</td> <td>82.35% <ø> (ø)</td> <td></td> </tr> <tr> <td>datasette/views/base.py</td> <td>93.77% <0.00%> (ø)</td> <td></td> </tr> <tr> <td>datasette/app.py</td> <td>96.38% <100.00%> (+0.15%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/blob_renderer.py</td> <td>100.00% <100.00%> (ø)</td> <td></td> </tr> <tr> <td>datasette/utils/asgi.py</td> <td>92.13% <100.00%> (+0.17%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/views/database.py</td> <td>97.04% <100.00%> (+0.07%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/views/table.py</td> <td>95.86% <100.00%> (-0.22%)</td> <td>:arrow_down:</td> </tr> </tbody> </table>

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d6f9ff7...1196d08. Read the comment docs.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.blob output renderer 732634375  
719042601 https://github.com/simonw/datasette/pull/1061#issuecomment-719042601 https://api.github.com/repos/simonw/datasette/issues/1061 MDEyOklzc3VlQ29tbWVudDcxOTA0MjYwMQ== simonw 9599 2020-10-29T21:45:35Z 2020-10-29T21:50:42Z OWNER

Moving the CSV work to a separate issue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.blob output renderer 732634375  
719043108 https://github.com/simonw/datasette/issues/1063#issuecomment-719043108 https://api.github.com/repos/simonw/datasette/issues/1063 MDEyOklzc3VlQ29tbWVudDcxOTA0MzEwOA== simonw 9599 2020-10-29T21:46:48Z 2020-10-29T21:46:48Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.csv should link to .blob downloads 732685643  
719035336 https://github.com/simonw/datasette/pull/1061#issuecomment-719035336 https://api.github.com/repos/simonw/datasette/issues/1061 MDEyOklzc3VlQ29tbWVudDcxOTAzNTMzNg== simonw 9599 2020-10-29T21:29:29Z 2020-10-29T21:29:29Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.blob output renderer 732634375  
719033013 https://github.com/simonw/datasette/pull/1061#issuecomment-719033013 https://api.github.com/repos/simonw/datasette/issues/1061 MDEyOklzc3VlQ29tbWVudDcxOTAzMzAxMw== simonw 9599 2020-10-29T21:27:14Z 2020-10-29T21:27:14Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.blob output renderer 732634375  
719031901 https://github.com/simonw/datasette/issues/1062#issuecomment-719031901 https://api.github.com/repos/simonw/datasette/issues/1062 MDEyOklzc3VlQ29tbWVudDcxOTAzMTkwMQ== simonw 9599 2020-10-29T21:25:54Z 2020-10-29T21:25:54Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows 732674148  
719021514 https://github.com/simonw/datasette/issues/1050#issuecomment-719021514 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxOTAyMTUxNA== simonw 9599 2020-10-29T21:05:08Z 2020-10-29T21:05:08Z OWNER

Idea: what if Datasette had a custom SQLite function that could be used to generate URLs to the row-level BLOB download for a value? Then custom SQL query authors could use that function to link to the relevant content.

This could be expanded to exposing other datasette.urls functionality as well.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
719001701 https://github.com/simonw/datasette/issues/1050#issuecomment-719001701 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxOTAwMTcwMQ== simonw 9599 2020-10-29T20:26:44Z 2020-10-29T20:26:44Z OWNER

I'll do the rest of the work on this in the pull request #1061.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718989895 https://github.com/simonw/datasette/issues/1050#issuecomment-718989895 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODk4OTg5NQ== simonw 9599 2020-10-29T20:04:15Z 2020-10-29T20:04:15Z OWNER

I'll use hashlib.sha256 for these hashes.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718987852 https://github.com/simonw/datasette/issues/1050#issuecomment-718987852 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODk4Nzg1Mg== simonw 9599 2020-10-29T20:00:32Z 2020-10-29T20:00:32Z OWNER

The reason I like the ?_blob_hash= solution is that it feels really misleading to provide a link to "download this binary" which could conceivably download some other data.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718980944 https://github.com/simonw/datasette/issues/1050#issuecomment-718980944 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODk4MDk0NA== simonw 9599 2020-10-29T19:46:19Z 2020-10-29T19:46:19Z OWNER

Had an idea in https://github.com/simonw/datasette/issues/1051#issuecomment-718980659

OK, alternative idea. The .blob output renderer from #1050 gets to see multiple rows at once.

For an arbitrary SQL query, how about if I link to this?

/db.blob?sql=...&_blob_column=data&_blob_hash=bc4c24181ed3ce666

Then the output renderer loops through all of the data results that are available to it and, if one of them hashes to that value, serves up that data?

If no matches are found it can show an error message telling you that the link has expired (presumably because the underlying database has changed since the link was generated).

I think this might be the best solution to the problem.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718980659 https://github.com/simonw/datasette/issues/1051#issuecomment-718980659 https://api.github.com/repos/simonw/datasette/issues/1051 MDEyOklzc3VlQ29tbWVudDcxODk4MDY1OQ== simonw 9599 2020-10-29T19:45:42Z 2020-10-29T19:45:42Z OWNER

OK, alternative idea. The .blob output renderer from #1050 gets to see multiple rows at once.

For an arbitrary SQL query, how about if I link to this?

/db.blob?sql=...&_blob_column=data&_blob_hash=bc4c24181ed3ce666

Then the output renderer loops through all of the data results that are available to it and, if one of them hashes to that value, serves up that data?

If no matches are found it can show an error message telling you that the link has expired (presumably because the underlying database has changed since the link was generated).

I think this might be the best solution to the problem.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better display of binary data on arbitrary query results page 729096595  
718976679 https://github.com/simonw/datasette/issues/1053#issuecomment-718976679 https://api.github.com/repos/simonw/datasette/issues/1053 MDEyOklzc3VlQ29tbWVudDcxODk3NjY3OQ== simonw 9599 2020-10-29T19:37:57Z 2020-10-29T19:37:57Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Document recommendations for plugin authors to design URLs 729604838  
718528252 https://github.com/simonw/datasette/pull/1049#issuecomment-718528252 https://api.github.com/repos/simonw/datasette/issues/1049 MDEyOklzc3VlQ29tbWVudDcxODUyODI1Mg== psychemedia 82988 2020-10-29T09:20:34Z 2020-10-29T09:20:34Z CONTRIBUTOR

That workaround is probably fine. I was trying to work out whether there might be other situations where a pre-external package load might be useful but couldn't offhand bring any other examples to mind. The static plugins option also looks interesting.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add template block prior to extra URL loaders 729017519  
718346019 https://github.com/simonw/datasette/issues/1050#issuecomment-718346019 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODM0NjAxOQ== simonw 9599 2020-10-29T04:05:07Z 2020-10-29T04:05:07Z OWNER

Yes, confirmed - this is a bug where if the BLOB column contains a null you get a nasty exception if you try to download it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718342036 https://github.com/simonw/datasette/issues/1050#issuecomment-718342036 https://api.github.com/repos/simonw/datasette/issues/1050 MDEyOklzc3VlQ29tbWVudDcxODM0MjAzNg== simonw 9599 2020-10-29T03:49:57Z 2020-10-29T03:49:57Z OWNER

@thadk from that error it looks like the problem may have been that you had a BLOB column containing a null value? If so that's definitely a bug, I'll fix that.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Switch to .blob render extension for BLOB downloads 729057388  
718340847 https://github.com/simonw/datasette/pull/1049#issuecomment-718340847 https://api.github.com/repos/simonw/datasette/issues/1049 MDEyOklzc3VlQ29tbWVudDcxODM0MDg0Nw== simonw 9599 2020-10-29T03:45:47Z 2020-10-29T03:48:26Z OWNER

thebe is the first time I've seen a library that requires you to set up some global JavaScript configuration before loading the script itself.

I'm hesitant to add an extra template block just to cover that one case since it's such a rare pattern. But it's important that thebelab can be used with Datasette.

Would this pattern work for you instead?

{% block extra_head %}
<script type="text/x-thebe-config">
  {
    requestKernel: true,
    binderOptions: {
      repo: "binder-examples/requirements",
    },
  }
</script>
<script src="https://unpkg.com/thebelab@latest/lib/index.js"></script>
{% endblock %}
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Add template block prior to extra URL loaders 729017519  

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);