issue_comments

Can I come up with a better name than should_suggest?

It's a check that sees if the current query is supported by the renderer plugin. Some options:

• can_render
• supports_query
• is_supported

I like can_render.

Request object is now documented: https://datasette.readthedocs.io/en/latest/internals.html#request-object

I'm going to leave .raw_args in for the moment but deliberately not document it. I'll hope to phase it out entirely at a later date.

What would a better name be?

• .simple_args
• .kv_args
• .pair_args
• .dict_args
• .args_dict

I dislike the last two the least.

It looks like I inherited .raw_args from Sanic - I use it in a few places: https://github.com/search?q=user%3Asimonw+raw_args&type=Code

New documentation can be seen here: https://github.com/simonw/datasette/blob/6d7cb02f00010d3cb4b4bac0460d41277652b80e/docs/internals.rst#request-object

It's inspired me to reconsider the name of the .raw_args property, which isn't particularly clear.

This is the full current implementation of the request object: https://github.com/simonw/datasette/blob/9424687e9e94401438896116898a071702b09d40/datasette/utils/asgi.py#L15-L95

(I wonder if this would be enough to allow really smart plugins to implement ETag/conditional get)

Right now a rendering callback returns the following: body - string or bytes, optional The response body, default empty content_type - string, optional The Content-Type header, default text/plain status_code - integer, optional The HTTP status code, default 200 I'm going to add an optional headers dictionary key, too.

But... https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/fixtures-bda7daa.json doesn't expose that hash: { "database": "fixtures", "size": 258048, "tables": [ { "name": "123_starts_with_digits", Likewise https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/fixtures-bda7daa/complex_foreign_keys.json { "database": "fixtures", "table": "complex_foreign_keys", "is_view": false, "human_description_en": "", "rows": [ [ "1", "1", "2", "1" ] ], And https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/fixtures-bda7daa/complex_foreign_keys/1.json { "database": "fixtures", "table": "complex_foreign_keys", "rows": [

https://latest.datasette.io/.json currently returns:

{ "fixtures": { "name": "fixtures", "hash": "87b3f2c55dfb81ff1452dd306c2623fa5550b90982cfa32bad404c4d8bbedde2", "color": "87b3f2", "path": "/fixtures", "tables_and_views_truncated": [

I published fixtures.db here like this:

datasette publish cloudrun fixtures.db --service datasette-hash-urls --extra-options '--config hash_urls:1'

https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/.json { "fixtures": { "name": "fixtures", "hash": "bda7daa889c23f9a8f06e46d7d280dd423c76275e9593c4c1cad7c53b19032fe", "color": "bda7da", "path": "/fixtures-bda7daa",

(I used GitHub code search to find code using this plugin hook: https://github.com/search?q=register_output_renderer&type=Code )

I'm going to break backwards compatibility directly here, without waiting for Datasette 1.0.

The reason is that https://github.com/russss/datasette-geo hasn't been updated in 13 months so is already broken against current Datasette, and the other two plugins using this hook are owned by me so I can upgrade them myself.

It bothers me that query_name here means the configured name of the canned query, but view_name means the name of the Datasette view class, NOT the name of an associated SQL view. That's in table.

Can I come up with clearer names for these?

Since I'm passing request I won't pass scope - if people want that they can access request.scope.

Rather than pass a request object (hence promoting that object into part of the documented, stable API) I think I'll pass the ASGI scope - that's already a stable, documented standard.

UPDATE: changed my mind since request is used by other plugins too, see #706.

Remove this xfail decorator once they are all tested: https://github.com/simonw/datasette/blob/da87e963bff24e47878a5bc2025c8bfc63d4bc93/tests/test_plugins.py#L23-L28

Actually passing the request object would be OK if I document it see https://github.com/simonw/datasette/issues/706

Closed in da87e963bff24e47878a5bc2025c8bfc63d4bc93

``` $ pytest -k test_plugin_hooks_have_tests -vv ====================================== test session starts ====================================== platform darwin -- Python 3.7.7, pytest-5.2.4, py-1.8.1, pluggy-0.13.1 -- /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/python cachedir: .pytest_cache rootdir: /Users/simon/Dropbox/Development/datasette, inifile: pytest.ini plugins: asyncio-0.10.0 collected 486 items / 475 deselected / 11 selected

tests/test_plugins.py::test_plugin_hooks_have_tests[asgi_wrapper] XPASS [ 9%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_body_script] XPASS [ 18%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_css_urls] XPASS [ 27%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_js_urls] XPASS [ 36%] tests/test_plugins.py::test_plugin_hooks_have_tests[extra_template_vars] XPASS [ 45%] tests/test_plugins.py::test_plugin_hooks_have_tests[prepare_connection] XPASS [ 54%] tests/test_plugins.py::test_plugin_hooks_have_tests[prepare_jinja2_environment] XFAIL [ 63%] tests/test_plugins.py::test_plugin_hooks_have_tests[publish_subcommand] XFAIL [ 72%] tests/test_plugins.py::test_plugin_hooks_have_tests[register_facet_classes] XFAIL [ 81%] tests/test_plugins.py::test_plugin_hooks_have_tests[register_output_renderer] XFAIL [ 90%] tests/test_plugins.py::test_plugin_hooks_have_tests[render_cell] XPASS [100%]

========================= 475 deselected, 4 xfailed, 7 xpassed in 1.70s =========================

Actually I'll land this using @pytest.mark.xfail.

I'll do the work for this in the pull request #772.

This seems to work: ```diff diff --git a/tests/test_plugins.py b/tests/test_plugins.py index 8b6a6b4..e9a40aa 100644 --- a/tests/test_plugins.py +++ b/tests/test_plugins.py @@ -7,7 +7,7 @@ from .fixtures import ( TestClient as _TestClient, ) # noqa from datasette.app import Datasette -from datasette.plugins import get_plugins, DEFAULT_PLUGINS +from datasette.plugins import get_plugins, DEFAULT_PLUGINS, pm from datasette.utils import sqlite3 import base64 import json @@ -20,6 +20,21 @@ import pytest import urllib

+def test_plugin_hooks_have_tests(): + "Every plugin hook should be referenced in this test module" + hooks = [name for name in dir(pm.hook) if not name.startswith("")] + tests_in_this_module = [t for t in globals().keys() if t.startswith('test')] + untested = [] + for hook in hooks: + ok = False + for test in tests_in_this_module: + if hook in test: + ok = True + if not ok: + untested.append(hook) + assert not untested, 'These plugin hooks are missing tests: {}'.format(untested) + + def test_plugins_dir_plugin_prepare_connection(app_client): response = app_client.get( "/fixtures.json?sql=select+convert_units(100%2C+'m'%2C+'ft')" ``` Based on how the documentation unit tests work.

Currently fails with:

AssertionError: These plugin hooks are missing tests:
['prepare_jinja2_environment', 'publish_subcommand', 'register_facet_classes', 'register_output_renderer']

Need to figure out how best to unit test this plugin hook.

Here's the function I just wrote for this: python def call_with_supported_arguments(fn, **kwargs): parameters = inspect.signature(fn).parameters.keys() call_with = [] for parameter in parameters: if parameter not in kwargs: raise TypeError("{} requires parameters {}".format(fn, tuple(parameters))) call_with.append(kwargs[parameter]) return fn(*call_with)

``` In [1]: import inspect

In [2]: def foo(view, sql, inspect): ...: pass ...:

In [3]: inspect.signature(foo)
Out[3]: <Signature (view, sql, inspect)>

In [4]: inspect.signature(foo).parameters
Out[4]: mappingproxy({'view': <Parameter "view">, 'sql': <Parameter "sql">, 'inspect': <Parameter "inspect">})

In [5]: inspect.signature(foo).parameters.keys()
Out[5]: odict_keys(['view', 'sql', 'inspect'])

In [6]: set(inspect.signature(foo).parameters.keys())
Out[6]: {'inspect', 'sql', 'view'} ```

https://docs.python.org/3/library/inspect.html#introspecting-callables-with-the-signature-object

New in version 3.3.

The Signature object represents the call signature of a callable object and its return annotation. To retrieve a Signature object, use the signature() function.

inspect.``signature(callable, *, follow_wrapped=True)

Return a Signature object for the given callable:

```

from inspect import signature def foo(a, , b:int, *kwargs): ... pass

sig = signature(foo)

str(sig) '(a, , b:int, *kwargs)'

str(sig.parameters['b']) 'b:int'

sig.parameters['b'].annotation <class 'int'> ```

I think I need a utility function for "call this function with this dictionary of arguments, but only pass the arguments which are inspected by the function".

The should_suggest callback will take the same arguments: https://github.com/simonw/datasette/issues/770#issuecomment-634880090

I'm going to rename callback to render but continue supporting callback until Datasette 1.0.

The existing render callback takes the following arguments:

args - dictionary The GET parameters of the request

data - dictionary The data to be rendered

view_name - string The name of the view where the renderer is being called. (index, database, table, and row are the most important ones.)

The data argument is a bit of a problem, because it tightly couples plugins to a currently undocumented datastructure within Datasette. Here's how datasette-atom picks that apart for example: https://github.com/simonw/datasette-atom/blob/095941c23c81b70c4787cdeef873c556b573b5fa/datasette_atom/init.py#L15-L66 - it does things like access data["query"]["sql"] to figure out the SQL query that was used.

I'm going to change the design of part of this ticket. I won't break the old data value just yet, but I'll mark it to be deprecated by Datasette 1.0.

I think the only plugins using it right now are my datasette-atom and datasette-ics and @russss's datasette-geo so hopefully changing this won't cause any wider damage.

I'll use #770 for the should_suggest mechanism - this issue is for the extra arguments passed to the rendering callback.

Here's the code that passes a list of renderers to the template:

https://github.com/simonw/datasette/blob/2d099ad9c657d2cab59de91cdb8bfed2da236ef6/datasette/views/base.py#L411-L423

A renderer is currently defined as a two-key dictionary: python @hookimpl def register_output_renderer(datasette): return { 'extension': 'test', 'callback': render_test } I can add a third key, "should_suggest" which is a function that returns True or False for a given query. If that key is missing it is assumed to return True.

One catch: what arguments should be passed to the should_suggest(...) function?

While I'm doing this, another feature I would like is the ability for renderers to opt-in / opt-out of being displayed as options on the page.

https://www.niche-museums.com/browse/museums for example shows a atom link because the datasette-atom plugin is installed... but clicking it will give you a 400 error because the correct columns are not present:

Here's the code that calls the renderers - this needs to be expanded to check for those extra optional arguments:

https://github.com/simonw/datasette/blob/2d099ad9c657d2cab59de91cdb8bfed2da236ef6/datasette/views/base.py#L387-L398

Thanks for helping test this @aborruso!

Dear @simonw thank you for your time, now IT WORKS!!!

I hope that this edit to datasette code is not for an exceptional case (my PC configuration) and that it will be useful to other users.

Thank you again!!

OK, here's a new branch you can try. Install it like this:

pip install https://github.com/simonw/datasette/archive/shutil-backport.zip

If it works for you I'll merge that branch into master.