https://docs.datasette.io/en/stable/changelog.html#v0-54

Last thing to do: write up the annotated version of these release notes, assign it a URL on my blog and link to it from the release notes here so I can publish them simultaneously.

Issues to reference in the commit message: #509, #1091, #1150, #1151, #1166, #1167, #1178, #1181, #1182, #1184, #1185, #1186, #1187, #1194, #1198

OK, release notes are ready to merge from that branch. I'll ship the release in the morning, to give me time to write the accompanying annotated release notes.

The two big ticket items are <script type="module"> support and the new _internal mechanism.

The big stuff:

• Database(memory_name=) for shared in-memory databases, closes #1151
• The _internal database - #1150
• script type=module support, closes #1186 , #1187
• Improved design for the .add_database() method 8919f99c2f7f245aca7f94bd53d5ac9d04aa42b5 - which means databases with the same stem can now be opened, #509
• Adopted Prettier #1166

Smaller:

• force_https_urls on for publish cloudrun, refs #1178
• Fixed bug in example nginx config, refs #1091
• Shrunk ecosystem docs in favour of datasette.io, closes #1182
• request.full_path property, closes #1184
• Better PRAGMA error message, closes #1185
• publish heroku now uses python-3.8.7
• Plugin testing documentation on using pytest-httpx Closes #1198
• Contributing docs for Black and Prettier, closes #1167
• All ?_ parameters now copied to hidden form fields, closes #1194
• Fixed bug loading database called 'test-database (1).sqlite' - Closes #1181.

Changes: https://github.com/simonw/datasette/compare/0.53...a5ede3cdd455e2bb1a1fb2f4e1b5a9855caf5179

... actually not going to include this in 0.54, I need to write a couple of plugins myself using it before I even make it available in preview.

Also: should the view name really be the same for both of these pages?

• https://latest.datasette.io/fixtures?sql=select+*+from+facetable
• https://latest.datasette.io/fixtures/neighborhood_search

Where one of them is a canned query and the other is an arbitrary query?

Here's a diff showing how far I got before I abandoned this particular effort:

```diff diff --git a/datasette/views/base.py b/datasette/views/base.py index a21b929..04e4aa9 100644 --- a/datasette/views/base.py +++ b/datasette/views/base.py @@ -120,7 +120,7 @@ class BaseView: handler = getattr(self, request.method.lower(), None) return await handler(request, args, *kwargs)

• async def render(self, templates, request, context=None):
• async def render(self, templates, request, context=None, view_name=None): context = context or {} template = self.ds.jinja_env.select_template(templates) template_context = { @@ -135,7 +135,7 @@ class BaseView: } return Response.html( await self.ds.render_template(
• template, template_context, request=request, view_name=self.name
• template, template_context, request=request, view_name=view_name ) )

@@ -155,7 +155,7 @@ class BaseView:

class DataView(BaseView): - name = "" + view_name = "no-view-name" re_named_parameter = re.compile(":([a-zA-Z0-9_]+)")

 async def options(self, request, *args, **kwargs):

@@ -414,6 +414,10 @@ class DataView(BaseView): args["table"] = urllib.parse.unquote_plus(args["table"]) return _format, args

• async def get_view_name(self, request, database, hash, **kwargs):

• Defaults to self.view_name, but can be over-ridden by subclasses

• return self.view_name + async def view_get(self, request, database, hash, correct_hash_provided, **kwargs): _format, kwargs = await self.get_format(request, database, kwargs)

@@ -424,6 +428,8 @@ class DataView(BaseView): # HTML views default to expanding all foreign key labels kwargs["default_labels"] = True

• view_name = await self.get_view_name(request, database, hash, **kwargs) + extra_template_data = {} start = time.perf_counter() status_code = 200 @@ -489,7 +495,7 @@ class DataView(BaseView): database=database, table=data.get("table"), request=request,
• view_name=self.name,
• view_name=view_name, # These will be deprecated in Datasette 1.0: args=request.args, data=data, @@ -533,7 +539,7 @@ class DataView(BaseView): database=database, table=data.get("table"), request=request,
• view_name=self.name,
• view_name=view_name, ) it_can_render = await await_me_maybe(it_can_render) if it_can_render: @@ -565,7 +571,7 @@ class DataView(BaseView): } if "metadata" not in context: context["metadata"] = self.ds.metadata
• r = await self.render(templates, request=request, context=context)
• r = await self.render(templates, request=request, context=context, view_name=view_name) r.status = status_code

   ttl = request.args.get("_ttl", None)
  

  diff --git a/datasette/views/database.py b/datasette/views/database.py index f6fd579..e425213 100644 --- a/datasette/views/database.py +++ b/datasette/views/database.py @@ -23,7 +23,11 @@ from .base import DatasetteError, DataView

class DatabaseView(DataView): - name = "database" + async def get_view_name(self, request, db_name, table_and_format): + if request.args.get("sql"): + return "query" + else: + return "database"

 async def data(self, request, database, hash, default_labels=False, _size=None):
     await self.check_permissions(

@@ -145,7 +149,7 @@ class DatabaseView(DataView):

class DatabaseDownload(DataView): - name = "database_download" + view_name = "database_download"

 async def view_get(self, request, database, hash, correct_hash_present, **kwargs):
     await self.check_permissions(

diff --git a/datasette/views/index.py b/datasette/views/index.py index b6b8cbe..d750e3d 100644 --- a/datasette/views/index.py +++ b/datasette/views/index.py @@ -16,7 +16,7 @@ COUNT_DB_SIZE_LIMIT = 100 * 1024 * 1024

class IndexView(BaseView): - name = "index" + view_name = "index"

 async def get(self, request, as_format):
     await self.check_permission(request, "view-instance")

diff --git a/datasette/views/special.py b/datasette/views/special.py index 9750dd0..dbd1e00 100644 --- a/datasette/views/special.py +++ b/datasette/views/special.py @@ -6,7 +6,7 @@ import secrets

class JsonDataView(BaseView): - name = "json_data" + view_name = "json_data"

 def __init__(self, datasette, filename, data_callback, needs_request=False):
     self.ds = datasette

@@ -42,7 +42,7 @@ class JsonDataView(BaseView):

class PatternPortfolioView(BaseView): - name = "patterns" + view_name = "patterns"

 async def get(self, request):
     await self.check_permission(request, "view-instance")

@@ -50,7 +50,7 @@ class PatternPortfolioView(BaseView):

class AuthTokenView(BaseView): - name = "auth_token" + view_name = "auth_token"

 async def get(self, request):
     token = request.args.get("token") or ""

@@ -68,7 +68,7 @@ class AuthTokenView(BaseView):

class LogoutView(BaseView): - name = "logout" + view_name = "logout"

 async def get(self, request):
     if not request.actor:

@@ -87,7 +87,7 @@ class LogoutView(BaseView):

class PermissionsDebugView(BaseView): - name = "permissions_debug" + view_name = "permissions_debug"

 async def get(self, request):
     await self.check_permission(request, "view-instance")

@@ -102,7 +102,7 @@ class PermissionsDebugView(BaseView):

class AllowDebugView(BaseView): - name = "allow_debug" + view_name = "allow_debug"

 async def get(self, request):
     errors = []

@@ -136,7 +136,7 @@ class AllowDebugView(BaseView):

class MessagesDebugView(BaseView): - name = "messages_debug" + view_name = "messages_debug"

 async def get(self, request):
     await self.check_permission(request, "view-instance")

diff --git a/datasette/views/table.py b/datasette/views/table.py index 0a3504b..45d298a 100644 --- a/datasette/views/table.py +++ b/datasette/views/table.py @@ -257,7 +257,16 @@ class RowTableShared(DataView):

class TableView(RowTableShared): - name = "table" + view_name = "table" + + async def get_view_name(self, request, db_name, table_and_format): + canned_query = await self.ds.get_canned_query( + db_name, table_and_format, request.actor + ) + if canned_query: + return "query" + else: + return "table"

 async def post(self, request, db_name, table_and_format):
     # Handle POST to a canned query

@@ -923,7 +932,7 @@ async def _sql_params_pks(db, table, pk_values):

class RowView(RowTableShared): - name = "row" + view_name = "row"

 async def data(self, request, database, hash, table, pk_path, default_labels=False):
     await self.check_permissions(

diff --git a/tests/test_plugins.py b/tests/test_plugins.py index 715c7c1..7ce2b1b 100644 --- a/tests/test_plugins.py +++ b/tests/test_plugins.py @@ -252,7 +252,7 @@ def test_plugin_config_file(app_client): }, ), ( - "/fixtures/", + "/fixtures", { "template": "database.html", "database": "fixtures", @@ -285,6 +285,38 @@ def test_plugin_config_file(app_client): ], }, ), + ( + "/fixtures?sql=select+1+as+one", + { + "template": "query.html", + "database": "fixtures", + "table": None, + "config": {"depth": "database"}, + "view_name": "query", + "request_path": "/fixtures", + "added": 15, + "columns": [ + "one", + ], + }, + ), + ( + "/fixtures/neighborhood_search", + { + "template": "query.html", + "database": "fixtures", + "table": None, + "config": {"depth": "database"}, + "view_name": "query", + "request_path": "/fixtures/neighborhood_search", + "added": 15, + "columns": [ + "neighborhood", + "name", + "state", + ], + }, + ), ], ) def test_hook_extra_body_script(app_client, path, expected_extra_body_script): ```

This has proved surprisingly difficult to implement, due to the weird way the QueryView is actually called. The class itself isn't used like other view classes - instead, the .data() methods of both DatabaseView and TableView dispatch out to QueryView.data() when they need to:

https://github.com/simonw/datasette/blob/07e163561592c743e4117f72102fcd350a600909/datasette/views/table.py#L259-L270

https://github.com/simonw/datasette/blob/07e163561592c743e4117f72102fcd350a600909/datasette/views/table.py#L290-L294

https://github.com/simonw/datasette/blob/07e163561592c743e4117f72102fcd350a600909/datasette/views/database.py#L39-L44

It turns out this is a bad pattern because it makes changes like this one WAY harder than they should be.

I think I should clean this up as part of #878.

Writing the tests will be a bit tricky since we need to confirm that the include_table_top(datasette, database, actor, table) arguments were all passed correctly but the only thing we get back from the plugin is a list of templates. Maybe encode those values into the template names somehow?

This is joined with #1191 now, which I've bumped from 0.54.

More work can happen in the PR: #1204

I've got a good prototype working now, but I'm dropping this from the Datasette 0.54 milestone because it requires a bunch of additional work to make sure it is really well tested and documented.

I'm going to copy across anything starting with an underscore.

New documentation section here (I documented Black as well): https://docs.datasette.io/en/latest/contributing.html#code-formatting

Thanks @benpickles, I just merged that in. I'll use it in the documentation.

# To check code is conformant
npm run prettier -- --check
# To fix it if it isn't
npm run fix

I'm going to ship a version of this in Datasette 0.54 with a warning that the interface should be considered unstable (see #1202) so that we can start trying this out.

Relevant existing tests: https://github.com/simonw/datasette/blob/461670a0b87efa953141b449a9a261919864ceb3/tests/test_utils.py#L365-L398

The challenge here is figuring out what the original path, without the .format, actually was - while taking into account that Datasette has a special case for tables that themselves end in a .something.

The path_with_format() function nearly does what we need here:

https://github.com/simonw/datasette/blob/b6a7b58fa01af0cd5a5e94bd17d686d283a46819/datasette/utils/init.py#L710-L729

It can be called with replace_format="csv" to REMOVE the .csv format and replace it with something else.

Problem is, we want to use it to get rid of the format entirely.

We could update path_with_format() to accept format='' to mean "remove the format entirely", but it's a bit messy. It may be better to reconsider the design of path_with_format() and related utility functions entirely.

I can combine this with #987 - each of these areas of the page can be wrapped in a <div> with a class that matches the name of the plugin hook, that way JavaScript plugins can append their content in the same place as Python plugins.

https://docs.datasette.io/en/latest/internals.html#the-internal-database

I'm going to use the existing .. warning:: pattern for this.