This needs to take into account the changes made here: - #1439

In the new encoding scheme, - has a special meaning in a table name: https://docs.datasette.io/en/latest/internals.html#dash-encoding

I think ~ is the right character to use to separate a database name from its hash. ~ should be a URL safe character according to Python's implementation of percent-encoding, see comment here: https://github.com/simonw/datasette/blob/c5791156d92615f25696ba93dae5bb2dcc192c98/datasette/utils/init.py#L1146-L1152

So the plugin could check for dbname~hash and react based on that.

I'm going to do a review of how URL routing works at the moment for the various views.

I edited down the full list a bit - these are the most relevant: python add_route(IndexView.as_view(self), r"/(?P<as_format>(\.jsono?)?$)") add_route( DatabaseView.as_view(self), r"/(?P<db_name>[^/]+?)(?P<as_format>" + renderer_regex + r"|.jsono|\.csv)?$", ) add_route( TableView.as_view(self), r"/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?$)", ) add_route( RowView.as_view(self), r"/(?P<db_name>[^/]+)/(?P<table>[^/]+?)/(?P<pk_path>[^/]+?)(?P<as_format>" + renderer_regex + r")?$", )

Also now live on https://datasette.io

I'm going to add this to the main Datasette repo (done) and the datasette.io website too.

Neat, GitHub have a template for this https://github.com/simonw/datasette/community/code-of-conduct/new?template=contributor-covenant

https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-code-of-conduct-to-your-project says this should be called CODE_OF_CONDUCT.md in order for GitHub to pick it up.

I'm going to use the widely adopted Contributor Covenant:

https://www.contributor-covenant.org/version/1/4/code-of-conduct/

My attempts to simplify get_format() keep resulting in errors like this one: File "/Users/simon/Dropbox/Development/datasette/datasette/views/base.py", line 474, in view_get response_or_template_contexts = await self.data( TypeError: TableView.data() missing 1 required positional argument: 'table' I really need to clean this up.

Wow, this code is difficult to follow! Look at this bit inside the get_format() method:

https://github.com/simonw/datasette/blob/bb499942c15c4e2cfa4b6afab8f8debe5948c009/datasette/views/base.py#L469-L478

That's modifying the arguments that were extracted from the path by the routing regular expressions to have table as dash-decoded value! So calling.get_format()` has the side effect of decoding the table names for you. Nasty.

There may be a short-term fix for this: table view could start accepting a ?_sort_sql=SQLfragment parameter, similar to the ?_where= parameter described here: https://docs.datasette.io/en/stable/json_api.html#special-table-arguments

That fragment could then be pre-populated in metadata. Makes me think maybe that ?_where= should be optionally settable in metadata too?

This is currently blocked on the fact that Datasette doesn't have a mechanism for sorting by more than one column: - #197

I found an obscure bug in #1650 which I can fix with this too. The following test should pass: python @pytest.mark.parametrize( "path,expected", ( ( "/fivethirtyeight/twitter-ratio%2Fsenators", "/fivethirtyeight/twitter-2Dratio-2Fsenators", ), ( "/fixtures/table%2Fwith%2Fslashes.csv", "/fixtures/table-2Fwith-2Fslashes-2Ecsv", ), # query string should be preserved ("/foo/bar%2Fbaz?id=5", "/foo/bar-2Fbaz?id=5"), ), ) def test_redirect_percent_encoding_to_dash_encoding(app_client, path, expected): response = app_client.get(path) assert response.status == 302 assert response.headers["location"] == expected It currently fails like this: ```

  assert response.headers["location"] == expected

E AssertionError: assert '/fixtures/table-2Fwith-2Fslashes.csv?_nofacet=1&_nocount=1' == '/fixtures/table-2Fwith-2Fslashes-2Ecsv' E - /fixtures/table-2Fwith-2Fslashes-2Ecsv E + /fixtures/table-2Fwith-2Fslashes.csv?_nofacet=1&_nocount=1 `` Because the logic in thatget_format()` function notices that the table exists, and then weird things happen here: https://github.com/simonw/datasette/blob/1baa030eca375f839f3471237547ab403523e643/datasette/views/base.py#L288-L303

Here's the problem: https://github.com/simonw/datasette/blob/020effe47bf89f35182960a9645f2383a42ebd54/datasette/utils/init.py#L1173-L1175

Which is called here:

https://github.com/simonw/datasette/blob/1baa030eca375f839f3471237547ab403523e643/datasette/views/base.py#L469-L473

So table%2Fwith%2Fslashes ends up decoded as if it was using dash encoding.

The problem seems to be that http://127.0.0.1:8002/fixtures/table%2Fwith%2Fslashes.csv doesn't result in a 404 at all. If it did, it would be redirected.

I didn't need to do any of the fancy regular expression routing stuff after all, since the new dash encoding format avoids using / so a simple [^/]+ can capture the correct segments from the URL.

Same problem here: https://fivethirtyeight.datasettes.com/fivethirtyeight/ahca-2Dpolls%2Fahca_polls should redirect but doesn't.

This doesn't seem to work.

https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv should be redirecting now that this is deployed - which it is, because https://latest.datasette.io/-/versions shows 644d25d1de78a36b105cca479e7b3e4375a6eadc - but I'm not getting that redirect.

Here's the relevant code:

https://github.com/simonw/datasette/blob/1baa030eca375f839f3471237547ab403523e643/datasette/utils/init.py#L753-L772

https://github.com/simonw/datasette/blob/1baa030eca375f839f3471237547ab403523e643/datasette/views/base.py#L451-L479

This is a bit tricky.

I tried this, sending a redirect only if a 404 happens:

```diff diff --git a/datasette/app.py b/datasette/app.py index 8c5480c..420664c 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -1211,6 +1211,10 @@ class DatasetteRouter: return await self.handle_404(request, send)

 async def handle_404(self, request, send, exception=None):

• If path contains % encoding, redirect to dash encoding

• if '%' in request.scope["path"]:
• await asgi_send_redirect(send, request.scope["path"].replace("%", "-"))

• return # If URL has a trailing slash, redirect to URL without it path = request.scope.get( "raw_path", request.scope["path"].encode("utf8") ``` But this URL didn't work:

• http://127.0.0.1:8001/fivethirtyeight/twitter-ratio%2Fsenators

I was expecting that to redirect to this page:

• http://127.0.0.1:8001/fivethirtyeight/twitter-2Dratio-2Fsenators

But instead it took me to another 404:

• http://127.0.0.1:8001/fivethirtyeight/twitter-ratio%2Fsenators

This is because that URL contains both a %-escaped / AND a plain - - which was not escaped in the old system but is escaped in the new system.