issue_comments

In the meantime I can catch these errors by changing the test to run each path twice, once with and once without the prefix. This should accurately simulate how Apache is working here.

I think what's happening here is Apache is actually making a request to /fixtures rather than making a request to /prefix/fixtures - and Datasette is replying to requests on both the prefixed and the non-prefixed paths.

This is pretty confusing! I think Datasette should ONLY reply to /prefix/fixtures instead and return a 404 for /fixtures - this would make things a whole lot easier to debug.

But shipping that change could break existing deployments. Maybe that should be a breaking change for 1.0.

https://docs.datasette.io/en/stable/deploying.html#apache-proxy-configuration says I should use ProxyPreserveHost on.

Figured it out! The test is not an accurate recreation of what is happening, because it doesn't simulate a request with a path of /fixtures that has been redirected by the proxy to /prefix/fixtures.

Still not clear why the tests pass but the live example fails.

Modified my Dockerfile to do this:

RUN pip install https://github.com/simonw/datasette/archive/ff0dd4da38d48c2fa9250ecf336002c9ed724e36.zip

And now the request in that debug ?_context=1 looks like this: "request": "<asgi.Request method=\"GET\" url=\"http://localhost:9000/fixtures?sql=select+*+from+compound_three_primary_keys+limit+1&_context=1\">" That explains the bug - that request doesn't maintain the original path prefix of http://localhost:5000/foo/bar/fixtures?sql= (also it's been rewritten to localhost:9000 instead of localhost:5000).

In the ?_context= debug view the request looks like this: "request": "<datasette.utils.asgi.Request object at 0x7faf9fe06200>", I'm going to add a repr() to it such that it's a bit more useful.

The implementations of path_with_removed_args and path_with_format:

https://github.com/simonw/datasette/blob/85849935292e500ab7a99f8fe0f9546e903baad3/datasette/utils/init.py#L228-L254

https://github.com/simonw/datasette/blob/85849935292e500ab7a99f8fe0f9546e903baad3/datasette/utils/init.py#L710-L729

Here's the code that generates edit_sql_url correctly: https://github.com/simonw/datasette/blob/85849935292e500ab7a99f8fe0f9546e903baad3/datasette/views/database.py#L416-L420

And here's the code for show_hide_link: https://github.com/simonw/datasette/blob/85849935292e500ab7a99f8fe0f9546e903baad3/datasette/views/database.py#L432-L433

And for url_csv: https://github.com/simonw/datasette/blob/85849935292e500ab7a99f8fe0f9546e903baad3/datasette/views/base.py#L600-L602

The relevant test is this one: https://github.com/simonw/datasette/blob/30255055150d7bc0affc8156adc18295495020ff/tests/test_html.py#L1608-L1649

I modified that test to add "/fixtures/facetable?sql=select+1" as one of the tested paths, and dropped in an assert False to pause it in the debugger: ``` @pytest.mark.parametrize( "path", [ "/", "/fixtures", "/fixtures/compound_three_primary_keys", "/fixtures/compound_three_primary_keys/a,a,a", "/fixtures/paginated_view", "/fixtures/facetable", "/fixtures?sql=select+1", ], ) def test_base_url_config(app_client_base_url_prefix, path): client = app_client_base_url_prefix response = client.get("/prefix/" + path.lstrip("/")) soup = Soup(response.body, "html.parser") if path == "/fixtures?sql=select+1":

      assert False

E assert False BUT... in the debugger: (Pdb) print(soup) ...

This data as json, testall, testnone, testresponse, CSV

`` Those all have the correct prefix! But that's not what I'm seeing in myDockerfile` reproduction of the issue.

Something very weird is going on here.

I added template_debug in the Dockerfile: datasette fixtures.db --setting template_debug 1 --setting base_url "/foo/bar/" -p 9000 &\n\ And then hit http://localhost:5000/foo/bar/fixtures?sql=select+*+from+compound_three_primary_keys+limit+1&_context=1 to view the template context - and it showed the bug, output edited to just show relevant keys:

json { "edit_sql_url": "/foo/bar/fixtures?sql=select+%2A+from+compound_three_primary_keys+limit+1", "settings": { "force_https_urls": false, "template_debug": true, "trace_debug": false, "base_url": "/foo/bar/" }, "show_hide_link": "/fixtures?sql=select+%2A+from+compound_three_primary_keys+limit+1&_context=1&_hide_sql=1", "show_hide_text": "hide", "show_hide_hidden": "", "renderers": { "json": "/fixtures.json?sql=select+*+from+compound_three_primary_keys+limit+1&_context=1" }, "url_csv": "/fixtures.csv?sql=select+*+from+compound_three_primary_keys+limit+1&_context=1&_size=max", "url_csv_path": "/fixtures.csv", "base_url": "/foo/bar/" } This is so strange. edit_sql_url and base_url are correct, but show_hide_link and url_csv and renderers.json are not.

And it's really strange that the bug doesn't show up in the tests.

Bug confirmed:

I now have a Dockerfile in https://github.com/simonw/datasette/issues/1521#issuecomment-974388295 that I can use to run a local Apache 2 with mod_proxy to investigate this class of bugs!

Having a live demo running on Cloud Run that proxies through Apache and uses base_url would be incredibly useful for replicating and debugging this kind of thing. I wonder how hard it is to run Apache and mod_proxy in the same Docker container as Datasette?

base_url has been a source of so many bugs like this! I often find them quite hard to replicate, likely because I haven't made myself a good Apache mod_proxy testing environment yet.