issue_comments

That seems to have fixed it. I'd love to get rid of this restore_working_directory hack entirely.

That broke things. Here's how pytest-cov fixed a similar issue: https://github.com/pytest-dev/pytest-cov/commit/7ccb1783bf8290447df58deeb41eae74296a6d9b

See also https://github.com/nedbat/coveragepy/issues/881 and https://github.com/pytest-dev/pytest-cov/issues/306 and https://github.com/nedbat/coveragepy/issues/824

I'm going to try removing that restore_working_directory fixture entirely.

Stack Overflow: https://stackoverflow.com/a/49367679/6083

The answer was that os.chdir() had been set to the deleted directory by accident. The directory was missing, but the error happened (it seems) at the attempt to get it with os.getcwd().

Worth noting that I found this issue myself, and to my knowledge it has not been uncovered by anyone else prior to the patch being released.

I've released fixes in both 0.56.1 and 0.57.

Release notes are in, pushing the release now.

See #1360 for the security fix.

There's also a security fix I need to make, so I'm not going to block this on wrapping up the work on the new Docker image testing from #1344 before putting out this release.

This is great, thank you.

Yeah that's a good point. I avoided making them sub-commands because datasette serve already supports the multitude of other arguments they also need... but actually that was just me being lazy - I can easily share arguments between multiple functions like I do in sqlite-utils itself: https://github.com/simonw/sqlite-utils/blob/d1a372b3006e6cf7d2017b3ddc484bf5c033e45d/sqlite_utils/cli.py#L46

Alternative idea: populate request.scope with a new route_path which is the base-url-stripped version, which we then use for other routing operations.

Got the test to pass by ensuring the tests don't accidentally double-rewrite the path.

Ran into a new problem: ``` @pytest.mark.asyncio @pytest.mark.parametrize( "prefix,expected_path", [(None, "/asgi-scope"), ("/prefix/", "/prefix/asgi-scope")] ) async def test_client_path(datasette, prefix, expected_path): original_base_url = datasette._settings["base_url"] try: if prefix is not None: datasette._settings["base_url"] = prefix response = await datasette.client.get("/asgi-scope") path = response.json()["path"]

      assert path == expected_path

E AssertionError: assert '/asgi-scope' == '/prefix/asgi-scope' E - /prefix/asgi-scope E ? ------- E + /asgi-scope `` That test confirms that messing around with thebase_url` doesn't modify the ASGI scope... but the fix I'm using for this issue DOES modify the ASGI scope.

The question raised here is: should the ASGI scope stay unmodified when base_url is used?

I think it should. It doesn't make sense to obscure the "real" path just to get custom pages to work properly.

Applying this fix worked when I manually tested it: diff base_url = self.ds.setting("base_url") if base_url != "/" and path.startswith(base_url): path = "/" + path[len(base_url) :] + scope = dict(scope, path=path, raw_path=path.encode("utf-8")) request = Request(scope, receive) But... the test I wrote still failed. My hunch is that this is because deep within the test framework requests go through ds.client which may be applying its own changes relevant to base_url:

https://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/utils/testing.py#L139

This looks like the cause: https://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/app.py#L1087-L1092

Note how path is modified... but then we create a new Request() that uses the old scope, which has unmodified scope["path"] - and then the code later on looks at request.scope["path"] when deciding if the request matches:

https://github.com/simonw/datasette/blob/afed51b1e36cf275c39e71c7cb262d6c5bdbaa31/datasette/app.py#L1154-L1155