issue_comments

Maybe instead of implementing datasette.get() and datasette.post() and datasette.request() and datasette.stream() I could instead have a nested object called datasette.client which is a preconfigured AsyncClient instance.

python response = await datasette.client.get("/") Or perhaps this should be a method in case I ever need to be able to await it: python response = await (await datasette.client()).get("/") This is a bit cosmetically ugly though, I'd rather avoid that if possible.

Maybe I could get this working by returning an object from .client() which provides a await obj.get() method: python response = await datasette.client().get("/") I don't think there's any benefit to that over await datasette.client.get() though.

Should I instantiate a single Client and reuse it for all internal requests, or can I instantiate a new Client for each request?

https://www.python-httpx.org/advanced/#why-use-a-client says that the main benefit of a Client instance is HTTP connection pooling - which isn't an issue for these internal requests since they won't be using the HTTP protocol at all, they'll be calling the ASGI application directly.

So I'm leaning towards instantiating a fresh client for every internal request. I'll run a microbenchmark to check that this doesn't have any unpleasant performance implications.

I think I can use async with httpx.AsyncClient(base_url="http://localhost/") as client: to ensure I don't need to use http://localhost/ on every call.

I could solve streaming using something like this: python async with datasette.stream("GET", "/fixtures/compound_three_primary_keys.csv?_stream=on&_size=max") as response: async for chunk in response.aiter_bytes(): print(chunk) Which would be a wrapper around AsyncClient.stream(method, url, ...) from https://www.python-httpx.org/async/#streaming-responses

What if datasette.get() was an alias for httpx.get(), pre-configured to route to the correct application? And with some sugar that added http://localhost/ to the beginning of the path if it was missing?

This would make httpx a dependency of core Datasette, which I think is OK.

It would also solve the return type problem: I would return whatever httpx returns.

I'm tempted to create a await datasette.request() method which can take any HTTP verb - then have datasette.get() and datasette.post() as thin wrappers around it.

Maybe these methods become the way most Datasette tests are written, replacing the existing TestClient mechanism?

So what should I do about streaming responses?

I could deliberately ignore them - through an exception if you attempt to run await datasette.get(...) against a streaming URL.

I could load the entire response into memory and return it as a wrapped object.

I could support some kind of asynchronous iterator mechanism. This would be pretty elegant if I could decide the right syntax for it - it would allow plugins to take advantage of other internal URLs that return streaming content without needing to load that content entirely into memory in order to process it.

I'm not going to mess around with formats - you'll get back the exact response that a web client would receive.

Question: what should the response object look like? e.g. if you do:

response = await datasette.get("/db/table.json")

What should response be?

I could reuse the Datasette Response class from datasette.utils.asgi. This would work well for regular responses which just have a status code, some headers and a response body. It wouldn't be great for streaming responses though such as you get back from ?_stream=1 CSV exports.