issue_comments

I really like the idea of making this effectively the same thing as the fully documented, stable JSON API that comes as part of 1.0. If you want to know what will be available to your templates, consult the API documentation.

This will probably happen as part of turning this into an officially documented API that serves the template context for the homepage: - #1510

Built a research prototype: ```diff diff --git a/datasette/app.py b/datasette/app.py index 5c8101a..5cd3e63 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -1,6 +1,7 @@ import asyncio import asgi_csrf import collections +import contextlib import datetime import functools import glob @@ -1490,3 +1491,11 @@ class DatasetteClient: return await client.request( method, self._fix(path, avoid_path_rewrites), kwargs ) + + @contextlib.asynccontextmanager + async def stream(self, method, path, kwargs): + async with httpx.AsyncClient(app=self.app) as client: + print("async with as client") + async with client.stream(method, self._fix(path), **kwargs) as response: + print("async with client.stream about to yield response") + yield response diff --git a/datasette/cli.py b/datasette/cli.py index 3c6e1b2..3025ead 100644 --- a/datasette/cli.py +++ b/datasette/cli.py @@ -585,11 +585,19 @@ def serve( asyncio.get_event_loop().run_until_complete(check_databases(ds))

 if get:

• client = TestClient(ds)
• response = client.get(get)
• click.echo(response.text)
• exit_code = 0 if response.status == 200 else 1
• sys.exit(exit_code) +
• async def _run_get():
• print("_run_get")
• async with ds.client.stream("GET", get) as response:
• print("Got response:", response)
• async for chunk in response.aiter_bytes(chunk_size=1024):
• print(" chunk")
• sys.stdout.buffer.write(chunk)
• sys.stdout.buffer.flush()
• exit_code = 0 if response.status_code == 200 else 1
• sys.exit(exit_code) +

• asyncio.get_event_loop().run_until_complete(_run_get()) return

  # Start the server But for some reason it didn't appear to stream out the response - it would print this out: % datasette covid.db --get '/covid/ny_times_us_counties.csv?_size=10&_stream=on' _run_get async with as client ``` And then hang. I would expect it to start printing out chunks of CSV data here, but instead it looks like it waited for everything to be generated before returning anything to the console.

No idea why. I dropped this for the moment.

Maybe it's because supports_table_xinfo() creates a brand new in-memory SQLite connection every time you call it?

https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/sqlite.py#L22-L35

Actually no, I'm caching that already:

https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/sqlite.py#L12-L19

That means taking on these issues:

• https://github.com/simonw/datasette/issues/1101
• https://github.com/simonw/datasette/issues/1096
• https://github.com/simonw/datasette/issues/1062

I think the way to get rid of most of the remaining complexity in DataView is to refactor how CSV stuff works - pulling it in line with other export factors and extracting the streaming mechanism. Opening a fresh issue for that.

Alternative idea for specifying types: accept a Python expression, then use Python type literal syntax. For example:

sqlite-utils insert-files gifs.db images *.gif \ -c path -c md5 -c last_modified:mtime \ -a file_type '"gif"'

Where -a indicates an additional column.

https://github.com/simonw/datasette/releases/tag/0.61a0

https://github.com/simonw/datasette/compare/0.60.2...main

I'm going to release this as a 0.61 alpha so I can more easily depend on it from datasette-hashed-urls.

That demo is now live:

• https://latest.datasette.io/alternative-route
• https://latest.datasette.io/alternative-route/attraction_characteristic

Building this plugin instantly revealed that all of the links - on the homepage and the database page and so on - are incorrect: ```python from datasette import hookimpl

@hookimpl def startup(datasette): db = datasette.get_database("fixtures2") db.route = "alternative-route" ```

I'm going to add a fixtures2.db database which has that as the name but alternative-route as the route. I'll set that up using a custom plugin in the plugins/ folder that gets deployed by https://github.com/simonw/datasette/blob/main/.github/workflows/deploy-latest.yml

I'd like to have a live demo of this up on latest.datasette.io too.