issue_comments

I think this all comes down to how the ?_extras= mechanism works (see #262), as first hinted at in a30c5b220c15360d575e94b0e67f3255e120b916 (see commit message) when I added this long-forgotten undocumented feature: https://latest.datasette.io/fixtures/attraction_characteristic/2.json?_extras=foreign_key_tables

Extras need to be able to execute additional SQL, since that would solve the problem we have now where the expensive "suggested facets" code runs on all .json output even when its results are not being shown.

OK, https://latest-with-plugins.datasette.io/ is running that now - e.g. https://latest-with-plugins.datasette.io/fixtures/roadside_attractions.json-preview or https://latest-with-plugins.datasette.io/fixtures/compound_three_primary_keys.json-preview

json { "rows": [ { "pk": 1, "name": "The Mystery Spot", "address": "465 Mystery Spot Road, Santa Cruz, CA 95065", "latitude": 37.0167, "longitude": -122.0024 }, { "pk": 2, "name": "Winchester Mystery House", "address": "525 South Winchester Boulevard, San Jose, CA 95128", "latitude": 37.3184, "longitude": -121.9511 }, { "pk": 3, "name": "Burlingame Museum of PEZ Memorabilia", "address": "214 California Drive, Burlingame, CA 94010", "latitude": 37.5793, "longitude": -122.3442 }, { "pk": 4, "name": "Bigfoot Discovery Museum", "address": "5497 Highway 9, Felton, CA 95018", "latitude": 37.0414, "longitude": -122.0725 } ], "total": 4, "next_url": null }

I'll update datasette-json-preview with that now.

Great point about CORS, I hadn't considered that.

I think I'm going to keep the Link: header (added in #1014) because I quite enjoy using it with GitHub and WordPress, but I'm not going to have it be the default way of doing pagination. For the default shape I'm now leaning towards this:

json { "total": 36, "rows": [{"id": 1, "name": "Cleo"}], "next_url": "https://latest-with-plugins.datasette.io/fixtures/facetable.json?_next=5" }

So three keys: total, rows and next_url. Then extra keys can be added using ?_extra= with various named bundles.

I vote against headers. It has a lot of strikes against it: poor discoverability, new developers often don’t know how to use them, makes CORS harder, makes it hard to use eg with JQ, needs ad hoc specification for each bit of metadata, etc.

The only advantage of headers is that you don’t need to do .rows, but that’s actually good as a data validation step anyway—if .rows is missing assume there’s an error and do your error handling path instead of parsing the rest.