I experimented with autocompleting the actual schema in https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25, but it would need some work (current problems with it listed in the commit message there)

https://github.com/Sphinxxxx/cm-resize isn't compatible with 6. There's a suggestion to try using CSS resize in https://discuss.codemirror.net/t/resizing-codemirror-6/3265/2

extraKeys is done - Shift+Enter is added in the helper function, and it appears that the Tab behavior now defaults to what the Tab: false setting was doing (allowing it to escape to the form)

Should also minify the bundled output

I have no particular point, I just want to say being around for the 1.0 release of Datasette seems historic and legendary to witness.

If the update succeeds it will return {"ok": true}.

For consistency with /db/table/-/insert you can pass "return": true and it will return a "row" key with the now-updated full row.

I'm also not going to implement "alter": true yet (which would add any missing columns based on the update) - I'll hold that off for a later feature.

I'm going to use the error format I've been experimenting with here: - #1875

json { "type": "https://example.net/validation-error", "title": "Your request is not valid.", "errors": [ { "detail": "must be a positive integer", "pointer": "#/age" }, { "detail": "must be 'green', 'red' or 'blue'", "pointer": "#/profile/color" } ] } I'm not quite ready to commit to a type URL though, so I'll leave that to be solved later should I fully embrace that RFC.

The initial design I'm going to implement will look like this:

POST /db/table/1/-/update Authorization: Bearer xxx Content-Type: application/json json { "update": { "name": "New name" } } Any fields that are not yet columns will return an error.

Should it enforce types, in as much as an integer column should have a JSON integer passed to it, or should it allow strings containing valid integers?

I'm going to allow strings, mainly as a workaround for the fact that JavaScript integers have a maximum size.

One slight concern: https://latest.datasette.io/ will increasingly reflect a version that isn't the most recent production release.

I might setup https://stable.datasette.io/ as a demo instance of the most recent non-alpha release to compensate for that.

I'll do this after the 1.0a0 release: - #1708

Datasette 1.0 alpha 1

This alpha release is a preview of Datasette 1.0.

Datasette 1.0 marks a significant milestone in the project: it is the point from which various aspects of Datasette can be considered "stable", in that code developed against them should expect not to be broken by future releases in the 1.x series.

This will hold true until the next major version release, Datasette 2.0 - which we hope to hold off releasing for as long as possible.

The following Datasette components should be be considered stable after 1.0:

• The plugin API. Plugins developed against 1.0 should continue to work unmodified throughout the 1.x series.
• The JSON API. Code written that interacts with Datasette's default JSON web API should continue to work.
• The template context. If you build custom templates against Datasette your custom pages should continue to work.

Note that none of these components will cease to introduce new features. New plugin hooks, new JSON APIs and new template context variables can be introduced without breaking existing code.

Since this alpha release previews features that will be frozen for 1.0, please test this thoroughly against your existing Datasette projects.

You can install the alpha using:

pip install datasette==1.0a0

JSON API changes

The most significant changes introduced in this new alpha concern Datasette's JSON API.

The default JSON returned by the /database/table.json endpoint has changed. It now returns an object with two keys: rows - which contains a list of objects representing the rows in the table or query, and more containing a boolean that shows if there are more rows or if this object contains them all.

json { "rows": [{ "id": 1, "name": "Name 1" }, { "id": 2, "name": "Name 2" }], "more": false } [ Initially I thought about going with next_url, which would be null if you have reached the last page of records. Maybe that would be better? But since next_url cannot be provided on query pages, should this be part of the default format at all? ]

Use ?_extra= to retrieve extra fields

The default format can be expanded using one or more ?_extra= parameters. This takes names of extra keys you would like to include. These can be comma-separated or ?_extra= can be applied multiple times.

For example:

/database/table.json?_extra=total

This adds a "total": 124 field to the returned JSON.

[ Question: if you do ?_facet=foo then do you still need to do ?_extra=facets - I think not? ]

Annoying: Mobile Safari doesn't seem to support separate labels and values. I should probably disable this feature on that browser, at least for foreign key facets (for the moment).

Looks like I can fix that like so: html <datalist id="datalist-_city_id"> <option label="San Francisco" value="1"></option> <option label="Los Angeles" value="2"></option> <option label="Detroit" value="3"></option> <option label="Memnonia" value="4"></option> </datalist>

Spotted a bug with this on https://latest.datasette.io/fixtures/facetable?_facet=_city_id - the _city_id column is a foreign key, so you need to type 1 or 2 - but the autocomplete list shows the full text names for the cities.

Demo now live here: https://congress-legislators.datasettes.com/legislators/legislator_terms?_facet=party - select party and start typing.

Wrote a TIL about <datalist>: https://til.simonwillison.net/html/datalist

I implemented this as part of /db/-/create.

https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table-from-example-data

That prototype actually works really well! I'm going to add that to table.js.

Here's a prototype: ```javascript function createDataLists() { var facetResults = document.querySelectorAll(".facet-results [data-column]"); Array.from(facetResults).forEach(function (facetResult) { // Use link text from all links in the facet result var linkTexts = Array.from( facetResult.querySelectorAll("li:not(.facet-truncated) a") ).map(function (link) { return link.textContent; }); // Create a datalist element var datalist = document.createElement("datalist"); datalist.id = "datalist-" + facetResult.dataset.column; // Create an option element for each link text linkTexts.forEach(function (linkText) { var option = document.createElement("option"); option.value = linkText; datalist.appendChild(option); }); // Add the datalist to the facet result facetResult.appendChild(datalist); }); } createDataLists();

// When any select with name=_filter_column changes, update the datalist document.body.addEventListener("change", function (event) { if (event.target.name === "_filter_column") { event.target .closest(".filter-row") .querySelector(".filter-value") .setAttribute("list", "datalist-" + event.target.value); } }); ```

This finds the right links on the page:

document.querySelectorAll('.facet-results [data-column] li:not(.facet-truncated) a')

This could start out as a purely JavaScript enhancement for pages that already figured out the available values through faceting, like you suggested.

I tried this out on https://congress-legislators.datasettes.com/legislators/legislator_terms for the party column - here's the demo:

I made this work by dropping the following HTML into the page in the browser DevTools: ```html <datalist id="party">

Oh interesting... this doesn't even need to be attached to the visible faceting feature, necessarily: Datasette could try to detect when a column has a limited number of options (which the faceting code handles already) and could turn those into an auto-complete interface.

There's actually a native HTML element for this these days: the <datalist> https://developer.mozilla.org/en-US/docs/Web/HTML/Element/datalist

Documentation:

• https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table
• https://docs.datasette.io/en/1.0-dev/json_api.html#creating-a-table-from-example-data

Wrote a TIL about how I wrote some of those tests with Copilot: https://til.simonwillison.net/gpt3/writing-test-with-copilot

Datasette usage comments for its 5th anniversary celebration:

I use Datasette and related tools for a Cosmology Researcher Talks database app project, which is described in the github Readme

The app hosted on the Google Cloud Run service also uses other Datasette-related tools developed by Simon - datasette-render-markdown, csvs-to-sqlite, datasette-template-sql, and datasette-block-robots. This is one of two apps used for querying the talks database, each has it pros/cons as described in the github Readme.

At present, over 170 different sites that host cosmology talks are scraped to collect new talks for import into the sqlite database. The shot-scraper and sqlite-utils tools are a major help for this.

I also use the Mastodon API to get my favorites, toots, and boosts into a local database so I can do searches on the data. This was done on Twitter and was then extended to the Mastodon data. Again, sqlite-utils is an important tool for this.

Rough initial prototype: ```diff diff --git a/datasette/views/table.py b/datasette/views/table.py index 8b987221..518ac578 100644 --- a/datasette/views/table.py +++ b/datasette/views/table.py @@ -1103,19 +1103,30 @@ class TableInsertView(BaseView): except json.JSONDecodeError as e: return _errors(["Invalid JSON: {}".format(e)]) if not isinstance(data, dict): - return _errors(["JSON must be a dictionary"]) + return _errors([{"detail": "JSON must be a dictionary", "pointer": "#/"}]) keys = data.keys()

     # keys must contain "row" or "rows"
     if "row" not in keys and "rows" not in keys:
         return _errors(['JSON must have one or other of "row" or "rows"'])
     rows = []

• was_single_row = False if "row" in keys: if "rows" in keys:
• return _errors(['Cannot use "row" and "rows" at the same time'])
• return _errors(
• [
• {
• "detail": 'Cannot use "row" and "rows" at the same time',
• "pointer": "#/row",
• }
• ]
• )
• was_single_row = True row = data["row"] if not isinstance(row, dict):
• return _errors(['"row" must be a dictionary'])
• return _errors(
• [{"detail": '"row" must be a dictionary', "pointer": "#/row"}]
• ) rows = [row] data["return"] = True else: @@ -1152,9 +1163,12 @@ class TableInsertView(BaseView): invalid_columns = set(row.keys()) - columns if invalid_columns: errors.append(
• "Row {} has invalid columns: {}".format(
• i, ", ".join(sorted(invalid_columns))
• )
• {
• "detail": "Invalid columns: {}".format(
• ", ".join(sorted(invalid_columns))
• ),
• "pointer": "#/blah/",
• } ) if errors: return _errors(errors) ```

Worth noting this bit in RFC 7807:

The fictional problem type here defines the "errors" extension, an array that describes the details of each validation error. Each member is an object containing "detail" to describe the issue, and "pointer" to locate the problem within the request's content using a JSON Pointer [JSON-POINTER].

So the list of "errors" with JSON Pointer isn't technically part of the spec, it's an imaginary extension.

It fits what I need to do though, so I'm inclined to stick with it anyway.