issue_comments

Should we empty out the fixture schema to avoid fixture autocomplete showing up on live databases in the interim, or are you planning to tackle #1897 shortly?

Alright with https://github.com/simonw/datasette/pull/1893/commits/f254be4b38936e95e7a7f25866e7c6b0520db96f we should be getting autocomplete on fixture data. Give that a test and see what you think

I can push up a commit that uses the static fixtures schema for testing, but given that the query used to generate it is authed we would still need some work to make that work on live data, right? Ideally it could come down to db and query views directly to avoid waiting on an extra xhr and managing that state change.On Nov 16, 2022, at 2:16 PM, Simon Willison @.***> wrote: Honestly I'm not too bothered if table names with weird characters don't work correctly here - I care about those in the Datasette fixtures.db database because Datasette aims to support ANY valid SQLite database, so I need stuff in the test suite that includes weird edge cases like this. But I would hope very few people actually create tables with spaces in their names, so it's not a huge concern to me if autocompletion doesn't work properly for those.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

I think the table completion still has some quirks to work out. Something like

schema: { "[123_starts_with_digits]": ["content"], }

Seems to work alright, although it will append it after any other numbers you've started typing - so you end up with select * from 12[123_starts_with_digits] if you typed "12" to get the completion to appear. This might just be an issue with numeric names, I haven't tested it in a lot of detail.

You can do

searchable: [ { label: "name with . and spaces", apply: "[name with . and spaces]", }, "pk", "text1", "text2", ],

Which is pretty neat and will show the non-escaped string but complete to the escaped one. You can't easily do that with the table names themselves (you can pass a tables array like so https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/src/sql.ts#L121 but it will overwrite the columns from the schema ).

It's buggy enough (bad output for these unusual table names) that I'd suggest that work gets moved into a follow up to the upgrade to 6. That would give space to sort out how to deliver that to the view directly, figure out where name escaping should happen, and have overall testing to uncover bugs and fix papercuts before enabling it.

Alright, added Cmd+Enter to submit (Ctrl+Enter on Windows as well bc of using Meta-Enter on codemirror). We can make that MacOS only by changing the combo to Cmd+Enter specifically but I think it's probably fine to have both.

Or I guess you could return only the escaped table name and then we could derive the unescaped from the client side (removing the outer [] when present)

Nice. And is it possible to include another field which is an escaped table name (only when necessary) - i.e. [123_starts_with_digits]. Or is that easy enough to derive on the client? I'm thinking we'd map those to Completion objects so that CM would show the non escaped text but complete to escaped.

UI issue I see on the autocomplete popup with overlapping icon & text. Screenshot's from Firefox, it seems even a little more pronounced on Safari

I checked and if I empty out app.css the bug goes away, so there's some kind of inheritance issue there. It's hard to debug bc the autocomplete popup goes away on blur (i.e. when trying to inspect it in devtools), but at least it's narrowed down a bit.

For escaped table names it looks like we could pass a Completion object (https://codemirror.net/docs/ref/#autocomplete) instead of a string which would allow the non escaped name to be a label and then the escaped name to actually complete in the editor, which might help with some of the funkiness I was seeing w/ completion

With

patch diff --git a/datasette/templates/_codemirror_foot.html b/datasette/templates/_codemirror_foot.html index ed709b3..74fe18e 100644 --- a/datasette/templates/_codemirror_foot.html +++ b/datasette/templates/_codemirror_foot.html @@ -7,7 +7,11 @@ sqlFormat.hidden = false; } if (sqlInput) { - var editor = (window.editor = cm.editorFromTextArea(sqlInput)); + var editor = (window.editor = cm.editorFromTextArea(sqlInput, { + schema: { + compound_three_primary_keys: ["pk1", "pk2", "pk3", "content"], + }, + }));

we get table autocompletion and column completion if you name the table in the query (see screencast). I do see bugs with escaped table names like "'123_starts_with_digits'": ["col1", "col2"] or "[123_starts_with_digits]": ["col1", "col2"] where it doesn't seem to pick up the column names though. I think it needs some further testing and debugging.

https://user-images.githubusercontent.com/95570/202238521-e613b4e2-ba92-4418-9068-fc022edaee93.mp4

Ha, nice idea! Updating the dialect with that list.

I'm thinking of also adding count to the list since that's a common thing people would want to autocomplete. I notice BQ console highlights count in the same manner as other keywords like select as well.

I added a commit to make our own dialect at https://github.com/simonw/datasette/pull/1893/commits/e273fc8ed5341bdf0b622e722d761bd2acc30a90. Pulled in the full list of keywords from https://www.sqlite.org/lang_keywords.html but haven't gone through and pruned it to only include common select keywords. @simonw you'll have better knowledge than me on that - do you want to take a first shot at narrowing that down to the set that people will be using in the editor?

Have you ever seen CodeMirror correctly auto-completing columns? I'm not entirely sure I believe that the feature works anywhere else.

I was thinking of the BigQuery console, like

But they must be doing something pretty custom & appears to be using Monaco anyway. I suspect some kind of lower level autocomplete integration could make this work, but if the table completion is a good-enough starting point I think it's not too hard. The main issue is that we don't pass the relevant table data down to QueryView.

UI issue I see on the autocomplete popup with overlapping icon & text. Screenshot's from Firefox, it seems even a little more pronounced on Safari

The resize handle doesn't appear on Mobile Safari on iPhone - I don't think that particularly matters though.

The textarea does get a weird border around it when focused on iPhone though.

The default focus styles appear to be

.c1.cm-editor.cm-focused { outline: 1px dotted #212121; }

Which I also see on desktop. Would be nice to changed to whatever the default UA textarea styles are to blend in better but I wouldn't recommend removing it entirely - just to keep the visual indication that the element is focused. Maybe followup material to have a theming pass

If you can get a version of this working with table and column autocompletion just using a static JavaScript object in the source code with the right tables and columns, I'm happy to take on the work of turning that static object into something that Datasette includes in the page itself with all of the correct values.

This version "sort of" works when on the main database page where the template passes the relevant data https://github.com/bgrins/datasette/commit/8431c98850c7a552dbcde2a4dd0c3dc942a97d25 by doing this and passing that into the schema object:

``` let TABLES_DATA = []; {% if tables is defined %}
TABLES_DATA = {{ tables | tojson(indent=2) }}; {% endif %}

// Turn into an object, shaped like https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/test/test-complete.ts#L27. const TABLES_SCHEMA = Object.fromEntries( new Map( TABLES_DATA.map((table) => { return [table.name, table.columns]; }) ).entries() ); ```

But there are a number of papercuts with it - it's not escaping table names with spaces (likely be fixable from the data being passed into the view) but mainly it doesn't seem to autocomplete columns. I think it might only want to do it when you first type the table name from my read of https://github.com/codemirror/lang-sql/blob/ebf115fffdbe07f91465ccbd82868c587f8182bc/test/test-complete.ts#L37. It's possible I'm just passing something wrong, but it may end up being something that needs feature work upstream.

Was just reviewing the SQL options and there's an upperCaseKeywords if we'd rather have SELECT vs select. Datasette seems to prefer lowercase so probably best to keep it as-is