issue_comments

While working on this I noticed that the arbitrary query page doesn't currently obey truncation, when it should do.

Hi @thewchan - thanks for that! Yes I'm interested, can you invite swillison @ Google's email provider please?

Here's the implementation for recipes at the moment: https://github.com/simonw/sqlite-utils/blob/5b969273f1244b1bcf3e4dc071cdf17dab35d5f8/sqlite_utils/utils.py#L434-L441

And here's the --functions implementation that doesn't expose them: https://github.com/simonw/sqlite-utils/blob/5b969273f1244b1bcf3e4dc071cdf17dab35d5f8/sqlite_utils/cli.py#L3029-L3040

I like the idea of defaulting this to the truncate_cells_html setting. It could be alternatively customized using a custom Render Cell plugin, but having Datasette do this by default makes sense to me.

Still needs documentation.

In that code self.table is the name of the current table, but can also be None if the faceting is being run against a custom SQL query instead (a future feature). self.database is the name of the current database.

So reading metadata via self.ds should work fine.

Relevant code: https://github.com/simonw/datasette/blob/c9d1943aede436fa3413fd49bc56335cbda4ad07/datasette/facets.py#L102-L110

I'm going to do this just at the table level - if you want it at the database level I would imagine usually you would be OK running --setting default_facet_size 10 instead, since most Datasette instances only expose a single database.

I may reconsider this in the future.

Relevant example from https://docs.datasette.io/en/stable/metadata.html#per-database-and-per-table-metadata json { "databases": { "database1": { "tables": { "example_table": { "sortable_columns": [ "height", "weight" ] } } } } } I think this becomes "facet_size": "..." at the table level, to match the name of the ?_facet_size=100 querystring parameter.

Interesting feature idea.

It's a weird edge-case, but it's the kind of thing I worry about: what if the user has a column called *? This is technically allowed in SQLite, even though it's vastly unlikely.

On that basis, I think ?_json_all=1 would be a better design here than ?_json=* - since it removes any possible ambiguity.

Thanks for this - I've seen this issue too. Good fix.

I don't agree with the pros and cons listed in that article.

Well I'm not proud of it, but that did seem to do the trick!

Trying this fix: diff diff --git a/datasette/app.py b/datasette/app.py index f2a6763a5..b17888a07 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -231,7 +231,13 @@ def __init__( self.inspect_data = inspect_data self.immutables = set(immutables or []) self.databases = collections.OrderedDict() - self._refresh_schemas_lock = asyncio.Lock() + try: + self._refresh_schemas_lock = asyncio.Lock() + except RuntimeError as rex: + if "There is no current event loop in thread" in str(rex): + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + self._refresh_schemas_lock = asyncio.Lock() self.crossdb = crossdb self.nolock = nolock if memory or crossdb or not self.files: This is mainly superstition at this point! I found that recipe on https://techoverflow.net/2020/10/01/how-to-fix-python-asyncio-runtimeerror-there-is-no-current-event-loop-in-thread/

The code that I control that shows up in the trace is this:

https://github.com/simonw/datasette/blob/fd1086c6867f3e3582b1eca456e4ea95f6cecf8b/datasette/app.py#L234

Removing the ensure_eventloop() hack produced the exact same test failure: https://github.com/simonw/datasette/runs/8197570109?check_suite_focus=true

I'm suspicious that this hack here may be hurting me:

https://github.com/simonw/datasette/blob/b91e17280c05bbb9cf97432081bdcea8665879f9/tests/test_cli.py#L25-L29

I added that two years ago in https://github.com/simonw/datasette/commit/3159263f05ac4baf968929d59384d9223a539071