While working on this I noticed that the arbitrary query page doesn't currently obey truncation, when it should do.

Hi @thewchan - thanks for that! Yes I'm interested, can you invite swillison @ Google's email provider please?

Here's the implementation for recipes at the moment: https://github.com/simonw/sqlite-utils/blob/5b969273f1244b1bcf3e4dc071cdf17dab35d5f8/sqlite_utils/utils.py#L434-L441

And here's the --functions implementation that doesn't expose them: https://github.com/simonw/sqlite-utils/blob/5b969273f1244b1bcf3e4dc071cdf17dab35d5f8/sqlite_utils/cli.py#L3029-L3040

I like the idea of defaulting this to the truncate_cells_html setting. It could be alternatively customized using a custom Render Cell plugin, but having Datasette do this by default makes sense to me.

Perhaps some ways to address this.

1. Add a horizontal scrollbar at the top of the table. There are some solutions here: https://stackoverflow.com/questions/3934271/horizontal-scrollbar-on-top-and-bottom-of-table
2. Use a fixed table header. It would be useful when you're lost in the middle of a very big table. Pure CSS solutions seem to exist: https://stackoverflow.com/questions/21168521/table-fixed-header-and-scrollable-body
3. Maybe a possibility to resize columns. Not sure about that because it would more work not to lose it after each reload.
4. A way to keep favorite views for each user. The process would be: I select the column I want or not (with existing "settings" icon of each column); then I select a "favoritize view" option somewhere; then I can recall all my favorite views from a menu. These data could be hosted on the browser.

Still needs documentation.

In that code self.table is the name of the current table, but can also be None if the faceting is being run against a custom SQL query instead (a future feature). self.database is the name of the current database.

So reading metadata via self.ds should work fine.

Relevant code: https://github.com/simonw/datasette/blob/c9d1943aede436fa3413fd49bc56335cbda4ad07/datasette/facets.py#L102-L110

I'm going to do this just at the table level - if you want it at the database level I would imagine usually you would be OK running --setting default_facet_size 10 instead, since most Datasette instances only expose a single database.

I may reconsider this in the future.

Relevant example from https://docs.datasette.io/en/stable/metadata.html#per-database-and-per-table-metadata json { "databases": { "database1": { "tables": { "example_table": { "sortable_columns": [ "height", "weight" ] } } } } } I think this becomes "facet_size": "..." at the table level, to match the name of the ?_facet_size=100 querystring parameter.

Tested PR just now in private org, fetched >2k repos infos flawlessly!

poetry run github-to-sqlite repos --organization github.db MYORG

Interesting feature idea.

It's a weird edge-case, but it's the kind of thing I worry about: what if the user has a column called *? This is technically allowed in SQLite, even though it's vastly unlikely.

On that basis, I think ?_json_all=1 would be a better design here than ?_json=* - since it removes any possible ambiguity.

Thanks for this - I've seen this issue too. Good fix.

I don't agree with the pros and cons listed in that article.

Codecov Report

Base: 91.66% // Head: 91.67% // Increases project coverage by +0.01% :tada:

Coverage data is based on head (2db6f9c) compared to base (51030df). Patch coverage: 100.00% of modified lines in pull request are covered.

:exclamation: Current head 2db6f9c differs from pull request most recent head 7b6a2b4. Consider uploading reports for the commit 7b6a2b4 to get more accurate results

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Well I'm not proud of it, but that did seem to do the trick!

Trying this fix: diff diff --git a/datasette/app.py b/datasette/app.py index f2a6763a5..b17888a07 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -231,7 +231,13 @@ def __init__( self.inspect_data = inspect_data self.immutables = set(immutables or []) self.databases = collections.OrderedDict() - self._refresh_schemas_lock = asyncio.Lock() + try: + self._refresh_schemas_lock = asyncio.Lock() + except RuntimeError as rex: + if "There is no current event loop in thread" in str(rex): + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + self._refresh_schemas_lock = asyncio.Lock() self.crossdb = crossdb self.nolock = nolock if memory or crossdb or not self.files: This is mainly superstition at this point! I found that recipe on https://techoverflow.net/2020/10/01/how-to-fix-python-asyncio-runtimeerror-there-is-no-current-event-loop-in-thread/

The code that I control that shows up in the trace is this:

https://github.com/simonw/datasette/blob/fd1086c6867f3e3582b1eca456e4ea95f6cecf8b/datasette/app.py#L234

Removing the ensure_eventloop() hack produced the exact same test failure: https://github.com/simonw/datasette/runs/8197570109?check_suite_focus=true

I'm suspicious that this hack here may be hurting me:

https://github.com/simonw/datasette/blob/b91e17280c05bbb9cf97432081bdcea8665879f9/tests/test_cli.py#L25-L29

I added that two years ago in https://github.com/simonw/datasette/commit/3159263f05ac4baf968929d59384d9223a539071