issue_comments

This work is now happening in the facets branch. Closing this in favor of #255.

If I'm going to expand column introspection in this way it would be useful to also capture column type information.

Here's a quick demo of that exploration: https://datasette-distinct-column-values.now.sh/-/inspect

Example output:

{ "antiquities-act/actions_under_antiquities_act": { "columns": [ "current_name", "states", "original_name", "current_agency", "action", "date", "year", "pres_or_congress", "acres_affected" ], "count": 344, "distinct_values_by_column": { "acres_affected": null, "action": null, "current_agency": [ "NPS", "State of Montana", "BLM", "State of Arizona", "USFS", "State of North Dakota", "NPS, BLM", "State of South Carolina", "State of New York", "FWS", "FWS, NOAA", "NPS, FWS", "NOAA", "BLM, USFS", "NOAA, FWS" ], "current_name": null, "date": null, "original_name": null, "pres_or_congress": null, "states": null, "year": null }, "foreign_keys": { "incoming": [], "outgoing": [] }, "fts_table": null, "hidden": false, "label_column": null, "name": "antiquities-act/actions_under_antiquities_act", "primary_keys": [] } }

We can solve this using the sqlite_timelimit(conn, 20) helper, which can tell SQLite to give up after 20ms. We can wrap that around the following SQL:

select distinct COLUMN from TABLE limit 21;

Then we look at the number of rows returned. If it's 21 or more we know that this table had more than 21 distinct values, so we'll treat it as "unlimited". Likewise, if the SQL times out before 20ms is up we will skip this introspection.