issue_comments

https://www.sqlite.org/lang_insert.html

Looks like it's as simple as INSERT OR IGNORE - and we already have code related to that here:

https://github.com/simonw/sqlite-utils/blob/1e28eeee8ce55ea68eddb228294a1eff6785b497/sqlite_utils/db.py#L498-L501

I'm going to generate statistics for every TEXT column.

Any column with more than 90% distinct rows (compared to the total count of rows) will be a candidate for the label.

I will then pick the candidate column with the shortest average length.

Fixed: https://latest.datasette.io/fixtures/roadside_attraction_characteristics?_sort=attraction_id&_size=2

I'm leaning towards supporting both hooks.

Here's an older version of what that custom table filtering plugin might look like: https://github.com/simonw/datasette/commit/5116c4ec8aed5091e1f75415424b80f613518dc6

```python from datasette.utils import TableFilter

@hookimpl def table_filter(): async def inner(view, name, table, request): extra_human_descriptions = [] where_clauses = [] params = {} # ... build those things here return TableFilter( human_description_extras=extra_human_descriptions, where_clauses=where_clauses, params=params, ) return inner ```

I built this for the https://github.com/simonw/russian-ira-facebook-ads-datasette project.

It's pretty neat. Maybe I should go with that?

Assuming I do go ahead with this plugin hook, the existing InFilter makes for a nice simple example that illustrates the two key methods: .where_clause() and .human_clause():

```python class InFilter(Filter): key = "in" display = "in"

def split_value(self, value):
    if value.startswith("["):
        return json.loads(value)
    else:
        return [v.strip() for v in value.split(",")]

def where_clause(self, table, column, value, param_counter):
    values = self.split_value(value)
    params = [":p{}".format(param_counter + i) for i in range(len(values))]
    sql = "{} in ({})".format(escape_sqlite(column), ", ".join(params))
    return sql, values

def human_clause(self, column, value):
    return "{} in {}".format(column, json.dumps(self.split_value(value)))

```

I'm having trouble coming up with interesting column-based filters which don't make sense to ship as default behaviour.

I wonder if this is the right hook?

The more likely case is that we need a hook that registers a new type of lookup entirely - ?_spatial_within={geojson} for example. These aren't necessarily attached to a specific column and may have entirely custom syntax - but they still need to be able to influence the query (like the _where= clause does) and the human-readable description of the page.

Is there a strong case for supporting both custom filter plugins AND custom table where plugins, or could those where plugins cover both bases?

It would be nice if this plugin was passed the current database/table so it can decide to enable new filters only for specific tables. This will require a bit of refactoring because the filters list is static at the moment - it would instead have to be returned by a function that runs when the table view is rendered.