issue_comments

I'm less convinced by the "rewrite the query in some way" optional idea. What kind of use-cases can you imagine for that?

My hunch is that it's much more likely to cause weird breakages than it is to allow for useful plugin extensions, but I'm willing to be convinced otherwise.

Interestingly enough there's actually a mechanism that looks like that a bit already: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/views/database.py#L496-L508

That validate_sql_select() function is defined here: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/utils/init.py#L256-L265

Against these constants:

https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/utils/init.py#L223-L253

Which isn't a million miles away from your suggestion to have a hook that can say if the query should be executed or not.

Are you talking just about queries submitted to /database?sql= using the interface on https://latest.datasette.io/fixtures?sql=select+*+from+facetable or are you interested in queries that are run to power other pages like https://latest.datasette.io/fixtures/facetable as well? I'll assume the former.

There are a few ways you could solve this at the moment.

The easiest would be with a piece of ASGI middleware that looks for URLs matching /dbname?sql=... and logs those. I played with a version of that a few years ago: https://simonwillison.net/2019/Dec/16/logging-sqlite-asgi-middleware/ - see also https://github.com/simonw/asgi-log-to-sqlite

Then you can load that middleware from a plugin using https://docs.datasette.io/en/stable/plugin_hooks.html#asgi-wrapper-datasette

That feels a bit delicate because it's relying on the URL design not changing, but I'm happy to confirm that URL is going to stay the same for Datasette 1.0 and I have no plans to change it ever.

There's also a tracing mechanism built into Datasette itself that you could hook into. The internals of that are documented here: https://docs.datasette.io/en/stable/internals.html#datasette-tracer - but I don't yet consider it a 100% stable API. I don't plan to change it but I won't promise not to either.

I used that mechanism in this plugin: https://datasette.io/plugins/datasette-pretty-traces - demonstrated here: https://latest-with-plugins.datasette.io/github/commits?_trace=1

The hackiest way to do this would be to patch Datasette itself and try to replace the query_view. This definitely isn't a documented, stable API though and would be very likely to break at arbitrary points in the future.

So my recommendation for the moment is the ASGI middleware option.

I'm definitely open to suggestions for plugin hooks that might make this kind of thing easier.

One idea I've been mulling is whether there should be a plugin hook that files on arbitrary views - similar to Django's process_view mechanism: https://docs.djangoproject.com/en/4.2/topics/http/middleware/#process-view

That would allow people to setup code that runs before or after any of the default views in Datasette.

I'm not yet 100% sold on the idea, because I worry about implementing it in a way that guarantees plugins won't break on future releases. But I'm open to considering it.