issue_comments

sql select sqlite_master.name as table_name, table_info.* from sqlite_master join pragma_table_info(sqlite_master.name) as table_info order by sqlite_master.name, table_info.cid https://latest.datasette.io/fixtures?sql=select%0D%0A++sqlite_master.name+as+table_name%2C%0D%0A++table_info.*%0D%0Afrom%0D%0A++sqlite_master%0D%0A++join+pragma_table_info%28sqlite_master.name%29+as+table_info%0D%0Aorder+by%0D%0A++sqlite_master.name%2C%0D%0A++table_info.cid

OK, I just released 0.41 with that and a bunch of other stuff: https://datasette.readthedocs.io/en/latest/changelog.html#v0-41

Actually I'm going to put that release out today. I was hoping to finish #698 first but that shouldn't delay those other features any longer.

I'll definitely get that out this week!

For the moment a trick I often use is to put a URL to the most recent commit in my requirements.txt - e.g. https://github.com/simonw/datasette/archive/0784f2ef9d3ff6dd9df05f54cb51de29a6d11764.zip

This should be safe because nothing lands on Datasette master without the full unit test suite passing. But you're right, there's a bunch of stuff now that needs to go out in a release: https://github.com/simonw/datasette/compare/0.40...0784f2ef9d3ff6dd9df05f54cb51de29a6d11764

It's interesting to hear from someone who's using this feature - I'm considering moving it out into a plugin #647.

Thanks for the suggestion! I'll add this.

This was a deliberate change from #651.

The _search_colname= alternative argument for doing this still works - compare these two:

• https://latest.datasette.io/fixtures/searchable?_search=dog
• https://latest.datasette.io/fixtures/searchable?_search_tex1=dog

If you want to use advanced search syntax on those pages you can do so using the &_searchmode=raw option - I added better documentation for this the other day: https://datasette.readthedocs.io/en/latest/full_text_search.html#advanced-sqlite-search-queries

Example: https://latest.datasette.io/fixtures/searchable?_search=text1:dog&_searchmode=raw

More demos here: https://github.com/simonw/til/blob/master/sqlite/list-all-columns-in-a-database.md

Potential recipe in https://github.com/simonw/til/blob/master/sqlite/list-all-columns-in-a-database.md

Interestingly https://latest.datasette.io/fixtures?sql=select+*+from+pragma_function_list() doesn't work, when it DOES work on my laptop.

latest.datasette.io currently runs SQLite 3.27.2 while my laptop runs 3.31.1

https://www.sqlite.org/changes.html#version_3_30_0 says that as-of 3.30.0:

The PRAGMA function_list, PRAGMA module_list, and PRAGMA pragma_list commands are now enabled in all builds by default. Disable them using -DSQLITE_OMIT_INTROSPECTION_PRAGMAS.

Some demos: * https://latest.datasette.io/fixtures?sql=select++from+pragma_database_list%28%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_foreign_key_list%28%27complex_foreign_keys%27%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_function_list%28%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_index_info%28%27idx_compound_three_primary_keys_content%27%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_index_list%28%27compound_three_primary_keys%27%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_index_xinfo%28%27idx_compound_three_primary_keys_content%27%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_page_count%28%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_max_page_count%28%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_page_size%28%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_schema_version%28%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_table_info%28%27complex_foreign_keys%27%29 * https://latest.datasette.io/fixtures?sql=select++from+pragma_table_xinfo%28%27complex_foreign_keys%27%29

For the moment I'll allow-list the following: * pragma_database_list * pragma_foreign_key_list * pragma_function_list * pragma_index_info * pragma_index_list * pragma_index_xinfo * pragma_page_count * pragma_max_page_count * pragma_page_size * pragma_schema_version * pragma_table_info * pragma_table_xinfo

I could allow-list some other useful pragma_x tables too.

SQLite calls these "pragma functions" - documented here: https://www.sqlite.org/pragma.html#pragfunc

They sound safe:

Table-valued functions exist only for PRAGMAs that return results and that have no side-effects.

Maybe use a negative lookahead assertion? https://docs.python.org/3/library/re.html#index-20

(?!...)

Matches if ... doesn’t match next. This is a negative lookahead assertion. For example, Isaac (?!Asimov) will match 'Isaac ' only if it’s not followed by 'Asimov'.

The rationale for blocking pragma entirely from statements is that it can be used to change the state of the SQLite database, e.g. from https://www.sqlite.org/pragma.html :

PRAGMA schema.application_id; PRAGMA schema.application_id = integer ; That second line is unsafe. I don't think it's possible to use the pragma_table_x variants to make writes in this way.

select * from pragma_table_info(tablename); is currently disallowed for user-provided queries via a regex restriction - but could help here too.

https://github.com/simonw/datasette/blob/d349d57cdf3d577afb62bdf784af342a4d5be660/datasette/utils/init.py#L174