issue_comments

Maybe I split this out into a separate Python library that gets tested against every SQLite release I can possibly try it against, and then bakes out the supported release versions into the library code itself?

I'm going to do this, and call the Python library sqlite-explain.

I did a bunch of work on this in #266. The library is now pretty thoroughly typed, and I even found a couple of bugs using mypy along the way: #313 and #315.

Some questions:

• Should this support compression formats other than gzip?
• Should memory learn to auto-detect gzipped data?

Something like this then:

sqlite-utils file.db "select * from t" --csv --gz > t.csv.gz

Maybe add a -o t.csv.gz option too so you don't have to use a >.

Thanks, this was a bug.

Documentation for table.search(..., quote=True): https://sqlite-utils.datasette.io/en/latest/python-api.html#searching-with-table-search

In the API reference: https://sqlite-utils.datasette.io/en/latest/reference.html#sqlite_utils.db.Table.search

And for the CLI --quote option: https://sqlite-utils.datasette.io/en/latest/cli.html#executing-searches

``` % sqlite-utils search fixtures.db searchable 'dog"' Error: malformed MATCH expression: [dog"]

Try running this again with the --quote option ```

sqlite-utils search fixtures.db searchable 'dog"' Error: malformed MATCH expression: [dog"] This error message could suggest retrying with --quote.

Also add sqlite-utils search ... --quote option.

More documentation: https://sqlite-utils.datasette.io/en/latest/python-api.html#quoting-characters-for-use-in-search

I think I'll do this as an optional table.search(..., escape=True) parameter.

Actually I'll do quote=True for consistency with the new db.quote_fts() method.

The db.quote_fts(value) method from #247 can now be used for this - documentation here: https://sqlite-utils.datasette.io/en/latest/reference.html#sqlite_utils.db.Database.quote_fts

I'll be adding further improvements relating to this (a table.search(q, quote=True) parameter) in #296.

This was also requested in #296.

This is a good call. I have a fix for this in Datasette but it's not in sqlite-utils yet: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/init.py#L824-L835

I'm just going to remove this - I added it when the library was mostly undocumented, but it has comprehensive documentation now.

Maybe I should use -/ to encode forward slashes too, to defend against any ASGI servers that might not implement raw_path correctly.

The documentation should definitely cover how table names become URLs, in case any third party code needs to be able to calculate this themselves.

So given the original examples, a table called table.csv would have the following URLs:

• /db/table-.csv - the HTML version
• /db/table-.csv.csv - the CSV version
• /db/table-.csv.json - the JSON version

And if for some horific reason you had a table with the name /db/table-.csv.csv (so /db/ was the first part of the actual table name in SQLite) the URLs would look like this:

• /db/%2Fdb%2Ftable---.csv-.csv - the HTML version
• /db/%2Fdb%2Ftable---.csv-.csv.csv - the CSV version
• /db/%2Fdb%2Ftable---.csv-.csv.json - the JSON version

Here's an alternative I just made up which I'm calling "dot dash" encoding:

```python def dot_dash_encode(s): return s.replace("-", "--").replace(".", "-.")

def dot_dash_decode(s): return s.replace("-.", ".").replace("--", "-") And some examples:python for example in ( "hello", "hello.csv", "hello-and-so-on.csv", "hello-.csv", "hello--and--so--on-.csv", "hello.csv.", "hello.csv.-", "hello.csv.--", ): print(example) print(dot_dash_encode(example)) print(example == dot_dash_decode(dot_dash_encode(example))) print() Outputs: hello hello True

hello.csv hello-.csv True

hello-and-so-on.csv hello--and--so--on-.csv True

hello-.csv hello---.csv True

hello--and--so--on-.csv hello----and----so----on---.csv True

hello.csv. hello-.csv-. True

hello.csv.- hello-.csv-.-- True

hello.csv.-- hello-.csv-.---- True ```

But... what if I invent my own escaping scheme?

I actually did this once before, in https://github.com/simonw/datasette/commit/9fdb47ca952b93b7b60adddb965ea6642b1ff523 - while I was working on porting Datasette to ASGI in https://github.com/simonw/datasette/issues/272#issuecomment-494192779 because ASGI didn't yet have the raw_path mechanism.

I could bring that back - it looked like this:

"table/and/slashes" => "tableU+002FandU+002Fslashes" "~table" => "U+007Etable" "+bobcats!" => "U+002Bbobcats!" "U+007Etable" => "UU+002B007Etable" But I didn't particularly like it - it was quite verbose.