issue_comments

Closing this in favour of rethinking how sanity checks work.

The technique from run_sanity_checks of running PRAGMA table_info({}) for every table seems to work just fine. It failed for the Apple Photos database for example: sqlite> pragma table_info(RKSceneInVersion_VirtualBufferReader); Error: no such module: VirtualBufferReaderModule So I think the solution to this ticket is going to be moving that logic into a new utility function.

This helped: $ sqlite3 /tmp/hearst.db SQLite version 3.24.0 2018-06-04 14:10:15 Enter ".help" for usage hints. sqlite> delete from spatial_ref_sys where srid != 4326; sqlitte> delete from spatial_ref_sys_aux where srid != 4326; sqlite> vacuum; sqlite> ^D $ ls -lah /tmp/hearst.db -rw-r--r-- 1 simonw wheel 216K Feb 14 12:01 /tmp/hearst.db

I tried to make the smallest SpatiaLite database file I could (to use for the tests), but it ended up over 5MB! $ echo '{"type":"Feature","properties":{"name":"Hearst Castle"},"geometry":{"type":"Point","coordinates":[-121.1686,35.685]}}' | geojson-to-sqlite /tmp/hearst.db places - --spatialite $ ls -lah /tmp/hearst.db -rw-r--r-- 1 simonw wheel 5.3M Feb 14 11:54 /tmp/hearst.db I imagine that's because of these tables:

I may be re-inventing this code at the moment: https://github.com/simonw/datasette/blob/3ffb8f3b98252531d11897fd431711e9b8045ace/datasette/app.py#L219-L237

For the unit tests I think I'm going to have to create minimal binary SQLite file examples and include them in the repo.

is_openable_sqlite perhaps?

I can extend this function (maybe also rename it):

https://github.com/simonw/datasette/blob/52ba34701cdbf510236de87d35b0e6df330626d1/datasette/utils/init.py#L595-L610

Here's how to test if the rtree virtual table is supported: ```

import sqlite3 c = sqlite3.connect(":memory:") c.execute("create virtual table blah using rtree (a, b, c)") <sqlite3.Cursor object at 0x107bc4180> c.execute("create virtual table blah2 using rtree2 (a, b, c)") Traceback (most recent call last): File "<stdin>", line 1, in <module> sqlite3.OperationalError: table blah already exists Also: c.execute('''CREATE VIRTUAL TABLE SpatialIndex USING VirtualSpatialIndex()''') Traceback (most recent call last): File "<stdin>", line 1, in <module> sqlite3.OperationalError: no such module: VirtualSpatialIndex c.enable_load_extension( ... True)

c.load_extension("/usr/local/lib/mod_spatialite.dylib") c.execute('''CREATE VIRTUAL TABLE SpatialIndex USING VirtualSpatialIndex()''') <sqlite3.Cursor object at 0x107bc4c70> ```

Amazingly, I get 0 search results on Google for RidList_VirtualReaderModule! I guess no-one has reverse engineered the Apple Photos SQLite database at that level yet.

This can be a function in utils/__init__.py.