html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137888,https://api.github.com/repos/simonw/sqlite-utils/issues/347,982137888,IC_kwDOCGYnMM46ijwg,9599,2021-11-29T23:50:54Z,2021-11-29T23:50:54Z,OWNER,If I'm going to `skipIf()` those tests I need a way to check if `pysqlite3` is being used.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1066603133, https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137293,https://api.github.com/repos/simonw/sqlite-utils/issues/347,982137293,IC_kwDOCGYnMM46ijnN,9599,2021-11-29T23:49:29Z,2021-11-29T23:49:29Z,OWNER,"A short term fix would be to skip those tests against `pysqlite3` - but longer term it would be good to address the underlying issue, particularly for the WAL ones (the FTS ones aren't too worrying since if you deliberately try and break the FTS table it's not hugely problematic if you corrupt your database).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1066603133, https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982136747,https://api.github.com/repos/simonw/sqlite-utils/issues/347,982136747,IC_kwDOCGYnMM46ijer,9599,2021-11-29T23:48:05Z,2021-11-29T23:48:05Z,OWNER,"Some interesting test failures in the version that runs with `pysqlite3`: ``` =========================== short test summary info ============================ FAILED tests/test_cli.py::test_enable_wal - assert 0 == 1 FAILED tests/test_cli.py::test_disable_wal - pysqlite3.dbapi2.OperationalErro... FAILED tests/test_fts.py::test_rebuild_fts[searchable] - pysqlite3.dbapi2.Dat... FAILED tests/test_fts.py::test_rebuild_fts[searchable_fts] - pysqlite3.dbapi2... FAILED tests/test_wal.py::test_enable_disable_wal - pysqlite3.dbapi2.Operatio... ================== 5 failed, 750 passed, 3 skipped in 15.20s =================== ``` https://github.com/simonw/sqlite-utils/runs/4360759085 The WAL errors look like this: ``` E pysqlite3.dbapi2.OperationalError: cannot change into wal mode from within a transaction ``` Triggered by a call to `db.enable_wal()` The FTS errors are caused by tests that try to deliberately corrupt the FTS index by running `fresh_db[""searchable_fts_data""].delete_where()` - and then rebuilding it using `rebuild_fts()`: ``` @pytest.mark.parametrize(""table_to_fix"", [""searchable"", ""searchable_fts""]) def test_rebuild_fts(fresh_db, table_to_fix): table = fresh_db[""searchable""] table.insert(search_records[0]) table.enable_fts([""text"", ""country""]) # Run a search rows = list(table.search(""tanuki"")) assert len(rows) == 1 assert { ""rowid"": 1, ""text"": ""tanuki are running tricksters"", ""country"": ""Japan"", ""not_searchable"": ""foo"", }.items() <= rows[0].items() # Delete from searchable_fts_data fresh_db[""searchable_fts_data""].delete_where() # This should have broken the index with pytest.raises(sqlite3.DatabaseError): list(table.search(""tanuki"")) # Running rebuild_fts() should fix it > fresh_db[table_to_fix].rebuild_fts() tests/test_fts.py:277: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sqlite_utils/db.py:1947: in rebuild_fts self.db.execute( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = > sql = ""INSERT INTO [searchable_fts]([searchable_fts]) VALUES('rebuild');"" parameters = None def execute( self, sql: str, parameters: Optional[Union[Iterable, dict]] = None ) -> sqlite3.Cursor: ""Execute SQL query and return a ``sqlite3.Cursor``."" if self._tracer: self._tracer(sql, parameters) if parameters is not None: return self.conn.execute(sql, parameters) else: > return self.conn.execute(sql) E pysqlite3.dbapi2.DatabaseError: database disk image is malformed ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1066603133, https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982133970,https://api.github.com/repos/simonw/sqlite-utils/issues/347,982133970,IC_kwDOCGYnMM46iizS,9599,2021-11-29T23:41:17Z,2021-11-29T23:41:17Z,OWNER,"Took a bit of experimenting to get both `mypy` AND `flake8` to ignore the same line. The incantation that worked was this one: https://github.com/simonw/sqlite-utils/blob/f990e134aa8219b687ff6c261330f36824b5df36/sqlite_utils/utils.py#L8 Order here matters - this did NOT work for both tools: ```python from sqlite3.dump import _iterdump as iterdump # noqa: F401 # type: ignore ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1066603133, https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982126665,https://api.github.com/repos/simonw/sqlite-utils/issues/347,982126665,IC_kwDOCGYnMM46ihBJ,9599,2021-11-29T23:26:01Z,2021-11-29T23:33:48Z,OWNER,"https://github.com/simonw/sqlite-utils/blob/93b059dd230eae9eaae472b7fbabd4a66feeb79d/.github/workflows/test.yml#L11-L20 This configuration means that the numpy=0, Python=3.10, os=Ubuntu build will additionally use `pysqlite3` with the SQLite 3.37.0. It's failing right now: https://github.com/simonw/sqlite-utils/runs/4360593156 - because `pysqlite3` doesn't provide `.iterdump()`. I can use the workaround from this comment: https://github.com/coleifer/pysqlite3/issues/24#issuecomment-982081267","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1066603133, https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129727,https://api.github.com/repos/simonw/sqlite-utils/issues/347,982129727,IC_kwDOCGYnMM46ihw_,9599,2021-11-29T23:31:58Z,2021-11-29T23:31:58Z,OWNER,"It failed on other Python versions with `mypy`: ``` sqlite_utils/utils.py:8: error: Cannot find implementation or library stub for module named ""sqlite3.dump"" sqlite_utils/utils.py:8: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1066603133, https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129218,https://api.github.com/repos/simonw/sqlite-utils/issues/347,982129218,IC_kwDOCGYnMM46ihpC,9599,2021-11-29T23:31:02Z,2021-11-29T23:31:02Z,OWNER,Here's the test run that's installing `pysqlite3` and that version of SQLite: https://github.com/simonw/sqlite-utils/runs/4360663292,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1066603133,