{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982123183", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982123183, "node_id": "IC_kwDOCGYnMM46igKv", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2021-11-29T23:20:35Z", "updated_at": "2021-12-11T01:02:19Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#347](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (71b6c38) into [main](https://codecov.io/gh/simonw/sqlite-utils/commit/213a0ff177f23a35f3b235386366ff132eb879f1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (213a0ff) will **increase** coverage by `0.00%`.\n> The diff coverage is `100.00%`.\n\n> :exclamation: Current head 71b6c38 differs from pull request most recent head 1a7ef2f. Consider uploading reports for the commit 1a7ef2f to get more accurate results\n[![Impacted file tree graph](https://codecov.io/gh/simonw/sqlite-utils/pull/347/graphs/tree.svg?width=650&height=150&src=pr&token=O0X3703L9P&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@           Coverage Diff           @@\n##             main     #347   +/-   ##\n=======================================\n  Coverage   96.51%   96.52%           \n=======================================\n  Files           5        5           \n  Lines        2270     2271    +1     \n=======================================\n+ Hits         2191     2192    +1     \n  Misses         79       79           \n```\n\n\n| [Impacted Files](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage \u0394 | |\n|---|---|---|\n| [sqlite\\_utils/cli.py](https://codecov.io/gh/simonw/sqlite-utils/pull/347/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2NsaS5weQ==) | `95.73% <100.00%> (\u00f8)` | |\n| [sqlite\\_utils/utils.py](https://codecov.io/gh/simonw/sqlite-utils/pull/347/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `93.68% <100.00%> (+0.03%)` | :arrow_up: |\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute <relative> (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [213a0ff...1a7ef2f](https://codecov.io/gh/simonw/sqlite-utils/pull/347?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-991397907", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 991397907, "node_id": "IC_kwDOCGYnMM47F4gT", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-11T01:01:40Z", "updated_at": "2021-12-11T01:01:40Z", "author_association": "OWNER", "body": "The change I made to that test in #354 might help with this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137888", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982137888, "node_id": "IC_kwDOCGYnMM46ijwg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-29T23:50:54Z", "updated_at": "2021-11-29T23:50:54Z", "author_association": "OWNER", "body": "If I'm going to `skipIf()` those tests I need a way to check if `pysqlite3` is being used.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982137293", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982137293, "node_id": "IC_kwDOCGYnMM46ijnN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-29T23:49:29Z", "updated_at": "2021-11-29T23:49:29Z", "author_association": "OWNER", "body": "A short term fix would be to skip those tests against `pysqlite3` - but longer term it would be good to address the underlying issue, particularly for the WAL ones (the FTS ones aren't too worrying since if you deliberately try and break the FTS table it's not hugely problematic if you corrupt your database).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982136747", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982136747, "node_id": "IC_kwDOCGYnMM46ijer", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-29T23:48:05Z", "updated_at": "2021-11-29T23:48:05Z", "author_association": "OWNER", "body": "Some interesting test failures in the version that runs with `pysqlite3`:\r\n```\r\n=========================== short test summary info ============================\r\nFAILED tests/test_cli.py::test_enable_wal - assert 0 == 1\r\nFAILED tests/test_cli.py::test_disable_wal - pysqlite3.dbapi2.OperationalErro...\r\nFAILED tests/test_fts.py::test_rebuild_fts[searchable] - pysqlite3.dbapi2.Dat...\r\nFAILED tests/test_fts.py::test_rebuild_fts[searchable_fts] - pysqlite3.dbapi2...\r\nFAILED tests/test_wal.py::test_enable_disable_wal - pysqlite3.dbapi2.Operatio...\r\n================== 5 failed, 750 passed, 3 skipped in 15.20s ===================\r\n```\r\nhttps://github.com/simonw/sqlite-utils/runs/4360759085\r\n\r\nThe WAL errors look like this:\r\n```\r\nE           pysqlite3.dbapi2.OperationalError: cannot change into wal mode from within a transaction\r\n```\r\nTriggered by a call to `db.enable_wal()`\r\n\r\nThe FTS errors are caused by tests that try to deliberately corrupt the FTS index by running `fresh_db[\"searchable_fts_data\"].delete_where()` - and then rebuilding it using `rebuild_fts()`:\r\n```\r\n    @pytest.mark.parametrize(\"table_to_fix\", [\"searchable\", \"searchable_fts\"])\r\n    def test_rebuild_fts(fresh_db, table_to_fix):\r\n        table = fresh_db[\"searchable\"]\r\n        table.insert(search_records[0])\r\n        table.enable_fts([\"text\", \"country\"])\r\n        # Run a search\r\n        rows = list(table.search(\"tanuki\"))\r\n        assert len(rows) == 1\r\n        assert {\r\n            \"rowid\": 1,\r\n            \"text\": \"tanuki are running tricksters\",\r\n            \"country\": \"Japan\",\r\n            \"not_searchable\": \"foo\",\r\n        }.items() <= rows[0].items()\r\n        # Delete from searchable_fts_data\r\n        fresh_db[\"searchable_fts_data\"].delete_where()\r\n        # This should have broken the index\r\n        with pytest.raises(sqlite3.DatabaseError):\r\n            list(table.search(\"tanuki\"))\r\n        # Running rebuild_fts() should fix it\r\n>       fresh_db[table_to_fix].rebuild_fts()\r\n\r\ntests/test_fts.py:277: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\nsqlite_utils/db.py:1947: in rebuild_fts\r\n    self.db.execute(\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nself = <Database <pysqlite3.dbapi2.Connection object at 0x7fd95d299c30>>\r\nsql = \"INSERT INTO [searchable_fts]([searchable_fts]) VALUES('rebuild');\"\r\nparameters = None\r\n\r\n    def execute(\r\n        self, sql: str, parameters: Optional[Union[Iterable, dict]] = None\r\n    ) -> sqlite3.Cursor:\r\n        \"Execute SQL query and return a ``sqlite3.Cursor``.\"\r\n        if self._tracer:\r\n            self._tracer(sql, parameters)\r\n        if parameters is not None:\r\n            return self.conn.execute(sql, parameters)\r\n        else:\r\n>           return self.conn.execute(sql)\r\nE           pysqlite3.dbapi2.DatabaseError: database disk image is malformed\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982133970", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982133970, "node_id": "IC_kwDOCGYnMM46iizS", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-29T23:41:17Z", "updated_at": "2021-11-29T23:41:17Z", "author_association": "OWNER", "body": "Took a bit of experimenting to get both `mypy` AND `flake8` to ignore the same line. The incantation that worked was this one:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/f990e134aa8219b687ff6c261330f36824b5df36/sqlite_utils/utils.py#L8\r\n\r\nOrder here matters - this did NOT work for both tools:\r\n\r\n```python\r\n from sqlite3.dump import _iterdump as iterdump   # noqa: F401 # type: ignore\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982126665", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982126665, "node_id": "IC_kwDOCGYnMM46ihBJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-29T23:26:01Z", "updated_at": "2021-11-29T23:33:48Z", "author_association": "OWNER", "body": "https://github.com/simonw/sqlite-utils/blob/93b059dd230eae9eaae472b7fbabd4a66feeb79d/.github/workflows/test.yml#L11-L20\r\n\r\nThis configuration means that the numpy=0, Python=3.10, os=Ubuntu build will additionally use `pysqlite3` with the SQLite 3.37.0.\r\n\r\nIt's failing right now: https://github.com/simonw/sqlite-utils/runs/4360593156 - because `pysqlite3` doesn't provide `.iterdump()`.\r\n\r\nI can use the workaround from this comment: https://github.com/coleifer/pysqlite3/issues/24#issuecomment-982081267", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129727", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982129727, "node_id": "IC_kwDOCGYnMM46ihw_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-29T23:31:58Z", "updated_at": "2021-11-29T23:31:58Z", "author_association": "OWNER", "body": "It failed on other Python versions with `mypy`:\r\n```\r\nsqlite_utils/utils.py:8: error: Cannot find implementation or library stub for module named \"sqlite3.dump\"\r\nsqlite_utils/utils.py:8: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/pull/347#issuecomment-982129218", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347", "id": 982129218, "node_id": "IC_kwDOCGYnMM46ihpC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-29T23:31:02Z", "updated_at": "2021-11-29T23:31:02Z", "author_association": "OWNER", "body": "Here's the test run that's installing `pysqlite3` and that version of SQLite: https://github.com/simonw/sqlite-utils/runs/4360663292", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1066603133, "label": "Test against pysqlite3 running SQLite 3.37"}, "performed_via_github_app": null}