{"id": 1690765434, "node_id": "I_kwDOBm6k_c5kxwh6", "number": 2067, "title": "Litestream-restored db: errors on 3.11 and 3.10.8; but works on py3.10.7 and 3.10.6", "user": {"value": 39538958, "label": "justmars"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2023-05-01T12:42:28Z", "updated_at": "2023-05-03T00:16:03Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi! Wondering if this issue is limited to my local system or if it affects others as well. \r\n\r\nIt seems like 3.11 errors out on a \"litestream-restored\" database. On further investigation, it also appears to conk out on 3.10.8 but works on 3.10.7 and 3.10.6.\r\n\r\nTo demo issue I created a test database, replicated it to an aws s3 bucket, then restored the same under various .pyenv-versioned shells where I test whether I can read the database via the sqlite3 cli.\r\n\r\n```sh\r\n# create new shell with 3.11.3\r\nlitestream restore -o data/db.sqlite s3://mytestbucketxx/db\r\nsqlite3 data/db.sqlite  \r\n# SQLite version 3.41.2 2023-03-22 11:56:21\r\n# Enter \".help\" for usage hints.\r\n# sqlite> .tables\r\n# _litestream_lock  _litestream_seq   movie           \r\n# sqlite> \r\n```\r\n\r\nHowever this get me an `OperationalError` when reading via datasette:\r\n\r\n<details>\r\n<summary>Error on 3.11.3 and 3.10.8</summary>\r\n\r\n```sh\r\ndatasette data/db.sqlite\r\n```\r\n\r\n```console\r\n/tester/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API\r\n  warnings.warn(\"pkg_resources is deprecated as an API\", DeprecationWarning)\r\nTraceback (most recent call last):\r\n  File \"/tester/.venv/bin/datasette\", line 8, in <module>\r\n    sys.exit(cli())\r\n             ^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/click/core.py\", line 1130, in __call__\r\n    return self.main(*args, **kwargs)\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/click/core.py\", line 1055, in main\r\n    rv = self.invoke(ctx)\r\n         ^^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/click/core.py\", line 1657, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/click/core.py\", line 1404, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/click/core.py\", line 760, in invoke\r\n    return __callback(*args, **kwargs)\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/datasette/cli.py\", line 143, in wrapped\r\n    return fn(*args, **kwargs)\r\n           ^^^^^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/datasette/cli.py\", line 615, in serve\r\n    asyncio.get_event_loop().run_until_complete(check_databases(ds))\r\n  File \"/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/asyncio/base_events.py\", line 653, in run_until_complete\r\n    return future.result()\r\n           ^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/datasette/cli.py\", line 660, in check_databases\r\n    await database.execute_fn(check_connection)\r\n  File \"/tester/.venv/lib/python3.11/site-packages/datasette/database.py\", line 213, in execute_fn\r\n    return await asyncio.get_event_loop().run_in_executor(\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/concurrent/futures/thread.py\", line 58, in run\r\n    result = self.fn(*self.args, **self.kwargs)\r\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/datasette/database.py\", line 211, in in_thread\r\n    return fn(conn)\r\n           ^^^^^^^^\r\n  File \"/tester/.venv/lib/python3.11/site-packages/datasette/utils/__init__.py\", line 951, in check_connection\r\n    for r in conn.execute(\r\n             ^^^^^^^^^^^^^\r\nsqlite3.OperationalError: unable to open database file\r\n```\r\n\r\n</details>\r\n\r\n\r\n<details>\r\n<summary>Works on 3.10.7, 3.10.6</summary>\r\n\r\n```sh\r\n# create new shell with 3.10.7 / 3.10.6\r\nlitestream restore -o data/db.sqlite s3://mytestbucketxx/db\r\ndatasette data/db.sqlite\r\n# ...\r\n# INFO:     Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)\r\n```\r\n\r\n</details>\r\n\r\nIn both scenarios, the only dependencies were the pinned python version and the latest Datasette version 0.64.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2067/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 735852274, "node_id": "MDU6SXNzdWU3MzU4NTIyNzQ=", "number": 1082, "title": "DigitalOcean buildpack memory errors for large sqlite db?", "user": {"value": 39538958, "label": "justmars"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2020-11-04T06:35:32Z", "updated_at": "2020-11-04T19:35:44Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "1. Have a sqlite db stored in Dropbox\r\n2. Previously tried the Digital Ocean build pack minimal approach (e.g. Procfile, requirements.txt, bin/post_compile)\r\n3. bin/post_compile with wget from Dropbox\r\n4. download of large sqlite db is successful \r\n5. log reveals that when building Docker container, Digital Ocean runs out of memory for 5gb+ sqlite db but works fine for 2gb+ sqlite db", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1082/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}