{"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-804261915", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 804261915, "node_id": "MDEyOklzc3VlQ29tbWVudDgwNDI2MTkxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T17:41:12Z", "updated_at": "2021-03-22T17:41:12Z", "author_association": "OWNER", "body": "Closing this because I've figured out the root of the problem now, and I have a potential solution.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803802957", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803802957, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzgwMjk1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T06:38:14Z", "updated_at": "2021-03-22T06:38:14Z", "author_association": "OWNER", "body": "Also worth trying is to change this code:\r\n```python\r\n n = 1000 \r\n if ms < 50: \r\n n = 1 \r\n```\r\nWhat happens with `n = 10` instead?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803784902", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803784902, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc4NDkwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:59:06Z", "updated_at": "2021-03-22T05:59:06Z", "author_association": "OWNER", "body": "Even if I implement that workaround in #1269 I'm concerned that this could still allow users to deliberately crash Datasette (if it's running SpatiaLite 5.0) by executing `select count(*) from SpatialIndex`.\r\n\r\nThat `interrupt` timeout mechanism is worth digging into further.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803782705", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803782705, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc4MjcwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:54:19Z", "updated_at": "2021-03-22T05:54:19Z", "author_association": "OWNER", "body": "Got two new TILs out of this:\r\n\r\n* [Tracing every executed Python statement](https://til.simonwillison.net/python/tracing-every-statement)\r\n* [Running gdb against a Python process in a running Docker container](https://til.simonwillison.net/docker/gdb-python-docker)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803777724", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803777724, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc3NzcyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:42:50Z", "updated_at": "2021-03-22T05:43:23Z", "author_association": "OWNER", "body": "\"tuscany_housenumbers__select___from_sqlite_master_where_sql_like__create_virtual_table__\"\r\n\r\nIf I want to avoid counting virtual tables, I need to detect which tables are virtual tables.\r\n\r\nThe safest way to do this is probably to pull the `sql` for every table and then, in Python, check for values that start with `create virtual table` after converting to lower case, using any number of spaces.\r\n\r\nThis would catch things like ` CREATE virtual TABLE` which might be missed by a SQL `like` query. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803775121", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803775121, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc3NTEyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:36:26Z", "updated_at": "2021-03-22T05:36:26Z", "author_association": "OWNER", "body": "So one fix could be to avoid running counts for anything that turns out to be a virtual table.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803774926", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803774926, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc3NDkyNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:35:56Z", "updated_at": "2021-03-22T05:35:56Z", "author_association": "OWNER", "body": "That's in this code here: https://github.com/simonw/datasette/blob/c4f1ec7f33fd7d5b93f0f895dafb5351cc3bfc5b/datasette/database.py#L221-L241", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803774518", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803774518, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc3NDUxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:34:57Z", "updated_at": "2021-03-22T05:34:57Z", "author_association": "OWNER", "body": "... and sure enough, adding this code fixed the problem:\r\n```diff\r\ndiff --git a/datasette/database.py b/datasette/database.py\r\nindex 3579cce..b466b12 100644\r\n--- a/datasette/database.py\r\n+++ b/datasette/database.py\r\n@@ -224,6 +226,9 @@ class Database:\r\n # Try to get counts for each table, $limit timeout for each count\r\n counts = {}\r\n for table in await self.table_names():\r\n+ if table == \"SpatialIndex\":\r\n+ counts[table] = 0\r\n+ continue\r\n try:\r\n table_count = (\r\n await self.execute(\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803773484", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803773484, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc3MzQ4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:32:29Z", "updated_at": "2021-03-22T05:32:29Z", "author_association": "OWNER", "body": "To figure out which SQL query triggers the problem I added this code to write to a log file:\r\n```python\r\n with sqlite_timelimit(conn, time_limit_ms):\r\n try:\r\n cursor = conn.cursor()\r\n with open(\"/tmp/sql.log\", \"ab\", buffering=0) as fp:\r\n fp.write((\"{}: {}\\n\".format(sql, params)).encode(\"utf-8\"))\r\n cursor.execute(sql, params if params is not None else {})\r\n```\r\nI had to use `ab` binary mode because Python doesn't allow `buffering=0` for non-binary file operations.\r\n\r\nWith the log enabled, I used `docker exec -it 589ae68de943 bash` to attach to the running container and `tail -f /tmp/sql.log` to see the logs. Here's where it broke:\r\n\r\n```\r\nselect count(*) from [idx_civici_geom_parent]: None\r\nselect count(*) from [sqlite_stat1]: None\r\nselect count(*) from [sqlite_stat3]: None\r\nselect count(*) from [SpatialIndex]: None\r\n```\r\nSo attempting to run a `count(*)` against the `SpatialIndex` virtual table is the thing that triggers the bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803764919", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803764919, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc2NDkxOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:11:11Z", "updated_at": "2021-03-22T05:11:11Z", "author_association": "OWNER", "body": "Maybe I could implement SQLite query timeouts using the `interrupt()` method instead of the progress handler hack I'm currently using?\r\n\r\nhttps://stackoverflow.com/questions/43240496/python-sqlite3-how-to-quickly-and-cleanly-interrupt-long-running-query-with-e has some tips.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803764200", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803764200, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc2NDIwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:09:13Z", "updated_at": "2021-03-22T05:09:13Z", "author_association": "OWNER", "body": "I tried building a container where the `conn.set_progress_handler(handler, n)` line was commented out... and it fixed the bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803762969", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803762969, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc2Mjk2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:05:51Z", "updated_at": "2021-03-22T05:05:51Z", "author_association": "OWNER", "body": "I had to run `docker kill 16197781a7b5` to kill the broken container - Ctrl+C in the Datasette console window didn't do anything.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803762609", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803762609, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc2MjYwOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T05:05:00Z", "updated_at": "2021-03-22T05:05:00Z", "author_association": "OWNER", "body": "Using https://til.simonwillison.net/docker/attach-bash-to-running-container - I figured out how to run `gdb`. I had to use `--privileged` here because otherwise `gdb` showed a \"Could not attach to process\" error.\r\n```\r\ndocker exec --privileged -it 16197781a7b5 bash\r\n# apt-get install gdb python3-dbg\r\n# gdb /usr/bin/python3 -p 20\r\n```\r\nThis paused the process. I tried running this:\r\n```\r\n(gdb) py-bt\r\nTraceback (most recent call first):\r\n File \"/usr/lib/python3.8/asyncio/base_events.py\", line 1845, in _run_once\r\n if handle._cancelled:\r\n File \"/usr/lib/python3.8/asyncio/base_events.py\", line 570, in run_forever\r\n self._run_once()\r\n File \"/usr/lib/python3.8/asyncio/base_events.py\", line 603, in run_until_complete\r\n self.run_forever()\r\n File \"/usr/local/lib/python3.8/dist-packages/uvicorn/server.py\", line 49, in run\r\n loop.run_until_complete(self.serve(sockets=sockets))\r\n File \"/usr/local/lib/python3.8/dist-packages/uvicorn/main.py\", line 386, in run\r\n server.run()\r\n File \"/usr/local/lib/python3.8/dist-packages/datasette/cli.py\", line 575, in serve\r\n uvicorn.run(ds.app(), **uvicorn_kwargs)\r\n File \"/usr/local/lib/python3.8/dist-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/usr/local/lib/python3.8/dist-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/usr/local/lib/python3.8/dist-packages/click/core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/usr/local/lib/python3.8/dist-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/usr/local/lib/python3.8/dist-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/usr/local/bin/datasette\", line 8, in \r\n sys.exit(cli())\r\n \r\n File \"/usr/lib/python3.8/trace.py\", line 450, in runctx\r\n exec(cmd, globals, locals)\r\n File \"/usr/lib/python3.8/trace.py\", line 6632, in main\r\n File \"/usr/lib/python3.8/trace.py\", line 756, in \r\n main()\r\n \r\n File \"/usr/lib/python3.8/runpy.py\", line 343, in _run_code\r\n File \"/usr/lib/python3.8/runpy.py\", line 450, in _run_module_as_main\r\n```\r\nNot sure if that's useful or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803759051", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803759051, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1OTA1MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:55:22Z", "updated_at": "2021-03-22T04:55:22Z", "author_association": "OWNER", "body": "So I think there's a bug in the way the `set_progress_handler()` mechanism works when used in conjunction with SpatiaLite 5.0 on Linux.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803758793", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803758793, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1ODc5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:54:32Z", "updated_at": "2021-03-22T04:54:32Z", "author_association": "OWNER", "body": "Hitting http://localhost:8001/tuscany_housenumbers triggers the bug. It gets stuck in a loop that looks like this:\r\n\r\n\"datasette_\u2014_root_16197781a7b5____\u2014_com_docker_cli_\u25c2_docker_run_-it_-p_8001_8001_-v___Dropbox_Development_datasette__mnt_datasette-spatialite_latest_bash_\u2014_195\u00d748_and_getIncidentsGit_\u2014_-zsh_\u2014_162\u00d760\"\r\n\r\nWhich looks to me like this code: https://github.com/simonw/datasette/blob/8e18c7943181f228ce5ebcea48deb59ce50bee1f/datasette/utils/__init__.py#L139-L158", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803758182", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803758182, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1ODE4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:52:15Z", "updated_at": "2021-03-22T04:52:15Z", "author_association": "OWNER", "body": "Hitting http://localhost:8001/ successfully shows the homepage (after a lot more scrolling).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803757746", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803757746, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1Nzc0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:50:40Z", "updated_at": "2021-03-22T04:51:52Z", "author_association": "OWNER", "body": "Here's a fun debugging trick:\r\n\r\n docker run -it -p 8001:8001 -v `pwd`:/mnt datasette-spatialite:latest bash\r\n root@16197781a7b5:/# python3 -m trace --trace $(which datasette) \\\r\n -p 8001 -h 0.0.0.0 /mnt/tuscany_housenumbers.sqlite \\\r\n --load-extension=spatialite\r\n\r\nA huge amount of stuff scrolls past as Datasette starts up, since we are tracing every executed line of Python.\r\n\r\nAfter about a minute it's finished starting and gets to this point:\r\n\r\n```\r\nselectors.py(452): if timeout is None:\r\nselectors.py(454): elif timeout <= 0:\r\nselectors.py(459): timeout = math.ceil(timeout * 1e3) * 1e-3\r\nselectors.py(464): max_ev = max(len(self._fd_to_key), 1)\r\nselectors.py(466): ready = []\r\nselectors.py(467): try:\r\nselectors.py(468): fd_event_list = self._selector.poll(timeout, max_ev)\r\n```\r\nNow I can make some HTTP requests against it.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1268#issuecomment-803756495", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1268", "id": 803756495, "node_id": "MDEyOklzc3VlQ29tbWVudDgwMzc1NjQ5NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-03-22T04:46:04Z", "updated_at": "2021-03-22T04:46:04Z", "author_association": "OWNER", "body": "`gdb` may be able to help debug this: https://www.podoliaka.org/2016/04/10/debugging-cpython-gdb/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 837308703, "label": "Figure out why SpatiaLite 5.0 hangs the database page on Linux"}, "performed_via_github_app": null}