html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/issues/1843#issuecomment-1302679026,https://api.github.com/repos/simonw/datasette/issues/1843,1302679026,IC_kwDOBm6k_c5NpU3y,9599,2022-11-03T21:22:42Z,2022-11-03T21:22:42Z,OWNER,Docs for the new `db.close()` method: https://docs.datasette.io/en/latest/internals.html#db-close,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1408757705,
https://github.com/simonw/datasette/issues/1843#issuecomment-1302678384,https://api.github.com/repos/simonw/datasette/issues/1843,1302678384,IC_kwDOBm6k_c5NpUtw,9599,2022-11-03T21:21:59Z,2022-11-03T21:21:59Z,OWNER,"I added extra debug info to `/-/threads` to see this for myself:

```diff
diff --git a/datasette/app.py b/datasette/app.py
index 02bd38f1..16579e28 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -969,6 +969,13 @@ class Datasette:
             ""threads"": [
                 {""name"": t.name, ""ident"": t.ident, ""daemon"": t.daemon} for t in threads
             ],
+            ""file_connections"": {
+                db.name: [
+                    [dict(r) for r in conn.execute(""pragma database_list"").fetchall()]
+                    for conn in db._all_file_connections
+                ]
+                for db in self.databases.values()
+            },
         }
         # Only available in Python 3.7+
         if hasattr(asyncio, ""all_tasks""):
```
Output after hitting refresh on a few `/fixtures` tables to ensure more threads started:

```
    ""file_connections"": {
        ""_internal"": [],
        ""fixtures"": [
            [
                {
                    ""seq"": 0,
                    ""name"": ""main"",
                    ""file"": ""/Users/simon/Dropbox/Development/datasette/fixtures.db""
                }
            ],
            [
                {
                    ""seq"": 0,
                    ""name"": ""main"",
                    ""file"": ""/Users/simon/Dropbox/Development/datasette/fixtures.db""
                }
            ],
            [
                {
                    ""seq"": 0,
                    ""name"": ""main"",
                    ""file"": ""/Users/simon/Dropbox/Development/datasette/fixtures.db""
                }
            ]
        ]
    },
```
I decided not to ship this feature though as it leaks the names of internal database files.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1408757705,
https://github.com/simonw/datasette/issues/1843#issuecomment-1302634332,https://api.github.com/repos/simonw/datasette/issues/1843,1302634332,IC_kwDOBm6k_c5NpJ9c,9599,2022-11-03T20:34:56Z,2022-11-03T20:34:56Z,OWNER,"Confirmed that calling `conn.close()` on each SQLite file-based connection is the way to fix this problem.

I'm adding a `db.close()` method (sync, not async - I tried async first but it was really hard to cause every thread in the pool to close its threadlocal database connection) which loops through all known open file-based connections and closes them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1408757705,
https://github.com/simonw/datasette/issues/1843#issuecomment-1302574330,https://api.github.com/repos/simonw/datasette/issues/1843,1302574330,IC_kwDOBm6k_c5No7T6,9599,2022-11-03T19:30:22Z,2022-11-03T19:30:22Z,OWNER,"This is affecting me a lot at the moment, on my laptop (runs fine in CI).

Here's a change to `conftest.py` which highlights the problem - it cause a failure the moment there are more than 5 open files according to `psutil`:

```diff
diff --git a/tests/conftest.py b/tests/conftest.py
index f4638a14..21d433c1 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,6 +1,7 @@
 import httpx
 import os
 import pathlib
+import psutil
 import pytest
 import re
 import subprocess
@@ -192,3 +193,8 @@ def ds_unix_domain_socket_server(tmp_path_factory):
     yield ds_proc, uds
     # Shut it down at the end of the pytest session
     ds_proc.terminate()
+
+
+def pytest_runtest_teardown(item: pytest.Item) -> None:
+    open_files = psutil.Process().open_files()
+    assert len(open_files) < 5
```
The first error I get from this with `pytest --pdb -x` is here:

```
tests/test_api.py ............E
>>>>> traceback >>>>>

item = <Function test_sql_time_limit>

    def pytest_runtest_teardown(item: pytest.Item) -> None:
        open_files = psutil.Process().open_files()
>       assert len(open_files) < 5
E       AssertionError: assert 5 < 5
E        +  where 5 = len([popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpfglrt4p2/fixtures.db', fd=14), popenfile(... fd=19), popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmphdi5b250/fixtures.dot.db', fd=20)])

/Users/simon/Dropbox/Development/datasette/tests/conftest.py:200: AssertionError
>>>>> entering PDB >>>>>

>>>>> PDB post_mortem (IO-capturing turned off) >>>>>
> /Users/simon/Dropbox/Development/datasette/tests/conftest.py(200)pytest_runtest_teardown()
-> assert len(open_files) < 5
```
That's this test:

https://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/test_api.py#L656-L673

Which uses this fixture:

https://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/fixtures.py#L228-L231

Which calls this function:

https://github.com/simonw/datasette/blob/2ec5583629005b32cb0877786f9681c5d43ca33f/tests/fixtures.py#L105-L122

So now I'm suspicious that, even though the fixture is meant to be session scoped, the way I'm using `with tempfile.TemporaryDirectory() as tmpdir:` is causing a whole load of files to be created and held open which are not later closed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1408757705,