issue_comments

Table actions

4 rows where issue = 1408757705 and "updated_at" is on date 2022-12-15 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • Intermittent "Too many open files" error running tests · 4 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1353690591 https://github.com/simonw/datasette/issues/1843#issuecomment-1353690591 https://api.github.com/repos/simonw/datasette/issues/1843 IC_kwDOBm6k_c5Qr63f simonw 9599 2022-12-15T20:49:05Z 2022-12-15T20:49:05Z OWNER

I have a nasty feeling the cleaner solution for this would involve porting my entire test suite from def test_blah(app_client) sync functions (which work due to a @async_to_sync call in TestClient) to async def test_blah(async_fixture): functions instead.

I've been using that latter pattern for new tests (and plugin tests) for quite a while now, but I never took on the job of refactoring all of the old ones.

A search for (app_client): across the whole project currently returns 194 results which might be a reasonable target to try switching to the new pattern as a starting point.

No idea if it will have much impact on the "Too many open files" errors though.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent "Too many open files" error running tests 1408757705  
1353522652 https://github.com/simonw/datasette/issues/1843#issuecomment-1353522652 https://api.github.com/repos/simonw/datasette/issues/1843 IC_kwDOBm6k_c5QrR3c simonw 9599 2022-12-15T18:21:27Z 2022-12-15T18:21:27Z OWNER

I'll still use on-disk test databases for is_immutable=True, but not for the majority of tests.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent "Too many open files" error running tests 1408757705  
1353522211 https://github.com/simonw/datasette/issues/1843#issuecomment-1353522211 https://api.github.com/repos/simonw/datasette/issues/1843 IC_kwDOBm6k_c5QrRwj simonw 9599 2022-12-15T18:21:02Z 2022-12-15T18:21:02Z OWNER

When I initially built this test suite Datasette didn't have the memory_name= mechanism for creating persistent in-memory databases.

I'm going to see if I can switch to that for the majority of Datasette's tests. Might find that doing so both fixes this "too many open files" issue AND gives me a significant speed improvement to the test site too!

Hopefully I can do most of the work on that in this big ugly function:

https://github.com/simonw/datasette/blob/dc18f62089e5672d03176f217d7840cdafa5c447/tests/fixtures.py#L104-L173

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent "Too many open files" error running tests 1408757705  
1353520615 https://github.com/simonw/datasette/issues/1843#issuecomment-1353520615 https://api.github.com/repos/simonw/datasette/issues/1843 IC_kwDOBm6k_c5QrRXn simonw 9599 2022-12-15T18:19:25Z 2022-12-15T18:19:25Z OWNER

I've been seeing this error again:

ERROR tests/test_api_write.py::test_create_table[input16-400-expected_response16] - OSError: [Errno 24] Too ... ERROR tests/test_api_write.py::test_create_table[input17-400-expected_response17] - OSError: [Errno 24] Too ... ERROR tests/test_api_write.py::test_create_table[input18-400-expected_response18] - OSError: [Errno 24] Too ...

It doesn't happen in CI, and it turns out that's because CI runs pytest -n auto which splits the tests across multiple parallel processes.

I've been working around the error on my laptop using pytest -n auto there too, but I'd rather not have to do that.

This is also getting in my way when I try to debug other issues, like this one:

  • 1955

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent "Too many open files" error running tests 1408757705  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);