{"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705889120", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705889120, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTg4OTEyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-08T23:59:01Z", "updated_at": "2020-10-08T23:59:01Z", "author_association": "OWNER", "body": "Needs tests and documentation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705899629", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705899629, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTg5OTYyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:37:02Z", "updated_at": "2020-10-09T00:37:02Z", "author_association": "OWNER", "body": "I'm going to route the existing `TestClient` through this mechanism to exercise it during the tests.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705902902", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705902902, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkwMjkwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T00:50:49Z", "updated_at": "2020-10-09T00:50:49Z", "author_association": "OWNER", "body": "Almost all of the tests are passing:\r\n```\r\n=========================== short test summary info ============================\r\nFAILED tests/test_api.py::test_table_with_slashes_in_name - assert 404 == 200\r\nFAILED tests/test_api.py::test_row_strange_table_name - assert 404 == 200\r\nFAILED tests/test_html.py::test_row_strange_table_name_with_url_hash - assert...\r\nFAILED tests/test_html.py::test_css_classes_on_body[/fixtures/table%2Fwith%2Fslashes.csv-expected_classes5]\r\nFAILED tests/test_html.py::test_templates_considered[/fixtures/table%2Fwith%2Fslashes.csv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys/a,a,a-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/paginated_view-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/facetable-https://example.com/]\r\nFAILED tests/test_messages.py::test_messages_are_displayed_and_cleared - KeyE...\r\nFAILED tests/test_plugins.py::test_hook_register_magic_parameters - Assertion...\r\n============ 11 failed, 718 passed, 6 warnings in 225.77s (0:03:45) ============\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705918844", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705918844, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkxODg0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:46:06Z", "updated_at": "2020-10-09T01:46:06Z", "author_association": "OWNER", "body": "For this failing test I'm suspicious that the AsyncClient may be persisting cookies in between requests:\r\n```\r\n def test_actor_cookie(app_client):\r\n \"A valid actor cookie sets request.scope['actor']\"\r\n cookie = app_client.actor_cookie({\"id\": \"test\"})\r\n response = app_client.get(\"/\", cookies={\"ds_actor\": cookie})\r\n> assert {\"id\": \"test\"} == app_client.ds._last_request.scope[\"actor\"]\r\nE AssertionError: assert {'id': 'test'} == {'id': 'root'}\r\nE Differing items:\r\nE {'id': 'test'} != {'id': 'root'}\r\nE Use -v to get the full diff\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705920055", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705920055, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyMDA1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:51:05Z", "updated_at": "2020-10-09T01:51:05Z", "author_association": "OWNER", "body": "The topic of disabling cookie persistence is discussed a little here: https://github.com/encode/httpx/issues/422#issuecomment-537906693\r\n\r\n> We have a cookiejar abstraction, I think setting it to an always-empty jar like you describe is best. :)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705920228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705920228, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyMDIyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:51:44Z", "updated_at": "2020-10-09T01:51:44Z", "author_association": "OWNER", "body": "I'm going to switch back to having each request run through a new client. I'm worried about the impact on test performance though. I'll run a microbenchmark before and after.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705921006", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705921006, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyMTAwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T01:55:01Z", "updated_at": "2020-10-09T01:55:01Z", "author_association": "OWNER", "body": "With the single client that is reused for all tests:\r\n```\r\n% time pytest tests/test_api.py\r\n...\r\n6.73s user 9.91s system 81% cpu 20.365 total\r\n```\r\nAfter switching back to this class:\r\n```python\r\nclass DatasetteClient:\r\n def __init__(self, ds):\r\n self.app = ds.app()\r\n\r\n def _fix(self, path):\r\n if path.startswith(\"/\"):\r\n path = \"http://localhost{}\".format(path)\r\n return path\r\n\r\n async def get(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.get(self._fix(path), **kwargs)\r\n\r\n async def options(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.options(self._fix(path), **kwargs)\r\n\r\n async def head(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.head(self._fix(path), **kwargs)\r\n\r\n async def post(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.post(self._fix(path), **kwargs)\r\n\r\n async def put(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.put(self._fix(path), **kwargs)\r\n\r\n async def patch(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.patch(self._fix(path), **kwargs)\r\n\r\n async def delete(self, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.delete(self._fix(path), **kwargs)\r\n\r\n async def request(self, method, path, **kwargs):\r\n async with httpx.AsyncClient(app=self.app) as client:\r\n return await client.request(method, self._fix(path), **kwargs)\r\n```\r\nThe time taken is:\r\n```\r\n% time pytest tests/test_api.py\r\n...\r\n7.26s user 10.02s system 82% cpu 21.014 total\r\n```\r\nThat's close enough that I don't feel I need to investigate this further.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705926035", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705926035, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkyNjAzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T02:14:14Z", "updated_at": "2020-10-09T02:14:14Z", "author_association": "OWNER", "body": "Still need to handle these six failing tests:\r\n```\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', 'elemen...\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys/a,a,a-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', '...\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/paginated_view-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', 'element_parent': '<...\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/facetable-https://example.com/] - AssertionError: {'base_url': 'https://example.com/', 'element_parent': '
FAILED tests/test_messages.py::test_messages_are_displayed_and_cleared - KeyError: 'ds_messages'\r\n\r\nThat one is caused by `response.cookies` skipping cookies that were set to the empty string. Same fix as this: https://github.com/simonw/datasette/blob/a1687351fb75b01f737fda4ad07e0781029de05c/tests/test_auth.py#L90-L95", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705937696", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705937696, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTkzNzY5Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T02:52:53Z", "updated_at": "2020-10-09T02:52:53Z", "author_association": "OWNER", "body": "These failures are giving me a severe \"how did this ever work in the first place?\" vibe:\r\n```\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/compound_three_primary_keys/a,a,a-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/paginated_view-https://example.com/]\r\nFAILED tests/test_html.py::test_base_url_config[/fixtures/facetable-https://example.com/]\r\n```\r\nI have a fix for them, no idea why they weren't already failing though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705940507", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705940507, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0MDUwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:04:15Z", "updated_at": "2020-10-09T03:04:15Z", "author_association": "OWNER", "body": "This is really weird: new set of test failures that I wasn't seeing before, and those tests aren't failing on my laptop:\r\n```\r\n=========================== short test summary info ============================\r\nFAILED tests/test_api.py::test_table_with_slashes_in_name - assert 404 == 200\r\nFAILED tests/test_api.py::test_row_strange_table_name - assert 404 == 200\r\nFAILED tests/test_html.py::test_row_strange_table_name_with_url_hash - assert...\r\nFAILED tests/test_html.py::test_css_classes_on_body[/fixtures/table%2Fwith%2Fslashes.csv-expected_classes5]\r\nFAILED tests/test_html.py::test_templates_considered[/fixtures/table%2Fwith%2Fslashes.csv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html]\r\n================== 5 failed, 738 passed in 194.73s (0:03:14) ===================\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705941580", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705941580, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0MTU4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:08:43Z", "updated_at": "2020-10-09T03:08:43Z", "author_association": "OWNER", "body": "Most likely reason for those failures is that `path` and `raw_path` are not being simulated correctly. I used to do those here:\r\n\r\nhttps://github.com/simonw/datasette/blob/402cf870b7d65f9b5fba9e23aa99433294bd4523/datasette/utils/testing.py#L116-L125\r\n\r\nBut now I'm delegating that to `httpx` to handle.\r\n\r\nWEIRD that it passes on my laptop but fails in GitHub Actions CI though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 717746043, "label": "datasette.client internal requests mechanism"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1000#issuecomment-705945591", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1000", "id": 705945591, "node_id": "MDEyOklzc3VlQ29tbWVudDcwNTk0NTU5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-10-09T03:24:48Z", "updated_at": "2020-10-09T03:24:48Z", "author_association": "OWNER", "body": "I'm testing this with a `print(scope)` and `pytest -k test_table_with_slashes_in_name -s`.\r\n\r\nAgainst the `main` branch:\r\n\r\n`{'type': 'http', 'http_version': '1.0', 'method': 'GET', 'path': '/fixtures/table/with/slashes.csv', 'raw_path': b'/fixtures/table%2Fwith%2Fslashes.csv', 'query_string': b'_shape=objects&_format=json', 'headers': [[b'host', b'localhost']], 'csrftoken':