{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524637376", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524637376, "node_id": "IC_kwDOBm6k_c5a4B7A", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T03:50:19Z", "updated_at": "2023-04-27T03:50:19Z", "author_association": "OWNER", "body": "Having trouble replicating this on my laptop. I tried a fresh virtual environment with fresh packages (in case this is a `httpx` change) but this passed:\r\n\r\n pytest tests/test_csv.py", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524638233", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524638233, "node_id": "IC_kwDOBm6k_c5a4CIZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T03:50:51Z", "updated_at": "2023-04-27T03:50:51Z", "author_association": "OWNER", "body": "Failure was on 3.7. I'll try that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524648995", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524648995, "node_id": "IC_kwDOBm6k_c5a4Ewj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T03:56:42Z", "updated_at": "2023-04-27T03:57:11Z", "author_association": "OWNER", "body": "I don't have 3.7 locally. Trying to install it with `pyenv`.\r\n\r\n brew install pyenv\r\n\r\nThen:\r\n\r\n pyenv install --list | grep 3.7\r\n\r\nInstalling:\r\n\r\n pyenv install 3.7.16\r\n\r\nOutput:\r\n\r\n Installed Python-3.7.16 to /Users/simon/.pyenv/versions/3.7.16\r\n\r\nSo the executable is `/Users/simon/.pyenv/versions/3.7.16/bin/python`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524655203", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524655203, "node_id": "IC_kwDOBm6k_c5a4GRj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T03:59:56Z", "updated_at": "2023-04-27T03:59:56Z", "author_association": "OWNER", "body": "In a fresh Datasette checkout I ran:\r\n\r\n pipenv shell --python /Users/simon/.pyenv/versions/3.7.16/bin/python\r\n\r\nThat gave me a virtual environment with 3.7.16 Python.\r\n\r\nThen I ran:\r\n\r\n pip install -e '.[test]'\r\n\r\nWeirdly that gave me a `which pytest` of `/opt/homebrew/bin/pytest` which ran the tests on 3.11.\r\n\r\nI figured out the location of the virtual environment with `which python` and then ran this:\r\n\r\n```\r\n% /Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/bin/pytest tests/test_csv.py \r\n============================================================================================== test session starts ===============================================================================================\r\nplatform darwin -- Python 3.7.16, pytest-7.3.1, pluggy-1.0.0\r\nSQLite: 3.39.5\r\nrootdir: /private/tmp/datasette\r\nconfigfile: pytest.ini\r\nplugins: asyncio-0.21.0, timeout-2.1.0, xdist-3.2.1, anyio-3.6.2\r\nasyncio: mode=strict\r\ncollected 15 items \r\n\r\ntests/test_csv.py ..........F.... [100%]\r\n\r\n==================================================================================================== FAILURES ====================================================================================================\r\n________________________________________________________________________________________________ test_max_csv_mb _________________________________________________________________________________________________\r\n\r\napp_client_csv_max_mb_one = \r\n\r\n def test_max_csv_mb(app_client_csv_max_mb_one):\r\n response = app_client_csv_max_mb_one.get(\r\n (\r\n \"/fixtures.csv?sql=select+'{}'+\"\r\n \"from+compound_three_primary_keys&_stream=1&_size=max\"\r\n> ).format(\"abcdefg\" * 10000)\r\n )\r\n\r\n/private/tmp/datasette/tests/test_csv.py:161: \r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:240: in __call__\r\n return call_result.result()\r\n/Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:428: in result\r\n return self.__get_result()\r\n/Users/simon/.pyenv/versions/3.7.16/lib/python3.7/concurrent/futures/_base.py:384: in __get_result\r\n raise self._exception\r\n/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/asgiref/sync.py:306: in main_wrap\r\n result = await self.awaitable(*args, **kwargs)\r\n/private/tmp/datasette/datasette/utils/testing.py:76: in get\r\n headers=headers,\r\n/private/tmp/datasette/datasette/utils/testing.py:167: in _request\r\n content=post_body,\r\n/private/tmp/datasette/datasette/app.py:1787: in request\r\n method, self._fix(path, avoid_path_rewrites), **kwargs\r\n/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:1528: in request\r\n extensions=extensions,\r\n/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:346: in build_request\r\n url = self._merge_url(url)\r\n/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_client.py:376: in _merge_url\r\n merge_url = URL(url)\r\n/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urls.py:113: in __init__\r\n self._uri_reference = urlparse(url, **kwargs)\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \r\n\r\nurl = \"http://localhost/fixtures.csv?sql=select+'abcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcde...gabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefgabcdefg'+from+compound_three_primary_keys&_stream=1&_size=max\"\r\nkwargs = {}\r\n\r\n def urlparse(url: str = \"\", **kwargs: typing.Optional[str]) -> ParseResult:\r\n # Initial basic checks on allowable URLs.\r\n # ---------------------------------------\r\n \r\n # Hard limit the maximum allowable URL length.\r\n if len(url) > MAX_URL_LENGTH:\r\n> raise InvalidURL(\"URL too long\")\r\nE httpx.InvalidURL: URL too long\r\n\r\n/Users/simon/.local/share/virtualenvs/datasette-cZYvnUqY/lib/python3.7/site-packages/httpx/_urlparse.py:155: InvalidURL\r\n============================================================================================ short test summary info =============================================================================================\r\nFAILED tests/test_csv.py::test_max_csv_mb - httpx.InvalidURL: URL too long\r\n========================================================================================== 1 failed, 14 passed in 1.83s ==========================================================================================\r\n(datasette) simon@Simons-MacBook-Pro datasette % \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524659084", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524659084, "node_id": "IC_kwDOBm6k_c5a4HOM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T04:02:07Z", "updated_at": "2023-04-27T04:02:07Z", "author_association": "OWNER", "body": "This is the failing test:\r\n\r\nhttps://github.com/simonw/datasette/blob/249fcf8e3e2a90e763f41b080c1b9ec8017f5005/tests/test_csv.py#L156-L167", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524660603", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524660603, "node_id": "IC_kwDOBm6k_c5a4Hl7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T04:02:55Z", "updated_at": "2023-04-27T04:02:55Z", "author_association": "OWNER", "body": "In the debugger:\r\n```\r\n(Pdb) MAX_URL_LENGTH\r\n65536\r\n```\r\nWeird this only seems to be a problem with `httpx` on Python 3.7 though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524666049", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524666049, "node_id": "IC_kwDOBm6k_c5a4I7B", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T04:06:18Z", "updated_at": "2023-04-27T04:06:18Z", "author_association": "OWNER", "body": "Most recent `httpx` release is 0.24 a couple of weeks ago. Here's what changed in that version: https://github.com/encode/httpx/compare/0.23.3...0.24.0\r\n\r\nIt looks like that URL limit is new: https://github.com/encode/httpx/commit/57daabf673705954afa94686c0002801c93d31f3#diff-78d8d93b5dd4c77d99c3e2b46b7286ba71a8fd60e92d8bd4eee5fb200b4f87bfR149-R155\r\n\r\n```python\r\ndef urlparse(url: str = \"\", **kwargs: typing.Optional[str]) -> ParseResult:\r\n # Initial basic checks on allowable URLs.\r\n # ---------------------------------------\r\n\r\n # Hard limit the maximum allowable URL length.\r\n if len(url) > MAX_URL_LENGTH:\r\n raise InvalidURL(\"URL too long\")\r\n```\r\nhttps://github.com/encode/httpx/blob/32e25497a36e6222cc64a758c98275b450dac28d/httpx/_urlparse.py#L153-L155", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524669124", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524669124, "node_id": "IC_kwDOBm6k_c5a4JrE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T04:10:44Z", "updated_at": "2023-04-27T04:10:52Z", "author_association": "OWNER", "body": "I need an alternative way of generating a long string with a shorter URL.\r\n\r\n```sql\r\nselect group_concat('abcabcabc', '') from json_each(json_array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10))\r\n```\r\nhttps://latest.datasette.io/_memory?sql=select+group_concat%28%27abcabcabc%27%2C+%27%27%29+from+json_each%28json_array%281%2C+2%2C+3%2C+4%2C+5%2C+6%2C+7%2C+8%2C+9%2C+10%29%29", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524675817", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524675817, "node_id": "IC_kwDOBm6k_c5a4LTp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T04:21:03Z", "updated_at": "2023-04-27T04:21:03Z", "author_association": "OWNER", "body": "I went with this to generate the long string: https://github.com/simonw/datasette/blob/0b0c5cd7a94fe3f151a3e10261b5c84ee64f2f18/tests/test_csv.py#L157-L176", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/2066#issuecomment-1524680160", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2066", "id": 1524680160, "node_id": "IC_kwDOBm6k_c5a4MXg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-04-27T04:27:50Z", "updated_at": "2023-04-27T04:27:50Z", "author_association": "OWNER", "body": "That fixed it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1686042269, "label": "Failing test: httpx.InvalidURL: URL too long"}, "performed_via_github_app": null}