{"html_url": "https://github.com/simonw/datasette/issues/1519#issuecomment-974701788", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1519", "id": 974701788, "node_id": "IC_kwDOBm6k_c46GMTc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-20T19:42:29Z", "updated_at": "2021-11-20T19:42:29Z", "author_association": "OWNER", "body": "> I think what's happening here is Apache is actually making a request to `/fixtures` rather than making a request to `/prefix/fixtures` - and Datasette is replying to requests on both the prefixed and the non-prefixed paths.\r\n> \r\n> This is pretty confusing! I think Datasette should ONLY reply to `/prefix/fixtures` instead and return a 404 for `/fixtures` - this would make things a whole lot easier to debug.\r\n> \r\n> But shipping that change could break existing deployments. Maybe that should be a breaking change for 1.0.\r\n\r\nOn further thought I'm not going to do this. Having Datasette work behind a proxy the way it does right now is clearly easy for people to deploy (now that I've fixed the bugs) and I trust my improved tests to catch problems in the future.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1058790545, "label": "base_url is omitted in JSON and CSV views"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1519#issuecomment-974697824", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1519", "id": 974697824, "node_id": "IC_kwDOBm6k_c46GLVg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-20T19:11:21Z", "updated_at": "2021-11-20T19:11:21Z", "author_association": "OWNER", "body": "OK, i think I got all of them this time!\r\n\r\nThe latest demo is now live at https://datasette-apache-proxy-demo.fly.dev/prefix/fixtures/sortable?_facet=pk2\r\n\r\nI'm closing this issue, but feel free to re-open it if you spot any that I missed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1058790545, "label": "base_url is omitted in JSON and CSV views"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1519#issuecomment-974562942", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1519", "id": 974562942, "node_id": "IC_kwDOBm6k_c46FqZ-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-20T00:59:32Z", "updated_at": "2021-11-20T00:59:32Z", "author_association": "OWNER", "body": "Ouch a nasty bug crept through there - https://datasette-apache-proxy-demo-j7hipcg4aq-uc.a.run.app/prefix/fixtures/compound_three_primary_keys says\r\n\r\n> 500: name 'ds' is not defined", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1058790545, "label": "base_url is omitted in JSON and CSV views"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1519#issuecomment-974561593", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1519", "id": 974561593, "node_id": "IC_kwDOBm6k_c46FqE5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-20T00:53:19Z", "updated_at": "2021-11-20T00:53:19Z", "author_association": "OWNER", "body": "Adding that test found (I hope!) all of the remaining `base_url` bugs. There were a bunch! I think I finally get to close #838 too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1058790545, "label": "base_url is omitted in JSON and CSV views"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1519#issuecomment-974559176", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1519", "id": 974559176, "node_id": "IC_kwDOBm6k_c46FpfI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-20T00:42:08Z", "updated_at": "2021-11-20T00:42:08Z", "author_association": "OWNER", "body": "> In the meantime I can catch these errors by changing the test to run each path twice, once with and once without the prefix. This should accurately simulate how Apache is working here.\r\n\r\nThis worked, I managed to get the tests to fail! Here's the change I made:\r\n\r\n```diff\r\ndiff --git a/tests/test_html.py b/tests/test_html.py\r\nindex f24165b..dbdfe59 100644\r\n--- a/tests/test_html.py\r\n+++ b/tests/test_html.py\r\n@@ -1614,12 +1614,19 @@ def test_metadata_sort_desc(app_client):\r\n         \"/fixtures/compound_three_primary_keys/a,a,a\",\r\n         \"/fixtures/paginated_view\",\r\n         \"/fixtures/facetable\",\r\n+        \"/fixtures?sql=select+1\",\r\n     ],\r\n )\r\n-def test_base_url_config(app_client_base_url_prefix, path):\r\n+@pytest.mark.parametrize(\"use_prefix\", (True, False))\r\n+def test_base_url_config(app_client_base_url_prefix, path, use_prefix):\r\n     client = app_client_base_url_prefix\r\n-    response = client.get(\"/prefix/\" + path.lstrip(\"/\"))\r\n+    path_to_get = path\r\n+    if use_prefix:\r\n+        path_to_get = \"/prefix/\" + path.lstrip(\"/\")\r\n+    response = client.get(path_to_get)\r\n     soup = Soup(response.body, \"html.parser\")\r\n+    if path == \"/fixtures?sql=select+1\":\r\n+        assert False\r\n     for el in soup.findAll([\"a\", \"link\", \"script\"]):\r\n         if \"href\" in el.attrs:\r\n             href = el[\"href\"]\r\n@@ -1642,11 +1649,12 @@ def test_base_url_config(app_client_base_url_prefix, path):\r\n             # If this has been made absolute it may start http://localhost/\r\n             if href.startswith(\"http://localhost/\"):\r\n                 href = href[len(\"http://localost/\") :]\r\n-            assert href.startswith(\"/prefix/\"), {\r\n+            assert href.startswith(\"/prefix/\"), json.dumps({\r\n                 \"path\": path,\r\n+                \"path_to_get\":  path_to_get,\r\n                 \"href_or_src\": href,\r\n                 \"element_parent\": str(el.parent),\r\n-            }\r\n+            }, indent=4, default=repr)\r\n \r\n \r\n def test_base_url_affects_metadata_extra_css_urls(app_client_base_url_prefix):\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1058790545, "label": "base_url is omitted in JSON and CSV views"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1519#issuecomment-974558267", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1519", "id": 974558267, "node_id": "IC_kwDOBm6k_c46FpQ7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-11-20T00:37:57Z", "updated_at": "2021-11-20T00:37:57Z", "author_association": "OWNER", "body": "Thanks to #1522 I have a live demo that exhibits this bug now: https://apache-proxy-demo.datasette.io/prefix/fixtures/attraction_characteristic", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1058790545, "label": "base_url is omitted in JSON and CSV views"}, "performed_via_github_app": null}