github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1024#issuecomment-709625063 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709625063 | MDEyOklzc3VlQ29tbWVudDcwOTYyNTA2Mw== | 9599 | 2020-10-15T22:33:22Z | 2020-10-15T22:33:22Z | OWNER | Of those errors... `http://localhost:8000/robots.txt` 404 is fine. `http://localhost:8000/datasette/%5C%22https://www.openstreetmap.org/copyright%5C%22` looks to me like a `wget` parsing bug where it got confused by this JavaScript: ``` window.DATASETTE_CLUSTER_MAP_TILE_LAYER_OPTIONS = {"maxZoom": 19, "detectRetina": true, "attribution": "© <a href=\"https://www.openstreetmap.org/copyright\">OpenStreetMap</a> contributors"};</script> ``` `http://localhost:8000/-/static-plugins/datasette_cluster_map/datasette-cluster-map.js` is a real bug. It's a bug in `datasette-cluster-map` but also requires me to solve #988 - mechanism for plugins to construct URLs that obey `base_url`. I'm not sure why I'm getting a hit to `http://localhost:8000/` since I wouldn't expect to link to `/` anywhere. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709622973 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709622973 | MDEyOklzc3VlQ29tbWVudDcwOTYyMjk3Mw== | 9599 | 2020-10-15T22:27:31Z | 2020-10-15T22:27:31Z | OWNER | Here's how I tested it: ``` time wget -r 'http://localhost:8000/datasette/' 2>&1 | grep -i -C 5 "failed\|error" > /tmp/errors.txt ``` This wrote out any errors (plus context) to the `errors.txt` log - and reported that the full crawl took 33s. Here's what I got in `errors.txt`: ``` 0K . 71.6M=0s 2020-10-15 15:23:09 (71.6 MB/s) - ‘localhost:8000/datasette/index.html’ saved [1276] Loading robots.txt; please ignore errors. --2020-10-15 15:23:09-- http://localhost:8000/robots.txt Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found -- --2020-10-15 15:23:09-- http://localhost:8000/robots.txt Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found 2020-10-15 15:23:09 ERROR 404: Not Found. --2020-10-15 15:23:09-- http://localhost:8000/datasette/-/static/app.css?b576be Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 200 OK Length: 8563 (8.4K) [text/css] -- -- 2020-10-15 15:23:13 (7.90 MB/s) - ‘localhost:8000/datasette/fixtures/primary_key_multiple_columns_explicit_label.json?_shape=object’ saved [58] --2020-10-15 15:23:13-- http://localhost:8000/-/static-plugins/datasette_cluster_map/datasette-cluster-map.js Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 404 Not Found 2020-10-15 15:23:13 ERROR 404: Not Found. --2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures?sql=select+pk%2C+name%2C+address%2C+latitude%2C+longitude+from+roadside_attractions+order+by+pk+limit+101 Reusing existing connection to localhost:8000. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] -- -- 2020-10-15 15:23:13 (84.3 MB/s) - ‘localhost:8000/datasette/fixtures/roadside_attractions.json?_shape=object’ saved [619] --2020-10-15 15:23:13-- http://localhost:8000/datasette/fixtures/%5C%22https://www.openstreetmap.org/copyright%5C%… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709600335 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709600335 | MDEyOklzc3VlQ29tbWVudDcwOTYwMDMzNQ== | 9599 | 2020-10-15T21:28:02Z | 2020-10-15T22:25:43Z | OWNER | This is working OK so far: <img width="897" alt="fixtures__compound_three_primary_keys__1_001_rows_where_sorted_by_pk2" src="https://user-images.githubusercontent.com/9599/96187878-96371980-0ef2-11eb-875e-5b1ae9b445fd.png"> I'll try crawling it with `wget -r` to see if I get any errors. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709598324 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709598324 | MDEyOklzc3VlQ29tbWVudDcwOTU5ODMyNA== | 9599 | 2020-10-15T21:23:33Z | 2020-10-15T21:26:55Z | OWNER | Combining these two examples, here's the config file I am going to use for this. I'll save this as `nginx.conf`: ``` daemon off; events { worker_connections 1024; } http { server { listen 8000; location /datasette { proxy_pass http://127.0.0.1:8001; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } } ``` Then start the server with: ``` nginx -p `pwd` -c `pwd`/nginx.conf ``` And start Datasette like this: ``` datasette fixtures.db --config base_url:/datasette/ ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709597589 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709597589 | MDEyOklzc3VlQ29tbWVudDcwOTU5NzU4OQ== | 9599 | 2020-10-15T21:21:53Z | 2020-10-15T21:23:25Z | OWNER | Here's a recipe for running nginx against a custom config file: https://gist.github.com/simonw/35f0ebf9c1d6df158759 ``` daemon off; events { worker_connections 1024; } http { access_log /dev/stdout; error_log /dev/stderr; types { text/html html htm shtml; text/css css; image/gif gif; image/jpeg jpeg jpg; application/javascript js; } server { listen 8002; index index.html; root app; } } ``` ``` nginx -p `pwd` -c `pwd`/nginx.conf ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709595960 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709595960 | MDEyOklzc3VlQ29tbWVudDcwOTU5NTk2MA== | 9599 | 2020-10-15T21:18:14Z | 2020-10-15T21:18:14Z | OWNER | Typing `nginx` starts it running as a daemon listening on port `http-alt` aka 8080. It uses the config file from ` /usr/local/etc/nginx/nginx.conf`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709590941 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709590941 | MDEyOklzc3VlQ29tbWVudDcwOTU5MDk0MQ== | 9599 | 2020-10-15T21:07:47Z | 2020-10-15T21:07:47Z | OWNER | On macOS I ran `brew install nginx`. I'm going to try running it on port 8000 so I don't have to run it as root. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709590337 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709590337 | MDEyOklzc3VlQ29tbWVudDcwOTU5MDMzNw== | 9599 | 2020-10-15T21:06:24Z | 2020-10-15T21:07:19Z | OWNER | From https://stackoverflow.com/questions/32549684/nginx-proxy-and-remove-proxy-pass-prefix/32550251 it looks like the config I should use is: ``` server { listen 80; server_name example.com; location /datasette/ { proxy_pass http://127.0.0.1:8001; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_read_timeout 90; } } ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 | |
https://github.com/simonw/datasette/issues/1024#issuecomment-709589297 | https://api.github.com/repos/simonw/datasette/issues/1024 | 709589297 | MDEyOklzc3VlQ29tbWVudDcwOTU4OTI5Nw== | 9599 | 2020-10-15T21:04:31Z | 2020-10-15T21:04:31Z | OWNER | I think nginx or Apache would be the best tools for this. I'm inclined to try with nginx first since I know it better. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
722674708 |