github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/1426#issuecomment-902263367 | https://api.github.com/repos/simonw/datasette/issues/1426 | 902263367 | IC_kwDOBm6k_c41x3JH | 9599 | 2021-08-19T21:33:51Z | 2021-08-19T21:36:28Z | OWNER | I was worried about if it's possible to allow access to `/fixtures` but deny access to `/fixtures?sql=...` From various answers on Stack Overflow it looks like this should handle that: ``` User-agent: * Disallow: /fixtures? ``` I could use this for tables too - it may well be OK to access table index pages while still avoiding pagination, facets etc. I think this should block both query strings and row pages while allowing the table page itself: ``` User-agent: * Disallow: /fixtures/searchable? Disallow: /fixtures/searchable/* ``` Could even accompany that with a `sitemap.xml` that explicitly lists all of the tables - which would mean adding sitemaps to Datasette core too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
964322136 | |
https://github.com/simonw/datasette/issues/1426#issuecomment-902260338 | https://api.github.com/repos/simonw/datasette/issues/1426 | 902260338 | IC_kwDOBm6k_c41x2Zy | 9599 | 2021-08-19T21:28:25Z | 2021-08-19T21:29:40Z | OWNER | Actually it looks like you can send a `sitemap.xml` to Google using an unauthenticated GET request to: https://www.google.com/ping?sitemap=FULL_URL_OF_SITEMAP According to https://developers.google.com/search/docs/advanced/sitemaps/build-sitemap | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
964322136 | |
https://github.com/simonw/datasette/issues/1426#issuecomment-902260799 | https://api.github.com/repos/simonw/datasette/issues/1426 | 902260799 | IC_kwDOBm6k_c41x2g_ | 9599 | 2021-08-19T21:29:13Z | 2021-08-19T21:29:13Z | OWNER | Bing's equivalent is: https://www.bing.com/webmasters/help/Sitemaps-3b5cf6ed http://www.bing.com/ping?sitemap=FULL_URL_OF_SITEMAP | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
964322136 |