{"html_url": "https://github.com/simonw/datasette/issues/1655#issuecomment-1062445113", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1655", "id": 1062445113, "node_id": "IC_kwDOBm6k_c4_U6A5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-09T01:01:24Z", "updated_at": "2022-03-09T01:01:24Z", "author_association": "OWNER", "body": "https://labordata.bunkum.us/-/settings shows `max_returned_rows` had been increased to 5,000 for that instance - the default of 1,000 would help a bit here.\r\n\r\nAny thoughts on how Datasette could handle this kind of thing better?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1163369515, "label": "query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1655#issuecomment-1062450649", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1655", "id": 1062450649, "node_id": "IC_kwDOBm6k_c4_U7XZ", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2022-03-09T01:10:46Z", "updated_at": "2022-03-09T01:10:46Z", "author_association": "CONTRIBUTOR", "body": "i increased the max_returned_row, because I have some scripts that get CSVs from this site, and this makes doing pagination of CSVs less annoying for many cases. i know that's streaming csvs is something you are hoping to address in 1.0. let me know if there's anything i can do to help with that.\r\n\r\nas for what if anything can be done about the size of the dom, I don't have any ideas right now, but i'll poke around.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1163369515, "label": "query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1655#issuecomment-1258166572", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1655", "id": 1258166572, "node_id": "IC_kwDOBm6k_c5K_hks", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2022-09-26T14:57:04Z", "updated_at": "2022-09-26T14:57:04Z", "author_association": "CONTRIBUTOR", "body": "I think that paginating, even in javascript, could be very helpful. Maybe render json or csv into the page and let javascript loading that into the dom?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1163369515, "label": "query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1655#issuecomment-1761630595", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1655", "id": 1761630595, "node_id": "IC_kwDOBm6k_c5pAFmD", "user": {"value": 6262071, "label": "yejiyang"}, "created_at": "2023-10-13T14:37:48Z", "updated_at": "2023-10-13T14:37:48Z", "author_association": "NONE", "body": "Hi @fgregg, I came across this issue and found your setup at labordata.bunkum.us can help me with a research project at https://database.zeropm.eu/. \r\nI really like the approach [here](https://labordata.bunkum.us/f7-06c761c?sql=select+*+from+f7) when dealing with a custom SQL query returning more than 1000 rows:\r\n1) At the table in HTML page, only first 1000 rows displayed;\r\n2) When click the \"Download this data as a CSV Spreadsheet(All Rows)\" button, a csv with ALL ROWS (could be > 100 Mb) get downloaded.\r\n\r\nI am trying to repeat the setup but have yet to be successful so far. \r\nWhat I tried:\r\n1) copy the query.html & table.html templates from this [github repo](https://github.com/labordata/warehouse/tree/main/templates) and use it my project\r\n2) use the same datasette version 1.0a2.\r\n\r\nDo you know what else I should try to set it correctly? I appreciate your help.\r\n\r\n@simonw I would like to use this opportunity to thank you for developing & maintaining such an amazing project. \r\nI introduce your datasette to several projects in my institute. I am also interested in your cloud version.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1163369515, "label": "query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1655#issuecomment-1766994810", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1655", "id": 1766994810, "node_id": "IC_kwDOBm6k_c5pUjN6", "user": {"value": 536941, "label": "fgregg"}, "created_at": "2023-10-17T19:01:59Z", "updated_at": "2023-10-17T19:01:59Z", "author_association": "CONTRIBUTOR", "body": "hi @yejiyang, have your tried using my fork of datasette: https://github.com/fgregg/datasette/tree/no_limit_csv_publish\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1163369515, "label": "query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1655#issuecomment-1767133832", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1655", "id": 1767133832, "node_id": "IC_kwDOBm6k_c5pVFKI", "user": {"value": 6262071, "label": "yejiyang"}, "created_at": "2023-10-17T20:37:18Z", "updated_at": "2023-10-17T21:12:48Z", "author_association": "NONE", "body": "@fgregg Thanks for your reply. I tried to use your fork branch `datasette = {url = \"https://github.com/fgregg/datasette/archive/refs/heads/no_limit_csv_publish.zip\"}` and got error - TypeError: 'str' object is not callable. I used the same templates as in your branch [here ](https://github.com/labordata/warehouse/tree/main/templates).\r\n\r\n```\r\nINFO: 127.0.0.1:47232 - \"GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1\" 200 OK\r\nTraceback (most recent call last):\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/datasette/app.py\", line 1632, in route_path\r\n response = await view(request, send)\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/datasette/app.py\", line 1814, in async_view_fn\r\n response = await async_call_with_supported_arguments(\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/datasette/utils/__init__.py\", line 1016, in async_call_with_supported_arguments\r\n return await fn(*call_with)\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/datasette/views/table.py\", line 673, in table_view\r\n response = await table_view_traced(datasette, request)\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/datasette/views/table.py\", line 822, in table_view_traced\r\n await datasette.render_template(\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/datasette/app.py\", line 1307, in render_template\r\n return await template.render_async(template_context)\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/jinja2/environment.py\", line 1324, in render_async\r\n return self.environment.handle_exception()\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/jinja2/environment.py\", line 936, in handle_exception\r\n raise rewrite_traceback_stack(source=source)\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/jinja2/environment.py\", line 1321, in \r\n [n async for n in self.root_render_func(ctx)] # type: ignore\r\n File \"templates/table.html\", line 1, in top-level template code\r\n {% extends \"base.html\" %}\r\n File \"/home/jiyang/github/global-chemical-inventory-database/.venv/lib/python3.10/site-packages/datasette/templates/base.html\", line 62, in top-level template code\r\n {% block content %}\r\n File \"templates/table.html\", line 24, in block 'content'\r\n