{"html_url": "https://github.com/simonw/datasette/issues/316#issuecomment-491533700", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/316", "id": 491533700, "node_id": "MDEyOklzc3VlQ29tbWVudDQ5MTUzMzcwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2019-05-11T18:26:25Z", "updated_at": "2019-05-11T18:26:25Z", "author_association": "OWNER", "body": "This will be fixed by #419 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 333238932, "label": "datasette inspect takes a very long time on large dbs"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/316#issuecomment-398133159", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/316", "id": 398133159, "node_id": "MDEyOklzc3VlQ29tbWVudDM5ODEzMzE1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-18T17:29:59Z", "updated_at": "2018-07-10T15:14:53Z", "author_association": "OWNER", "body": "For #271 I've been contemplating having Datasette work against an on-disk database that gets modified without needing to restart the server.\r\n\r\nFor that to work, I'll have to dramatically change the inspect() mechanism. It may be that inspect becomes an optional optimization in the future.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 333238932, "label": "datasette inspect takes a very long time on large dbs"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/316#issuecomment-398109204", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/316", "id": 398109204, "node_id": "MDEyOklzc3VlQ29tbWVudDM5ODEwOTIwNA==", "user": {"value": 132230, "label": "gavinband"}, "created_at": "2018-06-18T16:12:45Z", "updated_at": "2018-06-18T16:12:45Z", "author_association": "NONE", "body": "Hi Simon,\r\nThanks for the response.  Ok I'll try running `datasette inspect` up front.\r\nIn principle the db won't change.  However, the site's in development and it's likely I'll need to add views and some auxiliary (smaller) tables as I go along.  I will need to be careful with this if it involves an inspect step in each iteration, though.\r\ng.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 333238932, "label": "datasette inspect takes a very long time on large dbs"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/316#issuecomment-398101670", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/316", "id": 398101670, "node_id": "MDEyOklzc3VlQ29tbWVudDM5ODEwMTY3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2018-06-18T15:49:35Z", "updated_at": "2018-06-18T15:50:38Z", "author_association": "OWNER", "body": "Wow, I've gone as high as 7GB but I've never tried it against 600GB.\r\n\r\n`datasette inspect` is indeed expected to take a long time for large databases. That's why it's available as a separate command: by running `datasette inspect` to generate `inspect-data.json` you can execute it just once against a large database and then have `datasette serve` take advantage of that cached metadata (hence avoiding `datasette serve` hanging on startup).\r\n\r\nAs you spotted, most of the time is spent in those counts. I imagine you don't need those row counts in order for the rest of Datasette to function correctly (they are mainly used for display purposes - on the https://latest.datasette.io/fixtures index page for example).\r\n\r\nIf your database changes infrequently, for the moment I recommend running `datasette inspect` once to generate the `inspect-data.json` file (let me know how long it takes) and then passing that file to `datasette serve mydb.db --inspect-file=inspect-data.json`\r\n\r\nIf your database DOES change frequently then this workaround won't help you much. Let me know and I'll see how much work it would take to have those row counts be optional rather than required.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 333238932, "label": "datasette inspect takes a very long time on large dbs"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/316#issuecomment-398030903", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/316", "id": 398030903, "node_id": "MDEyOklzc3VlQ29tbWVudDM5ODAzMDkwMw==", "user": {"value": 132230, "label": "gavinband"}, "created_at": "2018-06-18T12:00:43Z", "updated_at": "2018-06-18T12:00:43Z", "author_association": "NONE", "body": "I should add that I'm using datasette version 0.22, Python 2.7.10 on Mac OS X.  Happy to send more info if helpful.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 333238932, "label": "datasette inspect takes a very long time on large dbs"}, "performed_via_github_app": null}