github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/datasette/issues/420#issuecomment-474407617 | https://api.github.com/repos/simonw/datasette/issues/420 | 474407617 | MDEyOklzc3VlQ29tbWVudDQ3NDQwNzYxNw== | 9599 | 2019-03-19T14:55:51Z | 2019-03-19T14:55:51Z | OWNER | A microbenchmark against `fivethirtyeight.db` (415 tables): In [1]: import sqlite3 In [2]: c = sqlite3.connect("fivethirtyeight.db") In [3]: %timeit c.execute("select name from sqlite_master where type = 'table'").fetchall() 283 µs ± 12.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) In [4]: tables = [r[0] for r in c.execute("select name from sqlite_master where type = 'table'").fetchall()] In [5]: len(tables) Out[5]: 415 In [6]: %timeit [c.execute("pragma foreign_keys([{}])".format(t)).fetchall() for t in tables] 1.81 ms ± 161 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) So running `pragma foreign_keys()` against 415 tables only takes 1.81ms. This is going to be fine. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
421971339 | |
https://github.com/simonw/datasette/issues/420#issuecomment-474399630 | https://api.github.com/repos/simonw/datasette/issues/420 | 474399630 | MDEyOklzc3VlQ29tbWVudDQ3NDM5OTYzMA== | 9599 | 2019-03-19T14:38:14Z | 2019-03-19T14:38:14Z | OWNER | Most of these can be replaced with relatively straight-forward direct introspection of the SQLite table. The one exception is the incoming foreign keys: these can only be found by inspecting ALL of the other tables. This requires running `PRAGMA foreign_key_list([table_name])` against every other table in the database. How expensive is doing this on a database with hundreds of tables? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
421971339 | |
https://github.com/simonw/datasette/issues/420#issuecomment-474398127 | https://api.github.com/repos/simonw/datasette/issues/420 | 474398127 | MDEyOklzc3VlQ29tbWVudDQ3NDM5ODEyNw== | 9599 | 2019-03-19T14:34:55Z | 2019-03-19T14:34:55Z | OWNER | I systematically reviewed the codebase for things that `.inspect()` is used for: In `app.py`: * `table_exists()` uses `table in self.inspect().get(database, {}).get("tables")` * `.execute()` looks up the database name to get the `info["file"]` (the correct filename with the `.db` extension) In `cli.py`: * The `datasette inspect` command dumps it to JSON * `datasette skeleton` iterates over it * `datasette serve` calls it on startup (to populate static cache of inspect data) In `base.py`: * `.database_url(database)` calls it to lookup the hash (if `hash_urls` config turned on) * `.resolve_db_name()` uses it to lookup the hash In `database.py`: * `DatabaseView` uses it to find up the list of tables and views to display, plus the size of the DB file in bytes * `DatabaseDownload` uses it to get the filepath for download In `index.py`: * `IndexView` uses it _extensively_ - to loop through every database and every table. This would make a good starting point for the refactor. In `table.py`: * `sortable_columns_for_table()` uses it to find the columns in a table * `expandable_columns()` uses it to find foreign keys * `expand_foreign_keys()` uses it to find foreign keys * `display_columns_and_rows()` uses it to find primary keys and foreign keys... but also has access to a `cursor.description` which it uses to list the columns * `TableView.data` uses it to lookup columns and primary keys and the `table_rows_count` (used if the thing isn't a view) and probably a few more things, this method is huge! * `RowView.data` uses it for primary keys * `foreign_key_tables()` uses it for foreign keys In the tests it's used by `test_api.test_inspect_json()` and by a couple of tests in `test_inspect`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
421971339 |