{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065597709", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065597709, "node_id": "IC_kwDOCGYnMM4_g7sN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T22:32:43Z", "updated_at": "2022-03-11T22:32:43Z", "author_association": "OWNER", "body": "Trying to figure out what that extra field in `table_info` compared to `table_xinfo` is:\r\n```\r\n>>> list(db.query(\"PRAGMA table_xinfo('t')\"))\r\n[{'cid': 0,\r\n  'name': 'body',\r\n  'type': 'TEXT',\r\n  'notnull': 0,\r\n  'dflt_value': None,\r\n  'pk': 0,\r\n  'hidden': 0},\r\n {'cid': 1,\r\n  'name': 'd',\r\n  'type': 'INT',\r\n  'notnull': 0,\r\n  'dflt_value': None,\r\n  'pk': 0,\r\n  'hidden': 2}]\r\n``\r\nPresumably `hidden` 0 v.s 2 v.s. other values has meaning.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065596417", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065596417, "node_id": "IC_kwDOCGYnMM4_g7YB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T22:30:15Z", "updated_at": "2022-03-11T22:30:15Z", "author_association": "OWNER", "body": "I tried it out in Jupyter and it works as advertised:\r\n\r\n<img width=\"1076\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/157980825-b362a162-b774-4fb5-8ec0-a7f539e62bd7.png\">\r\n\r\nIntrospection is a bit weird: there doesn't seem to be a way to introspect generated columns outside of parsing the stored SQL schema for the columns at the moment! And the `.columns` method doesn't return them at all:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/433813612ff9b4b501739fd7543bef0040dd51fe/sqlite_utils/db.py#L1207-L1213\r\n\r\nHere's why:\r\n\r\n```\r\n>>> db.execute(\"PRAGMA table_info('t')\").fetchall()\r\n[(0, 'body', 'TEXT', 0, None, 0)]\r\n>>> db.execute(\"PRAGMA table_xinfo('t')\").fetchall()\r\n[(0, 'body', 'TEXT', 0, None, 0, 0), (1, 'd', 'INT', 0, None, 0, 2)]\r\n```\r\nSo `table_xinfo()` is needed to get back columns including generated columns: https://www.sqlite.org/pragma.html#pragma_table_xinfo\r\n\r\n> **PRAGMA** *schema.***table_xinfo(***table-name***);**\r\n>\r\n> This pragma returns one row for each column in the named table, including [hidden columns](https://www.sqlite.org/vtab.html#hiddencol) in virtual tables. The output is the same as for [PRAGMA table_info](https://www.sqlite.org/pragma.html#pragma_table_info) except that hidden columns are shown rather than being omitted.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065402557", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065402557, "node_id": "IC_kwDOCGYnMM4_gMC9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T19:01:08Z", "updated_at": "2022-03-11T21:42:25Z", "author_association": "OWNER", "body": "Just spotted this in https://www.sqlite.org/gencol.html\r\n\r\n> The only functional difference is that one cannot add new STORED columns using the [ALTER TABLE ADD COLUMN](https://www.sqlite.org/lang_altertable.html#altertabaddcol) command. Only VIRTUAL columns can be added using ALTER TABLE.\r\n\r\nSo to add stored columns to an existing table we would need to use the `.transform()` trick. Which implies that this should actually be a capability of the various `.create()` methods, since transform works by creating a new table with those and then copying across the old data.\r\n\r\nHere's where `.transform()` calls `.create_table_sql()` under the hood:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/9388edf57aa15719095e3cf0952c1653cd070c9b/sqlite_utils/db.py#L1627-L1637", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065389386", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065389386, "node_id": "IC_kwDOCGYnMM4_gI1K", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T18:42:53Z", "updated_at": "2022-03-11T21:40:51Z", "author_association": "OWNER", "body": "The Python API could be:\r\n```python\r\ndb[table_name].add_generated_column(\"field\", str, \"json_extract(data, '$.field')\", stored=True)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065458729", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065458729, "node_id": "IC_kwDOCGYnMM4_gZwp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T19:58:50Z", "updated_at": "2022-03-11T20:00:25Z", "author_association": "OWNER", "body": "I'm coming round to your suggestion to have this as extra arguments to `sqlite-utils add-column` now, especially since you also need to pass a column type.\r\n\r\nI'd like to come up with syntax for `sqlite-utils create-table` as well.\r\n\r\nhttps://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-table\r\n\r\nMaybe extra `--generated-stored colname expression` (and `--generated`) options would work there.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065440445", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065440445, "node_id": "IC_kwDOCGYnMM4_gVS9", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T19:52:15Z", "updated_at": "2022-03-11T19:52:15Z", "author_association": "OWNER", "body": "Two new parameters to `.create_table()` and friends:\r\n\r\n - `generated={...}` - generated column definitions\r\n - `generated_stored={...}` generated stored column definitions\r\n \r\nThese columns will be added at the end of the table, but you can use the `column_order=` parameter to apply a different order.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/411#issuecomment-1065386352", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411", "id": 1065386352, "node_id": "IC_kwDOCGYnMM4_gIFw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-11T18:41:37Z", "updated_at": "2022-03-11T18:41:37Z", "author_association": "OWNER", "body": "I like `add-generated-column` - feels very clear to me, and is a nice place for adding logic that checks if the DB version supports it or not and shows a useful error.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1160034488, "label": "Support for generated columns"}, "performed_via_github_app": null}