{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-736030599", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 736030599, "node_id": "MDEyOklzc3VlQ29tbWVudDczNjAzMDU5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T20:41:41Z", "updated_at": "2020-11-30T20:41:41Z", "author_association": "OWNER", "body": "Here's the problem: https://www.sqlite.org/changes.html#version_3_26_0\r\n\r\n> ### 2018-12-01 (3.26.0)\r\n> \r\n> -  Added [PRAGMA table_xinfo](https://www.sqlite.org/pragma.html#pragma_table_xinfo) that works just like [PRAGMA table_info](https://www.sqlite.org/pragma.html#pragma_table_info) except that it also shows [hidden columns](https://www.sqlite.org/vtab.html#hiddencol) in virtual tables.\r\n\r\nCI is running 3.22.0.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-736015487", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 736015487, "node_id": "MDEyOklzc3VlQ29tbWVudDczNjAxNTQ4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T20:11:07Z", "updated_at": "2020-11-30T20:11:07Z", "author_association": "OWNER", "body": "Working on this in a pull request: https://github.com/simonw/datasette/pull/1117", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-736014372", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 736014372, "node_id": "MDEyOklzc3VlQ29tbWVudDczNjAxNDM3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T20:08:48Z", "updated_at": "2020-11-30T20:08:48Z", "author_association": "OWNER", "body": "Ouch, the tests pass on my laptop but failed in CI: https://github.com/simonw/datasette/actions/runs/392367997\r\n\r\nLots of failures look like this:\r\n```\r\nERROR: conn=<sqlite3.Connection object at 0x7f44f0494030>, sql = 'select rowid,  from facetable order by rowid limit 51', params = {}: near \"from\": syntax error\r\n```\r\nNote the `select rowid, from...` - so it looks like invalid SQL queries are being constructed maybe due to mis-detecting columns somehow.\r\n\r\nI wonder why it didn't fail on my laptop?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-736010720", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 736010720, "node_id": "MDEyOklzc3VlQ29tbWVudDczNjAxMDcyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T20:01:53Z", "updated_at": "2020-11-30T20:01:53Z", "author_association": "OWNER", "body": "I'm OK exposing hidden columns, unless someone comes up with a pressing reason not to.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-736005833", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 736005833, "node_id": "MDEyOklzc3VlQ29tbWVudDczNjAwNTgzMw==", "user": {"value": 2789593, "label": "nattaylor"}, "created_at": "2020-11-30T19:54:39Z", "updated_at": "2020-11-30T19:54:39Z", "author_association": "NONE", "body": "@simonw thanks for investigating so quickly.  If it is undesirable to change that hidden behavior, maybe something like this is a suitable workaround:\r\n```\r\nSELECT * FROM pragma_table_xinfo('deeds') where hidden in (0,2);\r\n0|body|TEXT|0||0|0\r\n1|id|INT GENERATED ALWAYS|0||0|2\r\n2|consideration|INT GENERATED ALWAYS|0||0|2\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-736004383", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 736004383, "node_id": "MDEyOklzc3VlQ29tbWVudDczNjAwNDM4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T19:51:51Z", "updated_at": "2020-11-30T19:51:51Z", "author_association": "OWNER", "body": "This change will also have an impact on how hidden virtual FTS tables are displayed, since apparently those have some hidden columns: https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_xinfo%28%27searchable_fts%27%29\r\n\r\n| cid | name | type | notnull | dflt_value | pk | hidden |\r\n| --- | --- | --- | --- | --- | --- | --- |\r\n| 0 | text1 |   | 0 |   | 0 | 0 |\r\n| 1 | text2 |   | 0 |   | 0 | 0 |\r\n| 2 | name with . and spaces |   | 0 |   | 0 | 0 |\r\n| 3 | searchable_fts |   | 0 |   | 0 | 1 |\r\n| 4 | docid |   | 0 |   | 0 | 1 |\r\n| 5 | __langid |   | 0 |   | 0 | 1 |", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-735995695", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 735995695, "node_id": "MDEyOklzc3VlQ29tbWVudDczNTk5NTY5NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T19:34:15Z", "updated_at": "2020-11-30T19:34:15Z", "author_association": "OWNER", "body": "Generated column support was added in SQLite 3.31.0, so any unit tests I write for this should use skipIf to only run on that version or later.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-735993935", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 735993935, "node_id": "MDEyOklzc3VlQ29tbWVudDczNTk5MzkzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T19:30:44Z", "updated_at": "2020-11-30T19:32:15Z", "author_association": "OWNER", "body": "It looks like `PRAGMA table_info` skips \"hidden\" columns: https://www.sqlite.org/pragma.html#pragma_table_info\r\n\r\nBut `PRAGMA table_xinfo` does not: https://www.sqlite.org/pragma.html#pragma_table_xinfo\r\n\r\nCompare https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_info%28%27searchable%27%29 to https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_xinfo%28%27searchable%27%29 - the `xinfo` one has an additional `hidden` column.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1116#issuecomment-735992106", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1116", "id": 735992106, "node_id": "MDEyOklzc3VlQ29tbWVudDczNTk5MjEwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-11-30T19:27:10Z", "updated_at": "2020-11-30T19:27:10Z", "author_association": "OWNER", "body": "I'm treating this as a bug - these columns should definitely be visible in Datasette.\r\n\r\nI created my own test database using SQLite from Homebrew like this:\r\n```\r\n/usr/local/Cellar/sqlite/3.33.0/bin/sqlite3 deeds.db << EOF\r\nCREATE TABLE deeds (\r\n    body TEXT,\r\n    id INT GENERATED ALWAYS AS (json_extract(body, '$.id')) STORED,\r\n    consideration INT GENERATED ALWAYS AS (json_extract(body, '$.consideration')) STORED\r\n);\r\nINSERT INTO deeds (body) VALUES ('{\r\n    \"id\": 1,\r\n    \"consideration\": \"This is the consideration\"\r\n}');\r\nEOF\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 753668177, "label": "GENERATED column support"}, "performed_via_github_app": null}