{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1075432283, "node_id": "IC_kwDOBm6k_c5AGctb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-22T17:39:04Z", "updated_at": "2022-03-22T17:43:12Z", "author_association": "OWNER", "body": "Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like `>`:\r\n\r\nhttps://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/datasette/filters.py#L203-L212\r\n\r\nThough... it looks like there's a bug in that? It doesn't account for `float` values - `\"3.5\".isdigit()` return `False` - probably for the best, because `int(3.5)` would break that value anyway.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1075435185", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1075435185, "node_id": "IC_kwDOBm6k_c5AGdax", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-22T17:42:09Z", "updated_at": "2022-03-22T17:42:09Z", "author_association": "OWNER", "body": "Also made me realize that this query:\r\n```sql\r\nselect * from sortable where sortable > :p0\r\n```\r\nOnly works here thanks to the column affinity thing kicking in too: https://latest.datasette.io/fixtures?sql=select+*+from+sortable+where+sortable+%3E+%3Ap0&p0=70", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1075428030", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1075428030, "node_id": "IC_kwDOBm6k_c5AGbq-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-22T17:34:30Z", "updated_at": "2022-03-22T17:34:30Z", "author_association": "OWNER", "body": "No, I think I need to use `cast` - I can't think of any way to ask SQLite \"for this query, what types are the columns that will come back from it?\"\r\n\r\nEven the details from the `explain` trick explored in #1293 don't seem to come back with column type information: https://latest.datasette.io/fixtures?sql=explain+select+pk%2C+text1%2C+text2%2C+[name+with+.+and+spaces]+from+searchable_view+where+%22pk%22+%3D+%3Ap0&p0=1", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1075425513", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1075425513, "node_id": "IC_kwDOBm6k_c5AGbDp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-22T17:31:53Z", "updated_at": "2022-03-22T17:31:53Z", "author_association": "OWNER", "body": "The alternative to using `cast` here would be for Datasette to convert the `\"1\"` to a `1` in Python code before passing it as a param.\r\n\r\nThis feels a bit neater to me, but I still then need to solve the problem of how to identify the \"type\" of a column that I want to use in a query.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1074478299", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1074478299, "node_id": "IC_kwDOBm6k_c5ACzzb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-21T22:20:26Z", "updated_at": "2022-03-21T22:20:26Z", "author_association": "OWNER", "body": "Thinking about options for fixing this...\r\n\r\nThe following query works fine:\r\n```sql\r\nselect * from test_view where cast(has_expired as text) = '1'\r\n```\r\nI don't want to start using this for every query, because one of the goals of Datasette is to help people who are learning SQL:\r\n- #1613\r\n\r\nIf someone clicks on \"View and edit SQL\" from a filtered table page I don't want them to have to wonder why that `cast` is there.\r\n\r\nBut... for querying views, the `cast` turns out to be necessary.\r\n\r\nSo one fix would be to get the SQL generating logic to use casts like this any time it is operating against a view.\r\n\r\nAn even better fix would be to detect which columns in a view come from a table and which ones might not, and only use casts for the columns that aren't definitely from a table.\r\n\r\nThe trick I was exploring here might be able to help with that:\r\n- #1293 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1074470568", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1074470568, "node_id": "IC_kwDOBm6k_c5ACx6o", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-21T22:11:14Z", "updated_at": "2022-03-21T22:12:49Z", "author_association": "OWNER", "body": "I wonder if this will be a problem with generated columns, or with SQLite strict tables?\r\n\r\nMy hunch is that strict tables will continue to work without any changes, because https://www.sqlite.org/stricttables.html says nothing about their impact on comparison operations. I should test this to make absolutely sure though.\r\n\r\nGenerated columns have a type, so my hunch is they will continue to work fine too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1074468450", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1074468450, "node_id": "IC_kwDOBm6k_c5ACxZi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-21T22:08:35Z", "updated_at": "2022-03-21T22:10:00Z", "author_association": "OWNER", "body": "Relevant section of the SQLite documentation: [3.2. Affinity Of Expressions](https://www.sqlite.org/datatype3.html#affinity_of_expressions):\r\n\r\n> When an expression is a simple reference to a column of a real table (not a [VIEW](https://www.sqlite.org/lang_createview.html) or subquery) then the expression has the same affinity as the table column.\r\n\r\nIn your example, `has_expired` is no longer a simple reference to a column of a real table, hence the bug.\r\n\r\nThen [4.2. Type Conversions Prior To Comparison](https://www.sqlite.org/datatype3.html#type_conversions_prior_to_comparison) fills in the rest:\r\n\r\n> SQLite may attempt to convert values between the storage classes INTEGER, REAL, and/or TEXT before performing a comparison. Whether or not any conversions are attempted before the comparison takes place depends on the type affinity of the operands. ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1671#issuecomment-1074465536", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1671", "id": 1074465536, "node_id": "IC_kwDOBm6k_c5ACwsA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-03-21T22:04:31Z", "updated_at": "2022-03-21T22:04:31Z", "author_association": "OWNER", "body": "Oh this is fascinating! I replicated the bug (thanks for the steps to reproduce) and it looks like this is down to the following:\r\n\r\n<img width=\"1276\" alt=\"image\" src=\"https://user-images.githubusercontent.com/9599/159370986-1e2fc513-6d6c-4a2f-96dd-dccf5a680fe4.png\">\r\n\r\nAgainst views, `where has_expired = 1` returns different results from `where has_expired = '1'`\r\n\r\nThis doesn't happen against tables because of SQLite's [type affinity](https://www.sqlite.org/datatype3.html#type_affinity) mechanism, which handles the type conversion automatically.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1174655187, "label": "Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply"}, "performed_via_github_app": null}