issue_comments

Table actions

4 rows where issue = 1174655187 and "updated_at" is on date 2022-03-21 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply · 4 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1074478299 https://github.com/simonw/datasette/issues/1671#issuecomment-1074478299 https://api.github.com/repos/simonw/datasette/issues/1671 IC_kwDOBm6k_c5ACzzb simonw 9599 2022-03-21T22:20:26Z 2022-03-21T22:20:26Z OWNER

Thinking about options for fixing this...

The following query works fine: sql select * from test_view where cast(has_expired as text) = '1' I don't want to start using this for every query, because one of the goals of Datasette is to help people who are learning SQL: - #1613

If someone clicks on "View and edit SQL" from a filtered table page I don't want them to have to wonder why that cast is there.

But... for querying views, the cast turns out to be necessary.

So one fix would be to get the SQL generating logic to use casts like this any time it is operating against a view.

An even better fix would be to detect which columns in a view come from a table and which ones might not, and only use casts for the columns that aren't definitely from a table.

The trick I was exploring here might be able to help with that: - #1293 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187  
1074470568 https://github.com/simonw/datasette/issues/1671#issuecomment-1074470568 https://api.github.com/repos/simonw/datasette/issues/1671 IC_kwDOBm6k_c5ACx6o simonw 9599 2022-03-21T22:11:14Z 2022-03-21T22:12:49Z OWNER

I wonder if this will be a problem with generated columns, or with SQLite strict tables?

My hunch is that strict tables will continue to work without any changes, because https://www.sqlite.org/stricttables.html says nothing about their impact on comparison operations. I should test this to make absolutely sure though.

Generated columns have a type, so my hunch is they will continue to work fine too.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187  
1074468450 https://github.com/simonw/datasette/issues/1671#issuecomment-1074468450 https://api.github.com/repos/simonw/datasette/issues/1671 IC_kwDOBm6k_c5ACxZi simonw 9599 2022-03-21T22:08:35Z 2022-03-21T22:10:00Z OWNER

Relevant section of the SQLite documentation: 3.2. Affinity Of Expressions:

When an expression is a simple reference to a column of a real table (not a VIEW or subquery) then the expression has the same affinity as the table column.

In your example, has_expired is no longer a simple reference to a column of a real table, hence the bug.

Then 4.2. Type Conversions Prior To Comparison fills in the rest:

SQLite may attempt to convert values between the storage classes INTEGER, REAL, and/or TEXT before performing a comparison. Whether or not any conversions are attempted before the comparison takes place depends on the type affinity of the operands. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187  
1074465536 https://github.com/simonw/datasette/issues/1671#issuecomment-1074465536 https://api.github.com/repos/simonw/datasette/issues/1671 IC_kwDOBm6k_c5ACwsA simonw 9599 2022-03-21T22:04:31Z 2022-03-21T22:04:31Z OWNER

Oh this is fascinating! I replicated the bug (thanks for the steps to reproduce) and it looks like this is down to the following:

Against views, where has_expired = 1 returns different results from where has_expired = '1'

This doesn't happen against tables because of SQLite's type affinity mechanism, which handles the type conversion automatically.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);