6 rows where user = 4068 sorted by updated_at descending

View and edit SQL

Suggested facets: issue_url, reactions, created_at (date), updated_at (date)

user

  • frafra · 6

author_association

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
862574390 https://github.com/simonw/sqlite-utils/issues/270#issuecomment-862574390 https://api.github.com/repos/simonw/sqlite-utils/issues/270 MDEyOklzc3VlQ29tbWVudDg2MjU3NDM5MA== frafra 4068 2021-06-16T17:34:49Z 2021-06-16T17:34:49Z NONE

Sorry, I got confused because SQLite has a JSON column type, even if it is treated as TEXT, and I though automatic facets were available for JSON arrays stored as JSON only :)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Cannot set type JSON 919314806  
860548546 https://github.com/simonw/datasette/issues/1375#issuecomment-860548546 https://api.github.com/repos/simonw/datasette/issues/1375 MDEyOklzc3VlQ29tbWVudDg2MDU0ODU0Ng== frafra 4068 2021-06-14T09:41:59Z 2021-06-14T09:41:59Z NONE

There is a feature for this at the moment, but it's a little bit hidden: you can use ?_json=col to tell
Datasette that you would like a specific column to be exported as nested JSON: https://docs.datasette.io/en/stable/json_api.html#special-json-arguments

Thanks :)

I considered trying to make this automatic - so it detects columns that appear to contain valid JSON and outputs them as nested objects - but the problem with that is that it can lead to inconsistent results - you might hit the API and find that not every column contains valid JSON (compared to the previous day) resulting in the API retuning string instead of the expected dictionary and breaking your code.

If a developer is not sure if the JSON fields are valid, but then retrieves and parse them, it should handle errors too. Handling inconsistent data is necessary due to the nature of SQLite. A global or dataset option to render the data as they have been defined (JSON, boolean, etc.) when requesting JSON could allow the user to download a regular JSON from the browser without having to rely on APIs. I would guess someone could just make a custom template with an extra JSON-parsed download button otherwise :)

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
JSON export dumps JSON fields as TEXT 919508498  
860047794 https://github.com/simonw/datasette/issues/1286#issuecomment-860047794 https://api.github.com/repos/simonw/datasette/issues/1286 MDEyOklzc3VlQ29tbWVudDg2MDA0Nzc5NA== frafra 4068 2021-06-12T12:36:15Z 2021-06-12T12:36:15Z NONE

@mroswell That is a very nice solution. I wonder if custom classes, like col-columnName-value could be automatically added to cells when facets on such column are enabled, to allow custom styling without having to modify templates or add custom JavaScript code.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Better default display of arrays of items 849220154  
860031217 https://github.com/simonw/sqlite-utils/issues/269#issuecomment-860031217 https://api.github.com/repos/simonw/sqlite-utils/issues/269 MDEyOklzc3VlQ29tbWVudDg2MDAzMTIxNw== frafra 4068 2021-06-12T10:01:53Z 2021-06-12T10:01:53Z NONE

sqlite-utils transform does not allow setting the column type to boolean:

Error: Invalid value for '--type': 'bool' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
bool type not supported 919250621  
860031071 https://github.com/simonw/sqlite-utils/issues/270#issuecomment-860031071 https://api.github.com/repos/simonw/sqlite-utils/issues/270 MDEyOklzc3VlQ29tbWVudDg2MDAzMTA3MQ== frafra 4068 2021-06-12T10:00:24Z 2021-06-12T10:00:24Z NONE

Sure, I am sorry if my message hasn't been clear enough. I am also a new user :)

At the beginning, I just call sqlite-utils insert "$db" "$table" "$jsonfile" to create the database. sqlite-utils convert JSON values into TEXT, when it tries to determine the schema automatically. I then try to transform the table to set JSON as type:

sqlite-utils transform species.sqlite species --type criteria json
Usage: sqlite-utils transform [OPTIONS] PATH TABLE
Try 'sqlite-utils transform --help' for help.

Error: Invalid value for '--type': 'json' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Cannot set type JSON 919314806  
859940977 https://github.com/simonw/sqlite-utils/issues/269#issuecomment-859940977 https://api.github.com/repos/simonw/sqlite-utils/issues/269 MDEyOklzc3VlQ29tbWVudDg1OTk0MDk3Nw== frafra 4068 2021-06-11T22:33:08Z 2021-06-11T22:33:08Z NONE

true and false json values are cast to integer, which is not optimal.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
bool type not supported 919250621  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);