issue_comments

This code works:

python import csv import sqlite_utils db = sqlite_utils.Database("test.db") reader = csv.DictReader(open("csv", encoding="utf-16-le").read().split("\r\n"), delimiter=";") db["test"].insert_all(reader, pk="Id")

I used iconv to change the encoding; sqlite-utils can import the resulting file, even if it stops at 98 %:

sqlite-utils insert --csv test test.db clean [------------------------------------] 0% [###################################-] 98% 00:00:00

I can confirm. This only happens with sqlite-utils. I am using gnome-terminal with bash.

Argument were specified in the wrong order. PATH TABLE FILE can be misleading :)

I tried to debug it using pdb, but it looks sqlite-utils catches the exception, so it is not quick to figure out where the failure is happening.

Sorry, I got confused because SQLite has a JSON column type, even if it is treated as TEXT, and I though automatic facets were available for JSON arrays stored as JSON only :)

There is a feature for this at the moment, but it's a little bit hidden: you can use ?_json=col to tell Datasette that you would like a specific column to be exported as nested JSON: https://docs.datasette.io/en/stable/json_api.html#special-json-arguments

Thanks :)

I considered trying to make this automatic - so it detects columns that appear to contain valid JSON and outputs them as nested objects - but the problem with that is that it can lead to inconsistent results - you might hit the API and find that not every column contains valid JSON (compared to the previous day) resulting in the API retuning string instead of the expected dictionary and breaking your code.

If a developer is not sure if the JSON fields are valid, but then retrieves and parse them, it should handle errors too. Handling inconsistent data is necessary due to the nature of SQLite. A global or dataset option to render the data as they have been defined (JSON, boolean, etc.) when requesting JSON could allow the user to download a regular JSON from the browser without having to rely on APIs. I would guess someone could just make a custom template with an extra JSON-parsed download button otherwise :)

@mroswell That is a very nice solution. I wonder if custom classes, like col-columnName-value could be automatically added to cells when facets on such column are enabled, to allow custom styling without having to modify templates or add custom JavaScript code.

sqlite-utils transform does not allow setting the column type to boolean: Error: Invalid value for '--type': 'bool' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.

Sure, I am sorry if my message hasn't been clear enough. I am also a new user :)

At the beginning, I just call sqlite-utils insert "$db" "$table" "$jsonfile" to create the database. sqlite-utils convert JSON values into TEXT, when it tries to determine the schema automatically. I then try to transform the table to set JSON as type:

``` sqlite-utils transform species.sqlite species --type criteria json Usage: sqlite-utils transform [OPTIONS] PATH TABLE Try 'sqlite-utils transform --help' for help.

Error: Invalid value for '--type': 'json' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'. ```

true and false json values are cast to integer, which is not optimal.