issue_comments

Relevant issue: - https://github.com/simonw/datasette/issues/1773

Explains this comment: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L697

Found more logic relating to this:

https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L684-L732

Crucial utility function: https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/init.py#L137-L160

I should turn sort and sort_desc into an object representing the sort order earlier in the code.

I should also create something that bundles together pks and use_rowid and maybe is_view as well.

The inputs and outputs for this are pretty complex.

Inputs:

• ?_next= from the query string
• is_view - is this for a table or view? If it's a view it uses offset/limit pagination - which could actually work for arbitrary queries too. Also views could have keyset pagination if they are known to be sorted by a particular column.
• sort and sort_desc reflecting the current sort order
• use_rowid for if the table is a rowid table with no primary key of its own
• pks - the primary keys for the table
• params - the current set of parameters, I think used just to count their length so new params can be added as p5 etc without collisions. This could be handled with a s0, s1 etc naming convention instead.

Outputs:

• where_clauses - a list of where clauses to add to the query
• params - additional parameters to use with the query due to the new where clauses
• order_by - the order by clause

HTML mode needs a list of renderers so it can show links to .geojson etc - can do that as a hidden extra (maybe called renderers), repeating this code:

https://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/views/base.py#L477-L497

I think that does make sense: ?_extra=table perhaps, which would add {"table": "..."}.

I think it's not currently possible: sqlite-utils requires that it be one of integer, text, float, blob (see code)

IMO, this is a bit of friction and it would be nice if it was more permissive. SQLite permits developers to use any data type when creating a table. For example, this is a perfectly cromulent sqlite session that creates a table with columns of type baz and bar:

``` sqlite> create table foo(column1 baz, column2 bar); sqlite> .schema foo CREATE TABLE foo(column1 baz, column2 bar); sqlite> select * from pragma_table_info('foo'); cid name type notnull dflt_value pk

0 column1 baz 0 0
1 column2 bar 0 0
```

The idea is that the application developer will know what meaning to ascribe to those types. For example, I'm working on a plugin to Datasette. Dates are tricky to handle. If you have some existing rows, you can look at the values in them to know how a user is serializing the dates -- as an ISO 8601 string? An RFC 3339 string? With millisecond precision? With timezone offset? But if you don't yet have any rows, you have to guess. If the column is of type TEXT, you don't even know that it's meant to hold a date! In this case, my plugin will look to see if the column is of type DATE or DATETIME, and assume a certain representation when writing.

Perhaps there is an argument that sqlite-utils is trying to conform to SQLite's strict mode, and that is why it limits the choices. In strict mode, SQLite requires that the data type be one of INT, INTEGER, REAL, TEXT, BLOB, ANY. But that can't be the case -- sqlite-utils supports FLOAT, which is not one of the valid types in strict mode, and it rejects INT, REAL and ANY, which are valid.

That said, it looks like the check is only enforced at the CLI level. If you use the API directly, I think it'll work.

SQLite doesn't have a native DATETIME type. It stores dates internally as strings and then has functions to work with date-like strings. Yes it's weird.

That's correct. But my issue is about the application level libraries that, i suppose, have better data understanding if see a specific type such as DATETIME.

I'm writing data with dataset i've mentioned. The lib changes its behavior depending on a type. I saw different behavior with types DATETIME, FLOAT, TEXT. Dataset, for their part, is built upon Sqlalchemy, you know what it is.

To be honest, i didn't dive into the details of why the behavior changes, but when i altered manually by other util a type of column to DATETIME things got back to normal.

On the matter, can i achieve it with Sqlite Utils at the moment?

SQLite doesn't have a native DATETIME type. It stores dates internally as strings and then has functions to work with date-like strings. Yes it's weird.