issue_comments

Table actions

5,814 rows sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

issue

user

author_association

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
860063190 https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-860063190 https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57 MDEyOklzc3VlQ29tbWVudDg2MDA2MzE5MA== stiles 232237 2021-06-12T14:46:44Z 2021-06-12T14:46:44Z NONE

I'm having the same issue (same versions of python and twitter-to-sqlite). It's the user-timeline command. Other commands are working. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Error: Use either --since or --since_id, not both 907645813  
860047794 https://github.com/simonw/datasette/issues/1286#issuecomment-860047794 https://api.github.com/repos/simonw/datasette/issues/1286 MDEyOklzc3VlQ29tbWVudDg2MDA0Nzc5NA== frafra 4068 2021-06-12T12:36:15Z 2021-06-12T12:36:15Z NONE

@mroswell That is a very nice solution. I wonder if custom classes, like col-columnName-value could be automatically added to cells when facets on such column are enabled, to allow custom styling without having to modify templates or add custom JavaScript code.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Better default display of arrays of items 849220154  
860031217 https://github.com/simonw/sqlite-utils/issues/269#issuecomment-860031217 https://api.github.com/repos/simonw/sqlite-utils/issues/269 MDEyOklzc3VlQ29tbWVudDg2MDAzMTIxNw== frafra 4068 2021-06-12T10:01:53Z 2021-06-12T10:01:53Z NONE

sqlite-utils transform does not allow setting the column type to boolean:

Error: Invalid value for '--type': 'bool' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 bool type not supported 919250621  
860031071 https://github.com/simonw/sqlite-utils/issues/270#issuecomment-860031071 https://api.github.com/repos/simonw/sqlite-utils/issues/270 MDEyOklzc3VlQ29tbWVudDg2MDAzMTA3MQ== frafra 4068 2021-06-12T10:00:24Z 2021-06-12T10:00:24Z NONE

Sure, I am sorry if my message hasn't been clear enough. I am also a new user :)

At the beginning, I just call sqlite-utils insert "$db" "$table" "$jsonfile" to create the database. sqlite-utils convert JSON values into TEXT, when it tries to determine the schema automatically. I then try to transform the table to set JSON as type:

sqlite-utils transform species.sqlite species --type criteria json
Usage: sqlite-utils transform [OPTIONS] PATH TABLE
Try 'sqlite-utils transform --help' for help.

Error: Invalid value for '--type': 'json' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Cannot set type JSON 919314806  
859986489 https://github.com/simonw/sqlite-utils/issues/270#issuecomment-859986489 https://api.github.com/repos/simonw/sqlite-utils/issues/270 MDEyOklzc3VlQ29tbWVudDg1OTk4NjQ4OQ== simonw 9599 2021-06-12T02:47:12Z 2021-06-12T02:47:12Z OWNER

Can you expand on what you'd like to change here? The library and CLI tool already allow JSON data to be stored in columns:

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Cannot set type JSON 919314806  
859940977 https://github.com/simonw/sqlite-utils/issues/269#issuecomment-859940977 https://api.github.com/repos/simonw/sqlite-utils/issues/269 MDEyOklzc3VlQ29tbWVudDg1OTk0MDk3Nw== frafra 4068 2021-06-11T22:33:08Z 2021-06-11T22:33:08Z NONE

true and false json values are cast to integer, which is not optimal.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 bool type not supported 919250621  
859898736 https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859898736 https://api.github.com/repos/simonw/sqlite-utils/issues/268 MDEyOklzc3VlQ29tbWVudDg1OTg5ODczNg== simonw 9599 2021-06-11T20:37:44Z 2021-06-11T20:37:44Z OWNER

From the prototype:

% sqlite-utils schema 24ways.db
CREATE TABLE [articles] (
               [title] TEXT  ,
   [contents] TEXT  ,
   [year] TEXT  ,
   [author] TEXT  ,
   [author_slug] TEXT  ,
   [published] TEXT  ,
   [url] TEXT  ,
   [topic] TEXT  
        );
CREATE VIRTUAL TABLE "articles_fts" USING FTS5 (
                title, author, contents,
                content="articles"
            );
CREATE TABLE 'articles_fts_data'(id INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE 'articles_fts_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
CREATE TABLE 'articles_fts_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
CREATE TABLE 'articles_fts_config'(k PRIMARY KEY, v) WITHOUT ROWID;
% sqlite-utils schema 24ways.db | sqlite3 /tmp/boo.db
Error: near line 15: table 'articles_fts_data' already exists
Error: near line 16: table 'articles_fts_idx' already exists
Error: near line 17: table 'articles_fts_docsize' already exists
Error: near line 18: table 'articles_fts_config' already exists

The problem here is that the CREATE VIRTUAL TABLE "articles_fts"... line causes those next four tables to be created - but that means that piping the output of this command into sqlite3 in order to re-create those tables throws errors.

I don't think this matters. I see this tool as more for introspection than for recreating table structures.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 db.schema property and sqlite-utils schema command 919181559  
859895540 https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859895540 https://api.github.com/repos/simonw/sqlite-utils/issues/268 MDEyOklzc3VlQ29tbWVudDg1OTg5NTU0MA== simonw 9599 2021-06-11T20:30:34Z 2021-06-11T20:30:34Z OWNER

You can currently see the sql on the CLI using:

% sqlite-utils rows fixtures.db sqlite_master -c name -c sql
name                                                            sql
--------------------------------------------------------------  ----------------------------------------------------------------------------------------------------------------------------------------------------------------
simple_primary_key                                              CREATE TABLE simple_primary_key (
                                                                  id varchar(30) primary key,
                                                                  content text
                                                                )
sqlite_autoindex_simple_primary_key_1
primary_key_multiple_columns                                    CREATE TABLE primary_key_multiple_columns (
                                                                  id varchar(30) primary key,
                                                                  content text,
                                                                  content2 text
                                                                )
sqlite_autoindex_primary_key_multiple_columns_1
primary_key_multiple_columns_explicit_label                     CREATE TABLE primary_key_multiple_columns_explicit_label (
                                                                  id varchar(30) primary key,
                                                                  content text,
                                                                  content2 text
                                                                )
sqlite_autoindex_primary_key_multiple_columns_explicit_label_1
compound_primary_key                                            CREATE TABLE compound_primary_key (
                                                                  pk1 varchar(30),
                                                                  pk2 varchar(30),
                                                                  content text,
                                                                  PRIMARY KEY (pk1, pk2)
                                                                )
sqlite_autoindex_compound_primary_key_1
compound_three_primary_keys                                     CREATE TABLE compound_three_primary_keys (
                                                                  pk1 varchar(30),
                                                                  pk2 varchar(30),
                                                                  pk3 varchar(30),
                                                                  content text,
                                                                  PRIMARY KEY (pk1, pk2, pk3)
                                                                )
sqlite_autoindex_compound_three_primary_keys_1
foreign_key_references                                          CREATE TABLE foreign_key_references (
                                                                  pk varchar(30) primary key,
                                                                  foreign_key_with_label varchar(30),
                                                                  foreign_key_with_no_label varchar(30),
                                                                  FOREIGN KEY (foreign_key_with_label) REFERENCES simple_primary_key(id),
                                                                  FOREIGN KEY (foreign_key_with_no_label) REFERENCES primary_key_multiple_columns(id)
                                                                )
sqlite_autoindex_foreign_key_references_1
sortable                                                        CREATE TABLE sortable (
                                                                  pk1 varchar(30),
                                                                  pk2 varchar(30),
                                                                  content text,
                                                                  sortable integer,
                                                                  sortable_with_nulls real,
                                                                  sortable_with_nulls_2 real,
                                                                  text text,
                                                                  PRIMARY KEY (pk1, pk2)
                                                                )
sqlite_autoindex_sortable_1
no_primary_key                                                  CREATE TABLE no_primary_key (
                                                                  content text,
                                                                  a text,
                                                                  b text,
                                                                  c text
                                                                )
123_starts_with_digits                                          CREATE TABLE [123_starts_with_digits] (
                                                                  content text
                                                                )
paginated_view                                                  CREATE VIEW paginated_view AS
                                                                    SELECT
                                                                        content,
                                                                        '- ' || content || ' -' AS content_extra
                                                                    FROM no_primary_key
Table With Space In Name                                        CREATE TABLE "Table With Space In Name" (
                                                                  pk varchar(30) primary key,
                                                                  content text
                                                                )
sqlite_autoindex_Table With Space In Name_1
table/with/slashes.csv                                          CREATE TABLE "table/with/slashes.csv" (
                                                                  pk varchar(30) primary key,
                                                                  content text
                                                                )
sqlite_autoindex_table/with/slashes.csv_1
complex_foreign_keys                                            CREATE TABLE "complex_foreign_keys" (
                                                                  pk varchar(30) primary key,
                                                                  f1 text,
                                                                  f2 text,
                                                                  f3 text,
                                                                  FOREIGN KEY ("f1") REFERENCES [simple_primary_key](id),
                                                                  FOREIGN KEY ("f2") REFERENCES [simple_primary_key](id),
                                                                  FOREIGN KEY ("f3") REFERENCES [simple_primary_key](id)
                                                                )
sqlite_autoindex_complex_foreign_keys_1
custom_foreign_key_label                                        CREATE TABLE "custom_foreign_key_label" (
                                                                  pk varchar(30) primary key,
                                                                  foreign_key_with_custom_label text,
                                                                  FOREIGN KEY ("foreign_key_with_custom_label") REFERENCES [primary_key_multiple_columns_explicit_label](id)
                                                                )
sqlite_autoindex_custom_foreign_key_label_1
units                                                           CREATE TABLE units (
                                                                  pk integer primary key,
                                                                  distance int,
                                                                  frequency int
                                                                )
searchable                                                      CREATE TABLE searchable (
                                                                  pk integer primary key,
                                                                  text1 text,
                                                                  text2 text,
                                                                  [name with . and spaces] text
                                                                )
searchable_fts                                                  CREATE VIRTUAL TABLE "searchable_fts"
                                                                    USING FTS3 (text1, text2, [name with . and spaces], content="searchable")
searchable_fts_content                                          CREATE TABLE 'searchable_fts_content'(docid INTEGER PRIMARY KEY, 'c0text1', 'c1text2', 'c2name with . and spaces', 'c3content')
searchable_fts_segments                                         CREATE TABLE 'searchable_fts_segments'(blockid INTEGER PRIMARY KEY, block BLOB)
searchable_fts_segdir                                           CREATE TABLE 'searchable_fts_segdir'(level INTEGER,idx INTEGER,start_block INTEGER,leaves_end_block INTEGER,end_block INTEGER,root BLOB,PRIMARY KEY(level, idx))
sqlite_autoindex_searchable_fts_segdir_1
select                                                          CREATE TABLE [select] (
                                                                  [group] text,
                                                                  [having] text,
                                                                  [and] text
                                                                )
facet_cities                                                    CREATE TABLE facet_cities (
                                                                    id integer primary key,
                                                                    name text
                                                                )
simple_view                                                     CREATE VIEW simple_view AS
                                                                    SELECT content, upper(content) AS upper_content FROM simple_primary_key

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 db.schema property and sqlite-utils schema command 919181559  
859894105 https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859894105 https://api.github.com/repos/simonw/sqlite-utils/issues/268 MDEyOklzc3VlQ29tbWVudDg1OTg5NDEwNQ== simonw 9599 2021-06-11T20:28:52Z 2021-06-11T20:28:52Z OWNER

Out of interest, here are the rows from that table where sql is null: https://latest.datasette.io/fixtures?sql=select%0D%0A++*%0D%0Afrom%0D%0A++sqlite_master%0D%0Awhere%0D%0A++sql+is+null

type,name,tbl_name,rootpage,sql
index,sqlite_autoindex_simple_primary_key_1,simple_primary_key,3,
index,sqlite_autoindex_primary_key_multiple_columns_1,primary_key_multiple_columns,5,
index,sqlite_autoindex_primary_key_multiple_columns_explicit_label_1,primary_key_multiple_columns_explicit_label,7,
index,sqlite_autoindex_compound_primary_key_1,compound_primary_key,9,
index,sqlite_autoindex_compound_three_primary_keys_1,compound_three_primary_keys,11,
index,sqlite_autoindex_foreign_key_references_1,foreign_key_references,14,
index,sqlite_autoindex_sortable_1,sortable,16,
index,sqlite_autoindex_Table With Space In Name_1,Table With Space In Name,20,
index,sqlite_autoindex_table/with/slashes.csv_1,table/with/slashes.csv,22,
index,sqlite_autoindex_complex_foreign_keys_1,complex_foreign_keys,24,
index,sqlite_autoindex_custom_foreign_key_label_1,custom_foreign_key_label,26,
index,sqlite_autoindex_tags_1,tags,31,
index,sqlite_autoindex_searchable_tags_1,searchable_tags,34,
index,sqlite_autoindex_searchable_fts_segdir_1,searchable_fts_segdir,37,

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 db.schema property and sqlite-utils schema command 919181559  
859888469 https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859888469 https://api.github.com/repos/simonw/sqlite-utils/issues/268 MDEyOklzc3VlQ29tbWVudDg1OTg4ODQ2OQ== simonw 9599 2021-06-11T20:26:20Z 2021-06-11T20:26:20Z OWNER

sqlite-utils schema data.db could output the same thing to the console.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 db.schema property and sqlite-utils schema command 919181559  
859572791 https://github.com/simonw/datasette/pull/1374#issuecomment-859572791 https://api.github.com/repos/simonw/datasette/issues/1374 MDEyOklzc3VlQ29tbWVudDg1OTU3Mjc5MQ== codecov[bot] 22429695 2021-06-11T13:12:58Z 2021-06-11T13:12:58Z NONE

Codecov Report

Merging #1374 (0ef0dd5) into main (cd7678f) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1374   +/-   ##
=======================================
  Coverage   91.68%   91.68%           
=======================================
  Files          34       34           
  Lines        4340     4340           
=======================================
  Hits         3979     3979           
  Misses        361      361

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd7678f...0ef0dd5. Read the comment docs.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Bump black from 21.5b2 to 21.6b0 918730335  
858831895 https://github.com/simonw/datasette/issues/858#issuecomment-858831895 https://api.github.com/repos/simonw/datasette/issues/858 MDEyOklzc3VlQ29tbWVudDg1ODgzMTg5NQ== rachelll4 56045588 2021-06-10T17:44:09Z 2021-06-10T17:44:09Z NONE

any fixes for that recursive issue with temp file? I get it using both heroku and cloudrun, although it seems to still publish and deploy fine

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 publish heroku does not work on Windows 10 642388564  
858813675 https://github.com/simonw/datasette/issues/858#issuecomment-858813675 https://api.github.com/repos/simonw/datasette/issues/858 MDEyOklzc3VlQ29tbWVudDg1ODgxMzY3NQ== rachelll4 56045588 2021-06-10T17:27:46Z 2021-06-10T17:27:46Z NONE

shell=True is added to line 56 (I guess it used to be 54) of heroku.py as detailed in the original issue. (for posterity)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 publish heroku does not work on Windows 10 642388564  
858099514 https://github.com/simonw/datasette/issues/1371#issuecomment-858099514 https://api.github.com/repos/simonw/datasette/issues/1371 MDEyOklzc3VlQ29tbWVudDg1ODA5OTUxNA== simonw 9599 2021-06-09T21:03:49Z 2021-06-09T21:03:49Z OWNER

I'll release these as an alpha straight away - it makes sense to have plugin hook changes available for people to test as alpha dependencies ASAP.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Menu plugin hooks should include the request 915455228  
857684605 https://github.com/simonw/datasette/pull/1373#issuecomment-857684605 https://api.github.com/repos/simonw/datasette/issues/1373 MDEyOklzc3VlQ29tbWVudDg1NzY4NDYwNQ== codecov[bot] 22429695 2021-06-09T13:15:31Z 2021-06-09T13:15:31Z NONE

Codecov Report

Merging #1373 (d117ba7) into main (a3faf37) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1373   +/-   ##
=======================================
  Coverage   91.68%   91.68%           
=======================================
  Files          34       34           
  Lines        4340     4340           
=======================================
  Hits         3979     3979           
  Misses        361      361

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3faf37...d117ba7. Read the comment docs.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update trustme requirement from <0.8,>=0.7 to >=0.7,<0.9 916183914  
857298526 https://github.com/simonw/datasette/pull/1370#issuecomment-857298526 https://api.github.com/repos/simonw/datasette/issues/1370 MDEyOklzc3VlQ29tbWVudDg1NzI5ODUyNg== eyeseast 25778 2021-06-09T01:18:59Z 2021-06-09T01:18:59Z NONE

I'm happy to grab some or all of these in this PR, if you want. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Ensure db.path is a string before trying to insert into internal database 914130834  
857139881 https://github.com/simonw/datasette/pull/1370#issuecomment-857139881 https://api.github.com/repos/simonw/datasette/issues/1370 MDEyOklzc3VlQ29tbWVudDg1NzEzOTg4MQ== simonw 9599 2021-06-08T20:58:41Z 2021-06-08T20:58:41Z OWNER

We can remove a bunch of unnecessary str(path) calls too - this search finds a bunch of possible candidates: https://ripgrep.datasette.io/-/ripgrep?pattern=str%5C%28.*%28db%7Cpath%29&glob=datasette%2F**%2F*.py

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Ensure db.path is a string before trying to insert into internal database 914130834  
856182547 https://github.com/simonw/datasette/pull/1368#issuecomment-856182547 https://api.github.com/repos/simonw/datasette/issues/1368 MDEyOklzc3VlQ29tbWVudDg1NjE4MjU0Nw== brandonrobertz 2670795 2021-06-07T18:59:47Z 2021-06-07T23:04:25Z NONE

Note that if we went with a "update_metadata" hook, the hook signature would look something like this (it would return nothing):

update_metadata(
  datasette=self, metadata=metadata, key=key, database=database, table=table,
  fallback=fallback
)

The Datasette function _metadata_recursive_update(self, orig, updated) would disappear into the plugins. Doing this, though, we'd lose the easy ability to make the local metadata.yaml immutable (since we'd no longer have the recursive update).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 DRAFT: A new plugin hook for dynamic metadata 913865304  
856231119 https://github.com/simonw/sqlite-utils/issues/266#issuecomment-856231119 https://api.github.com/repos/simonw/sqlite-utils/issues/266 MDEyOklzc3VlQ29tbWVudDg1NjIzMTExOQ== simonw 9599 2021-06-07T20:26:05Z 2021-06-07T20:26:05Z OWNER

https://github.com/python/cpython/blob/2ab27c4af4ddf7528e1375e77c787c7fbb09b5e6/Lib/typing.py#L2173-L2195

In Python 3.6 or higher can do this:

class Employee(NamedTuple):
    name: str
    id: int

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Add some types, enforce with mypy 913135723  
856212136 https://github.com/simonw/datasette/issues/1365#issuecomment-856212136 https://api.github.com/repos/simonw/datasette/issues/1365 MDEyOklzc3VlQ29tbWVudDg1NjIxMjEzNg== simonw 9599 2021-06-07T19:54:04Z 2021-06-07T19:54:04Z OWNER

I've hit this one too. I agree, fixing this in Datasette itself is better than fixing it in the tests across multiple other projects.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 pathlib.Path breaks internal schema 913017577  
856208637 https://github.com/simonw/datasette/issues/1369#issuecomment-856208637 https://api.github.com/repos/simonw/datasette/issues/1369 MDEyOklzc3VlQ29tbWVudDg1NjIwODYzNw== simonw 9599 2021-06-07T19:47:23Z 2021-06-07T19:47:23Z OWNER

No point in showing the IDs twice if the blue label doesn't differ from the gray ID

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Don't show foreign key IDs twice if no label 913900374  
856160770 https://github.com/simonw/datasette/issues/1367#issuecomment-856160770 https://api.github.com/repos/simonw/datasette/issues/1367 MDEyOklzc3VlQ29tbWVudDg1NjE2MDc3MA== simonw 9599 2021-06-07T18:22:33Z 2021-06-07T18:22:33Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Navigation menu display bug 913823889  
856147969 https://github.com/simonw/datasette/issues/1366#issuecomment-856147969 https://api.github.com/repos/simonw/datasette/issues/1366 MDEyOklzc3VlQ29tbWVudDg1NjE0Nzk2OQ== simonw 9599 2021-06-07T18:03:03Z 2021-06-07T18:03:03Z OWNER

Here's an example of a test that uses it. It's necessary because sometimes fixtures that create temporary directories break in unexpected ways:

https://github.com/simonw/datasette/blob/0a7621f96f8ad14da17e7172e8a7bce24ef78966/tests/test_plugins.py#L658-L666

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Get rid of this `restore_working_directory` hack entirely 913809802  
856147450 https://github.com/simonw/datasette/issues/1366#issuecomment-856147450 https://api.github.com/repos/simonw/datasette/issues/1366 MDEyOklzc3VlQ29tbWVudDg1NjE0NzQ1MA== simonw 9599 2021-06-07T18:02:13Z 2021-06-07T18:02:13Z OWNER

The hack in question is this fixture, which I've been using in an ad-hoc manner to work around errors while running the tests: https://github.com/simonw/datasette/blob/030deb4b25cda842ff7129ab7c18550c44dd8379/tests/conftest.py#L62-L75

I don't understand the underlying issue well enough to know how to get rid of it.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Get rid of this `restore_working_directory` hack entirely 913809802  
855611939 https://github.com/simonw/sqlite-utils/issues/266#issuecomment-855611939 https://api.github.com/repos/simonw/sqlite-utils/issues/266 MDEyOklzc3VlQ29tbWVudDg1NTYxMTkzOQ== simonw 9599 2021-06-07T06:07:41Z 2021-06-07T06:07:41Z OWNER

Looks like this is the way to do this:

Point = typing.NamedTuple(
    "Point", [('x', int), ('y', int)]
)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Add some types, enforce with mypy 913135723  
855430317 https://github.com/simonw/datasette/issues/1362#issuecomment-855430317 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQzMDMxNw== simonw 9599 2021-06-06T17:07:48Z 2021-06-06T17:07:48Z OWNER

I guess I can offer a disable_csp setting so that people with complex custom templates aren't completely blocked from using them with Datasette, but maybe it would be better not to offer that? Or to offer it as a datasette-insecure-csp plugin instead?

I like the idea of very actively encouraging CSP across all Datasette projects, but I'm nervous about making the software unusable for certain edge cases.

Maybe require CSP and wait for someone to complain?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855429111 https://github.com/simonw/datasette/issues/1362#issuecomment-855429111 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQyOTExMQ== simonw 9599 2021-06-06T16:59:05Z 2021-06-06T17:00:15Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855428601 https://github.com/simonw/datasette/issues/1362#issuecomment-855428601 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQyODYwMQ== simonw 9599 2021-06-06T16:55:33Z 2021-06-06T16:55:33Z OWNER

No, because Vary header is about request headers that cause the response to vary, not response headers.

Hah, of course! Thanks for the correction. So the nonce mechanism would actually be pretty great here, especially for the extra_body_script() hook.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855428296 https://github.com/simonw/datasette/issues/1362#issuecomment-855428296 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQyODI5Ng== dracos 154364 2021-06-06T16:53:20Z 2021-06-06T16:53:20Z NONE

Presumably this would also require adding Content-Security-Policy to the Vary header though, which will have a nasty effect on Cloudflare and Fastly and such like.

No, because Vary header is about request headers that cause the response to vary, not response headers.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855427396 https://github.com/simonw/datasette/issues/1362#issuecomment-855427396 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQyNzM5Ng== simonw 9599 2021-06-06T16:46:17Z 2021-06-06T16:46:17Z OWNER

Mind you, since that plugin hook looks like this:

@hookimpl
def extra_body_script():
    return {
        "module": True,
        "script": "console.log('Your JavaScript goes here...')"
    }

Having it calculate a sha256 hash wouldn't be difficult.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855426750 https://github.com/simonw/datasette/issues/1362#issuecomment-855426750 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQyNjc1MA== simonw 9599 2021-06-06T16:41:30Z 2021-06-06T16:44:49Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855426516 https://github.com/simonw/datasette/issues/1362#issuecomment-855426516 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQyNjUxNg== simonw 9599 2021-06-06T16:39:34Z 2021-06-06T16:39:34Z OWNER

The reason Datasette uses small inline scripts right now is to avoid the overhead of an extra HTTP request for a JavaScript file - but these are both inherently cachable and perform much better under HTTP/2 so that's likely a false optimization.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855426314 https://github.com/simonw/datasette/issues/1362#issuecomment-855426314 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQyNjMxNA== simonw 9599 2021-06-06T16:38:04Z 2021-06-06T16:38:04Z OWNER

The other option for inline scripts is the CSP nonce:

Content-Security-Policy: script-src 'nonce-2726c7f26c'

Then:

<script nonce="2726c7f26c">
  var inline = 1;
</script>

Since an attacker can't guess what the nonce will be it prevents them from injecting their own script block - this seems easier to make available to plugins than a full hashing mechanism, just make {{ csp_nonce() }} available to the template.

That template function can then be smart enough to set a flag which Datasette uses to decide if the script-src 'nonce-2726c7f26c' policy should be sent or not.

Presumably this would also require adding Content-Security-Policy to the Vary header though, which will have a nasty effect on Cloudflare and Fastly and such like.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855418899 https://github.com/simonw/datasette/issues/1362#issuecomment-855418899 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQxODg5OQ== simonw 9599 2021-06-06T15:42:55Z 2021-06-06T15:42:55Z OWNER

Another consideration: testing that this works correctly could require adoption of a real browser test environment (probably Cypress or maybe Playwright) to execute tests that will fail if CSP is violated.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855418698 https://github.com/simonw/datasette/issues/1362#issuecomment-855418698 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQxODY5OA== simonw 9599 2021-06-06T15:41:24Z 2021-06-06T15:41:24Z OWNER

I think the best way to answer these questions is with some prototyping - of both Datasette and some of the existing JavaScript plugins.

I can start with a datasette-experimental-csp plugin that sets the header (and could even run an optional report URI mechanism).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855418401 https://github.com/simonw/datasette/issues/1362#issuecomment-855418401 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQxODQwMQ== simonw 9599 2021-06-06T15:39:38Z 2021-06-06T15:39:38Z OWNER

The security benefit of forcing all JavaScript plugins to be written as CSP-friendly external scripts is very compelling though.

Other plugin-heavy ecosystems such as WordPress have suffered greatly from insecurely written plugins - could this be a huge security win for the Datasette ecosystem generally?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855418065 https://github.com/simonw/datasette/issues/1362#issuecomment-855418065 https://api.github.com/repos/simonw/datasette/issues/1362 MDEyOklzc3VlQ29tbWVudDg1NTQxODA2NQ== simonw 9599 2021-06-06T15:37:11Z 2021-06-06T15:37:11Z OWNER

The easiest way to apply CSP is to remove all inline <script> blocks (Datasette has a few) and instead serve JavaScript as separate linked files.

It's possible to keep inline script blocks by calculating a hash of their content and adding a Content-Security-Policy: script-src 'sha256-B2yPHKaXnvFWtRChIbabYmUBFZdVfKKXHbWtWidDVF8=' to the policy. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src

This could be achieved with some Django template tricks, but it feels very risky - and done carelessly could end up calculating a hash of a reflected XSS attack!

The biggest challenge I see around here involves plugins and custom templates. Adopting CSP would require plugins to avoid using any inline scripts, instead keeping their entire implementations in .js files.

That's maybe not a bad thing, but it represents a big commitment. It would need to be adopted before Datasette 1.0.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Consider using CSP to protect against future XSS 912864936  
855369819 https://github.com/simonw/datasette/issues/283#issuecomment-855369819 https://api.github.com/repos/simonw/datasette/issues/283 MDEyOklzc3VlQ29tbWVudDg1NTM2OTgxOQ== simonw 9599 2021-06-06T09:40:18Z 2021-06-06T09:40:18Z OWNER

One note on using this pragma I got an error on starting datasette no such table: pragma_database_list.

I diagnosed this to an older version of sqlite3 (3.14.2) and upgrading to a newer version (3.34.2) fixed the issue.

That issue is fixed in #1276.

 
{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 1,
    "rocket": 0,
    "eyes": 0
}
 Support cross-database joins 325958506  
855308811 https://github.com/simonw/datasette/issues/1361#issuecomment-855308811 https://api.github.com/repos/simonw/datasette/issues/1361 MDEyOklzc3VlQ29tbWVudDg1NTMwODgxMQ== simonw 9599 2021-06-05T23:16:21Z 2021-06-05T23:16:21Z OWNER

That seems to have fixed it. I'd love to get rid of this restore_working_directory hack entirely.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent CI failure: restore_working_directory FileNotFoundError 912485040  
855307292 https://github.com/simonw/datasette/issues/1361#issuecomment-855307292 https://api.github.com/repos/simonw/datasette/issues/1361 MDEyOklzc3VlQ29tbWVudDg1NTMwNzI5Mg== simonw 9599 2021-06-05T22:59:35Z 2021-06-05T22:59:35Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent CI failure: restore_working_directory FileNotFoundError 912485040  
855306497 https://github.com/simonw/datasette/issues/1361#issuecomment-855306497 https://api.github.com/repos/simonw/datasette/issues/1361 MDEyOklzc3VlQ29tbWVudDg1NTMwNjQ5Nw== simonw 9599 2021-06-05T22:51:01Z 2021-06-05T22:51:01Z OWNER

I'm going to try removing that restore_working_directory fixture entirely.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent CI failure: restore_working_directory FileNotFoundError 912485040  
855306347 https://github.com/simonw/datasette/issues/1361#issuecomment-855306347 https://api.github.com/repos/simonw/datasette/issues/1361 MDEyOklzc3VlQ29tbWVudDg1NTMwNjM0Nw== simonw 9599 2021-06-05T22:49:30Z 2021-06-05T22:49:30Z OWNER

Stack Overflow: https://stackoverflow.com/a/49367679/6083

The answer was that os.chdir() had been set to the deleted directory by accident. The directory was missing, but the error happened (it seems) at the attempt to get it with os.getcwd().

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Intermittent CI failure: restore_working_directory FileNotFoundError 912485040  
855303776 https://github.com/simonw/datasette/issues/1360#issuecomment-855303776 https://api.github.com/repos/simonw/datasette/issues/1360 MDEyOklzc3VlQ29tbWVudDg1NTMwMzc3Ng== simonw 9599 2021-06-05T22:23:23Z 2021-06-05T22:23:23Z OWNER

Worth noting that I found this issue myself, and to my knowledge it has not been uncovered by anyone else prior to the patch being released.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Security flaw, to be fixed in 0.56.1 and 0.57 912464443  
855303649 https://github.com/simonw/datasette/issues/1360#issuecomment-855303649 https://api.github.com/repos/simonw/datasette/issues/1360 MDEyOklzc3VlQ29tbWVudDg1NTMwMzY0OQ== simonw 9599 2021-06-05T22:22:06Z 2021-06-05T22:22:06Z OWNER

I've released fixes in both 0.56.1 and 0.57.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Security flaw, to be fixed in 0.56.1 and 0.57 912464443  
855302339 https://github.com/simonw/datasette/issues/1358#issuecomment-855302339 https://api.github.com/repos/simonw/datasette/issues/1358 MDEyOklzc3VlQ29tbWVudDg1NTMwMjMzOQ== simonw 9599 2021-06-05T22:08:16Z 2021-06-05T22:08:16Z OWNER

Release notes are in, pushing the release now.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 0.57 912418094  
855302320 https://github.com/simonw/datasette/issues/1358#issuecomment-855302320 https://api.github.com/repos/simonw/datasette/issues/1358 MDEyOklzc3VlQ29tbWVudDg1NTMwMjMyMA== simonw 9599 2021-06-05T22:08:06Z 2021-06-05T22:08:06Z OWNER

See #1360 for the security fix.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 0.57 912418094  
855288228 https://github.com/simonw/datasette/issues/1358#issuecomment-855288228 https://api.github.com/repos/simonw/datasette/issues/1358 MDEyOklzc3VlQ29tbWVudDg1NTI4ODIyOA== simonw 9599 2021-06-05T19:57:18Z 2021-06-05T19:57:18Z OWNER

There's also a security fix I need to make, so I'm not going to block this on wrapping up the work on the new Docker image testing from #1344 before putting out this release.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Release Datasette 0.57 912418094  
855287200 https://github.com/simonw/datasette/pull/1291#issuecomment-855287200 https://api.github.com/repos/simonw/datasette/issues/1291 MDEyOklzc3VlQ29tbWVudDg1NTI4NzIwMA== simonw 9599 2021-06-05T19:48:36Z 2021-06-05T19:48:36Z OWNER

This is great, thank you.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update docs: explain allow_download setting 849582643  
812815358 https://github.com/simonw/datasette/pull/1291#issuecomment-812815358 https://api.github.com/repos/simonw/datasette/issues/1291 MDEyOklzc3VlQ29tbWVudDgxMjgxNTM1OA== codecov[bot] 22429695 2021-04-03T05:32:50Z 2021-06-05T19:48:30Z NONE

Codecov Report

Merging #1291 (8789157) into main (0a7621f) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 8789157 differs from pull request most recent head 9bf089f. Consider uploading reports for the commit 9bf089f to get more accurate results

@@           Coverage Diff           @@
##             main    #1291   +/-   ##
=======================================
  Coverage   91.51%   91.51%           
=======================================
  Files          34       34           
  Lines        4255     4255           
=======================================
  Hits         3894     3894           
  Misses        361      361

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a7621f...9bf089f. Read the comment docs.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update docs: explain allow_download setting 849582643  
855282466 https://github.com/simonw/datasette/issues/1356#issuecomment-855282466 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1NTI4MjQ2Ng== simonw 9599 2021-06-05T19:05:06Z 2021-06-05T19:05:06Z OWNER

Yeah that's a good point. I avoided making them sub-commands because datasette serve already supports the multitude of other arguments they also need... but actually that was just me being lazy - I can easily share arguments between multiple functions like I do in sqlite-utils itself: https://github.com/simonw/sqlite-utils/blob/d1a372b3006e6cf7d2017b3ddc484bf5c033e45d/sqlite_utils/cli.py#L46

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
855281774 https://github.com/simonw/datasette/pull/1357#issuecomment-855281774 https://api.github.com/repos/simonw/datasette/issues/1357 MDEyOklzc3VlQ29tbWVudDg1NTI4MTc3NA== codecov[bot] 22429695 2021-06-05T18:59:07Z 2021-06-05T18:59:07Z NONE

Codecov Report

Merging #1357 (1b27643) into main (6e9b07b) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1357   +/-   ##
=======================================
  Coverage   91.71%   91.72%           
=======================================
  Files          34       34           
  Lines        4333     4336    +3     
=======================================
+ Hits         3974     3977    +3     
  Misses        359      359
<table> <thead> <tr> <th>Impacted Files</th> <th>Coverage Δ</th> <th></th> </tr> </thead> <tbody> <tr> <td>datasette/utils/testing.py</td> <td>95.38% <ø> (ø)</td> <td></td> </tr> <tr> <td>datasette/app.py</td> <td>95.71% <100.00%> (+0.01%)</td> <td>:arrow_up:</td> </tr> </tbody> </table>

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e9b07b...1b27643. Read the comment docs.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make custom pages compatible with base_url setting 912394511  
855278998 https://github.com/simonw/datasette/issues/1238#issuecomment-855278998 https://api.github.com/repos/simonw/datasette/issues/1238 MDEyOklzc3VlQ29tbWVudDg1NTI3ODk5OA== simonw 9599 2021-06-05T18:37:16Z 2021-06-05T18:37:16Z OWNER

Alternative idea: populate request.scope with a new route_path which is the base-url-stripped version, which we then use for other routing operations.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Custom pages don't work with base_url setting 813899472  
855278540 https://github.com/simonw/datasette/issues/1238#issuecomment-855278540 https://api.github.com/repos/simonw/datasette/issues/1238 MDEyOklzc3VlQ29tbWVudDg1NTI3ODU0MA== simonw 9599 2021-06-05T18:33:25Z 2021-06-05T18:33:25Z OWNER

Got the test to pass by ensuring the tests don't accidentally double-rewrite the path.

Ran into a new problem:

    @pytest.mark.asyncio
    @pytest.mark.parametrize(
        "prefix,expected_path", [(None, "/asgi-scope"), ("/prefix/", "/prefix/asgi-scope")]
    )
    async def test_client_path(datasette, prefix, expected_path):
        original_base_url = datasette._settings["base_url"]
        try:
            if prefix is not None:
                datasette._settings["base_url"] = prefix
            response = await datasette.client.get("/asgi-scope")
            path = response.json()["path"]
>           assert path == expected_path
E           AssertionError: assert '/asgi-scope' == '/prefix/asgi-scope'
E             - /prefix/asgi-scope
E             ? -------
E             + /asgi-scope

That test confirms that messing around with the base_url doesn't modify the ASGI scope... but the fix I'm using for this issue DOES modify the ASGI scope.

The question raised here is: should the ASGI scope stay unmodified when base_url is used?

I think it should. It doesn't make sense to obscure the "real" path just to get custom pages to work properly.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Custom pages don't work with base_url setting 813899472  
855272693 https://github.com/simonw/datasette/issues/1238#issuecomment-855272693 https://api.github.com/repos/simonw/datasette/issues/1238 MDEyOklzc3VlQ29tbWVudDg1NTI3MjY5Mw== simonw 9599 2021-06-05T17:45:42Z 2021-06-05T17:45:42Z OWNER

Applying this fix worked when I manually tested it:

         base_url = self.ds.setting("base_url")
         if base_url != "/" and path.startswith(base_url):
             path = "/" + path[len(base_url) :]
+            scope = dict(scope, path=path, raw_path=path.encode("utf-8"))
         request = Request(scope, receive)

But... the test I wrote still failed. My hunch is that this is because deep within the test framework requests go through ds.client which may be applying its own changes relevant to base_url:

https://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/utils/testing.py#L139

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Custom pages don't work with base_url setting 813899472  
855270917 https://github.com/simonw/datasette/issues/1238#issuecomment-855270917 https://api.github.com/repos/simonw/datasette/issues/1238 MDEyOklzc3VlQ29tbWVudDg1NTI3MDkxNw== simonw 9599 2021-06-05T17:32:29Z 2021-06-05T17:32:29Z OWNER

This looks like the cause: https://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/app.py#L1087-L1092

Note how path is modified... but then we create a new Request() that uses the old scope, which has unmodified scope["path"] - and then the code later on looks at request.scope["path"] when deciding if the request matches:

https://github.com/simonw/datasette/blob/afed51b1e36cf275c39e71c7cb262d6c5bdbaa31/datasette/app.py#L1154-L1155

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Custom pages don't work with base_url setting 813899472  
853895159 https://github.com/simonw/datasette/issues/1356#issuecomment-853895159 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1Mzg5NTE1OQ== eyeseast 25778 2021-06-03T14:03:59Z 2021-06-03T14:03:59Z NONE

(Putting thoughts here to keep the conversation in one place.)

I think using datasette for this use-case is the right approach. I usually have both datasette and sqlite-utils installed in the same project, and that's where I'm trying out queries, so it probably makes the most sense to have datasette also manage the output (and maybe the input, too).

It seems like both --get and --query could work better as subcommands, rather than options, if you're looking at building out a full CLI experience in datasette. It would give a cleaner separation in what you're trying to do and let each have its own dedicated options. So something like this:

# run an arbitrary query
datasette query covid.db "select * from ny_times_us_counties limit 1" --format yaml

# run a canned query
datasette get covid.db some-canned-query --format yaml

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853567861 https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853567861 https://api.github.com/repos/simonw/sqlite-utils/issues/264 MDEyOklzc3VlQ29tbWVudDg1MzU2Nzg2MQ== simonw 9599 2021-06-03T05:12:21Z 2021-06-03T05:12:21Z OWNER

I think this is more likely to happen in Datasette than in sqlite-utils - see https://github.com/simonw/datasette/issues/1356 for thoughts on this.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Supporting additional output formats, like GeoJSON 907642546  
853567413 https://github.com/simonw/datasette/issues/526#issuecomment-853567413 https://api.github.com/repos/simonw/datasette/issues/526 MDEyOklzc3VlQ29tbWVudDg1MzU2NzQxMw== simonw 9599 2021-06-03T05:11:27Z 2021-06-03T05:11:27Z OWNER

Another potential way to implement this would be to hold the SQLite connection open and execute the full query there.

I've avoided this in the past due to concerns of resource exhaustion - if multiple requests attempt this at the same time all of the connections in the pool will become tied up and the site will be unable to respond to further requests.

But... now that Datasette has authentication there's the possibility of making this feature only available to specific authenticated users - the --root user for example. Which avoids the danger while unlocking a super-useful feature.

Not to mention people who are running Datasette privately on their own laptop, or the proposed --query CLI feature in #1356.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Stream all results for arbitrary SQL and canned queries 459882902  
853566337 https://github.com/simonw/datasette/issues/1356#issuecomment-853566337 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1MzU2NjMzNw== simonw 9599 2021-06-03T05:08:32Z 2021-06-03T05:08:32Z OWNER

Also relevant: CSV streaming for canned queries in #526 - even better if we could stream ANY SQL query.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853566011 https://github.com/simonw/datasette/issues/1062#issuecomment-853566011 https://api.github.com/repos/simonw/datasette/issues/1062 MDEyOklzc3VlQ29tbWVudDg1MzU2NjAxMQ== simonw 9599 2021-06-03T05:07:42Z 2021-06-03T05:07:42Z OWNER

Implementing this would make #1356 a whole lot more interesting.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows 732674148  
853565850 https://github.com/simonw/datasette/issues/1356#issuecomment-853565850 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1MzU2NTg1MA== simonw 9599 2021-06-03T05:07:21Z 2021-06-03T05:07:21Z OWNER

A problem with this is that if you're using --query you likely want ALL of the results - at the moment the only Datasette output type that can stream everything is .csv and plugin formats can't handle full streams, see #1062 and #1177.

So there's not much point implementing this unless we first make plugins able to add custom streaming formats.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853562891 https://github.com/simonw/datasette/issues/1356#issuecomment-853562891 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1MzU2Mjg5MQ== simonw 9599 2021-06-03T04:59:40Z 2021-06-03T04:59:40Z OWNER

It's weird that --get is documented on this page right now: https://docs.datasette.io/en/stable/getting_started.html#datasette-get

If I implement this I should build a new "Datasette on the command-line" documentation page to cover both --get and --query.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853560870 https://github.com/simonw/datasette/issues/1356#issuecomment-853560870 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1MzU2MDg3MA== simonw 9599 2021-06-03T04:53:47Z 2021-06-03T04:53:56Z OWNER

This is also interesting when used in conjunction with the proposed datasette insert command from #1163 - Datasette would become a plugin-driven CLI tool for both ingesting and outputting data, as a side-effect of its web features.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853560389 https://github.com/simonw/datasette/issues/1356#issuecomment-853560389 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1MzU2MDM4OQ== simonw 9599 2021-06-03T04:52:13Z 2021-06-03T04:52:13Z OWNER

I should implement #1355 for more efficient --csv streaming as part of this.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853560237 https://github.com/simonw/datasette/issues/1356#issuecomment-853560237 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1MzU2MDIzNw== simonw 9599 2021-06-03T04:51:49Z 2021-06-03T04:51:49Z OWNER

This feels like a relatively simple feature to implement that unlocks a whole new set of possible uses for Datasette - as described by @eyeseast in https://github.com/simonw/sqlite-utils/issues/264#issue-907642546.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853559915 https://github.com/simonw/datasette/issues/1356#issuecomment-853559915 https://api.github.com/repos/simonw/datasette/issues/1356 MDEyOklzc3VlQ29tbWVudDg1MzU1OTkxNQ== simonw 9599 2021-06-03T04:50:52Z 2021-06-03T04:50:52Z OWNER

What happens if you pass multiple databases? The --query would be executed against the first one. And if you pass --crossdb it would be executed against the /_memory database and would support cross-database joins.

Key thing here is that output plugins are supported (also plugins that add new SQL functions), making many Datasette plugins usable from the command-line.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Research: syntactic sugar for using --get with SQL queries, maybe "datasette query" 910092577  
853558741 https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853558741 https://api.github.com/repos/simonw/sqlite-utils/issues/264 MDEyOklzc3VlQ29tbWVudDg1MzU1ODc0MQ== simonw 9599 2021-06-03T04:47:19Z 2021-06-03T04:47:19Z OWNER

This inspired me to re-examine how --get works, hence this issue: https://github.com/simonw/datasette/issues/1355

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Supporting additional output formats, like GeoJSON 907642546  
853557439 https://github.com/simonw/datasette/issues/1355#issuecomment-853557439 https://api.github.com/repos/simonw/datasette/issues/1355 MDEyOklzc3VlQ29tbWVudDg1MzU1NzQzOQ== simonw 9599 2021-06-03T04:43:14Z 2021-06-03T04:43:14Z OWNER

It's using TestClient at the moment which is a wrapper around httpx (as of ) that uses the @async_to_sync decorator to hide the async nature.

https://github.com/simonw/datasette/blob/f78ebdc04537a6102316d6dbbf6c887565806078/datasette/utils/testing.py#L102-L156

Maybe the fix here is to switch the --get implementation to using httpx directly with https://www.python-httpx.org/async/#streaming-responses

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 datasette --get should efficiently handle streaming CSV 910088936  
853553754 https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853553754 https://api.github.com/repos/simonw/sqlite-utils/issues/264 MDEyOklzc3VlQ29tbWVudDg1MzU1Mzc1NA== simonw 9599 2021-06-03T04:32:42Z 2021-06-03T04:36:36Z OWNER

This is a really interesting thought. I've so far resisted the temptation to add plugins to sqlite-utils, partly to avoid overlap with Datasette - but I'm open to discussing it.

There's actually a way for you to do what you're describing using datasette on the command-line, though it's a little obscure - also Datasette doesn't yet have a GeoJSON output extension, though it really should have one.

Here's an example using datasette-yaml:

datasette /tmp/covid.db --get='/covid/ny_times_us_counties.yaml'
- rowid: 1
  date: '2020-01-21'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0
- rowid: 2
  date: '2020-01-22'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0

It even works with arbitrary SQL queries, though you might have to apply URL encoding to the --get string (this seems to work though):

datasette /tmp/covid.db --get='/covid.yaml?sql=select * from ny_times_us_counties limit 2' 
- date: '2020-01-21'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0
- date: '2020-01-22'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0

Here's the documentation for --get: https://docs.datasette.io/en/latest/getting_started.html#datasette-get

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Supporting additional output formats, like GeoJSON 907642546  
853554550 https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853554550 https://api.github.com/repos/simonw/sqlite-utils/issues/263 MDEyOklzc3VlQ29tbWVudDg1MzU1NDU1MA== simonw 9599 2021-06-03T04:34:38Z 2021-06-03T04:34:38Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils indexes` command 906356331  
853548442 https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853548442 https://api.github.com/repos/simonw/sqlite-utils/issues/263 MDEyOklzc3VlQ29tbWVudDg1MzU0ODQ0Mg== simonw 9599 2021-06-03T04:16:49Z 2021-06-03T04:16:49Z OWNER

Needs to show the table each index applies to.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils indexes` command 906356331  
853547681 https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853547681 https://api.github.com/repos/simonw/sqlite-utils/issues/263 MDEyOklzc3VlQ29tbWVudDg1MzU0NzY4MQ== simonw 9599 2021-06-03T04:14:28Z 2021-06-03T04:14:28Z OWNER

This looks good:

% sqlite-utils indexes /tmp/covid.db -t      
index_name                                                seqno    cid  name                 desc  coll      key
------------------------------------------------------  -------  -----  -----------------  ------  ------  -----
idx_johns_hopkins_csse_daily_reports_combined_key             0     12  combined_key            0  BINARY      1
idx_johns_hopkins_csse_daily_reports_country_or_region        0      1  country_or_region       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_province_or_state        0      2  province_or_state       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_day                      0      0  day                     0  BINARY      1
idx_ny_times_us_counties_date                                 0      0  date                    1  BINARY      1
idx_ny_times_us_counties_fips                                 0      3  fips                    0  BINARY      1
idx_ny_times_us_counties_county                               0      1  county                  0  BINARY      1
idx_ny_times_us_counties_state                                0      2  state                   0  BINARY      1

% sqlite-utils indexes /tmp/covid.db -t --aux
index_name                                                seqno    cid  name                 desc  coll      key
------------------------------------------------------  -------  -----  -----------------  ------  ------  -----
idx_johns_hopkins_csse_daily_reports_combined_key             0     12  combined_key            0  BINARY      1
idx_johns_hopkins_csse_daily_reports_combined_key             1     -1                          0  BINARY      0
idx_johns_hopkins_csse_daily_reports_country_or_region        0      1  country_or_region       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_country_or_region        1     -1                          0  BINARY      0
idx_johns_hopkins_csse_daily_reports_province_or_state        0      2  province_or_state       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_province_or_state        1     -1                          0  BINARY      0
idx_johns_hopkins_csse_daily_reports_day                      0      0  day                     0  BINARY      1
idx_johns_hopkins_csse_daily_reports_day                      1     -1                          0  BINARY      0
idx_ny_times_us_counties_date                                 0      0  date                    1  BINARY      1
idx_ny_times_us_counties_date                                 1     -1                          0  BINARY      0
idx_ny_times_us_counties_fips                                 0      3  fips                    0  BINARY      1
idx_ny_times_us_counties_fips                                 1     -1                          0  BINARY      0
idx_ny_times_us_counties_county                               0      1  county                  0  BINARY      1
idx_ny_times_us_counties_county                               1     -1                          0  BINARY      0
idx_ny_times_us_counties_state                                0      2  state                   0  BINARY      1
idx_ny_times_us_counties_state                                1     -1                          0  BINARY      0

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils indexes` command 906356331  
853546818 https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853546818 https://api.github.com/repos/simonw/sqlite-utils/issues/263 MDEyOklzc3VlQ29tbWVudDg1MzU0NjgxOA== simonw 9599 2021-06-03T04:11:46Z 2021-06-03T04:11:46Z OWNER

By default I won't return auxiliary columns, but I'll offer a --aux option to return them.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils indexes` command 906356331  
853545743 https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853545743 https://api.github.com/repos/simonw/sqlite-utils/issues/263 MDEyOklzc3VlQ29tbWVudDg1MzU0NTc0Mw== simonw 9599 2021-06-03T04:08:04Z 2021-06-03T04:08:04Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils indexes` command 906356331  
853544493 https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853544493 https://api.github.com/repos/simonw/sqlite-utils/issues/263 MDEyOklzc3VlQ29tbWVudDg1MzU0NDQ5Mw== simonw 9599 2021-06-03T04:03:59Z 2021-06-03T04:03:59Z OWNER

Here's how sqlite-utils triggers works: https://github.com/simonw/sqlite-utils/blob/9c67cb925253cd5ef54a1fe0496e0ff9caeacfd6/sqlite_utils/cli.py#L1266-L1277

Running it from a SQL query makes it easy to support modifiers like --csv and -t.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils indexes` command 906356331  
853541869 https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853541869 https://api.github.com/repos/simonw/sqlite-utils/issues/261 MDEyOklzc3VlQ29tbWVudDg1MzU0MTg2OQ== simonw 9599 2021-06-03T03:54:14Z 2021-06-03T03:54:14Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `table.xindexes` using `PRAGMA index_xinfo(table)` 906345899  
853535559 https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853535559 https://api.github.com/repos/simonw/sqlite-utils/issues/261 MDEyOklzc3VlQ29tbWVudDg1MzUzNTU1OQ== simonw 9599 2021-06-03T03:32:47Z 2021-06-03T03:32:47Z OWNER

New design:

def test_xindexes(fresh_db):
    fresh_db.executescript(
        """
        create table Gosh (c1 text, c2 text, c3 text);
        create index Gosh_c1 on Gosh(c1);
        create index Gosh_c2c3 on Gosh(c2, c3 desc);
    """
    )
    assert fresh_db["Gosh"].xindexes == [
        XIndex(
            name="Gosh_c2c3",
            columns=[
                XIndexColumn(seqno=0, cid=1, name="c2", desc=0, coll="BINARY", key=1),
                XIndexColumn(seqno=1, cid=2, name="c3", desc=1, coll="BINARY", key=1),
                XIndexColumn(seqno=2, cid=-1, name=None, desc=0, coll="BINARY", key=0),
            ],
        ),
        XIndex(
            name="Gosh_c1",
            columns=[
                XIndexColumn(seqno=0, cid=0, name="c1", desc=0, coll="BINARY", key=1),
                XIndexColumn(seqno=1, cid=-1, name=None, desc=0, coll="BINARY", key=0),
            ],
        ),
    ]

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `table.xindexes` using `PRAGMA index_xinfo(table)` 906345899  
853534732 https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853534732 https://api.github.com/repos/simonw/sqlite-utils/issues/261 MDEyOklzc3VlQ29tbWVudDg1MzUzNDczMg== simonw 9599 2021-06-03T03:30:10Z 2021-06-03T03:30:10Z OWNER

I'm going to return XIndex(name, columns) - where columns is a list of XIndexColumn.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `table.xindexes` using `PRAGMA index_xinfo(table)` 906345899  
853530348 https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853530348 https://api.github.com/repos/simonw/sqlite-utils/issues/261 MDEyOklzc3VlQ29tbWVudDg1MzUzMDM0OA== simonw 9599 2021-06-03T03:16:33Z 2021-06-03T03:16:33Z OWNER

In prototyping this out I realize that I actually want to get back the name of each index, then for each of them the detailed list of index columns. Here's the test from my initial prototype:

def test_xindexes(fresh_db):
    fresh_db.executescript(
        """
        create table Gosh (c1 text, c2 text, c3 text);
        create index Gosh_c1 on Gosh(c1);
        create index Gosh_c2c3 on Gosh(c2, c3 desc);
    """
    )
    assert fresh_db["Gosh"].xindexes == [
        (
            "Gosh_c2c3",
            [
                XIndex(seqno=0, cid=1, name="c2", desc=0, coll="BINARY", key=1),
                XIndex(seqno=1, cid=2, name="c3", desc=1, coll="BINARY", key=1),
                XIndex(seqno=2, cid=-1, name=None, desc=0, coll="BINARY", key=0),
            ],
        ),
        (
            "Gosh_c1",
            [
                XIndex(seqno=0, cid=0, name="c1", desc=0, coll="BINARY", key=1),
                XIndex(seqno=1, cid=-1, name=None, desc=0, coll="BINARY", key=0),
            ],
        ),
    ]

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `table.xindexes` using `PRAGMA index_xinfo(table)` 906345899  
853525036 https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853525036 https://api.github.com/repos/simonw/sqlite-utils/issues/261 MDEyOklzc3VlQ29tbWVudDg1MzUyNTAzNg== simonw 9599 2021-06-03T03:02:22Z 2021-06-03T03:02:22Z OWNER

This would be a breaking change - and the fact that it returns auxiliary columns isn't particularly useful for most cases - "Auxiliary columns are additional columns needed to locate the table entry that corresponds to each index entry".

Instead, I'm going to add a new property table.xindexes which exposes this.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `table.xindexes` using `PRAGMA index_xinfo(table)` 906345899  
852712469 https://github.com/simonw/datasette/issues/893#issuecomment-852712469 https://api.github.com/repos/simonw/datasette/issues/893 MDEyOklzc3VlQ29tbWVudDg1MjcxMjQ2OQ== simonw 9599 2021-06-02T04:29:56Z 2021-06-02T04:29:56Z OWNER

I think I fixed this a while ago, though I can't find the commit now. Please re-open if the bug still happens!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 pip3 install datasette not serving static on linuxbrew. 656959584  
852712106 https://github.com/simonw/datasette/issues/1127#issuecomment-852712106 https://api.github.com/repos/simonw/datasette/issues/1127 MDEyOklzc3VlQ29tbWVudDg1MjcxMjEwNg== simonw 9599 2021-06-02T04:28:55Z 2021-06-02T04:28:55Z OWNER

This became resizable in #1236.

 
{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Make the custom SQL query text box larger or resizable 756818250  
852710117 https://github.com/simonw/datasette/pull/1306#issuecomment-852710117 https://api.github.com/repos/simonw/datasette/issues/1306 MDEyOklzc3VlQ29tbWVudDg1MjcxMDExNw== simonw 9599 2021-06-02T04:24:33Z 2021-06-02T04:24:33Z OWNER

I have a test - I'll merge this and then add that test.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Avoid error sorting by relationships if related tables are not allowed 864979486  
852708574 https://github.com/simonw/datasette/pull/1306#issuecomment-852708574 https://api.github.com/repos/simonw/datasette/issues/1306 MDEyOklzc3VlQ29tbWVudDg1MjcwODU3NA== simonw 9599 2021-06-02T04:20:18Z 2021-06-02T04:20:18Z OWNER

https://github.com/simonw/datasette/blob/115332ce76c0e867d9936406aaf4bcee6b1ef3cb/datasette/views/index.py#L77 helps here - looks like we can pass ?_sort=relationships to trigger the bug.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Avoid error sorting by relationships if related tables are not allowed 864979486  
852707676 https://github.com/simonw/datasette/pull/1306#issuecomment-852707676 https://api.github.com/repos/simonw/datasette/issues/1306 MDEyOklzc3VlQ29tbWVudDg1MjcwNzY3Ng== simonw 9599 2021-06-02T04:17:41Z 2021-06-02T04:17:41Z OWNER

We need to add a test that illustrates the bug in #1305 so we can prove that this definitely fixes it.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Avoid error sorting by relationships if related tables are not allowed 864979486  
852706970 https://github.com/simonw/datasette/issues/619#issuecomment-852706970 https://api.github.com/repos/simonw/datasette/issues/619 MDEyOklzc3VlQ29tbWVudDg1MjcwNjk3MA== simonw 9599 2021-06-02T04:15:54Z 2021-06-02T04:15:54Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 "Invalid SQL" page should let you edit the SQL 520655983  
852703357 https://github.com/simonw/datasette/issues/619#issuecomment-852703357 https://api.github.com/repos/simonw/datasette/issues/619 MDEyOklzc3VlQ29tbWVudDg1MjcwMzM1Nw== simonw 9599 2021-06-02T04:08:03Z 2021-06-02T04:08:03Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 "Invalid SQL" page should let you edit the SQL 520655983  
852693854 https://github.com/simonw/datasette/pull/1346#issuecomment-852693854 https://api.github.com/repos/simonw/datasette/issues/1346 MDEyOklzc3VlQ29tbWVudDg1MjY5Mzg1NA== codecov[bot] 22429695 2021-06-02T03:44:35Z 2021-06-02T03:44:35Z NONE

Codecov Report

Merging #1346 (3bffc35) into main (7b106e1) will increase coverage by 0.14%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1346      +/-   ##
==========================================
+ Coverage   91.56%   91.71%   +0.14%     
==========================================
  Files          34       34              
  Lines        4282     4332      +50     
==========================================
+ Hits         3921     3973      +52     
+ Misses        361      359       -2
<table> <thead> <tr> <th>Impacted Files</th> <th>Coverage Δ</th> <th></th> </tr> </thead> <tbody> <tr> <td>datasette/app.py</td> <td>95.69% <ø> (-0.15%)</td> <td>:arrow_down:</td> </tr> <tr> <td>datasette/renderer.py</td> <td>94.20% <100.00%> (+0.17%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/utils/__init__.py</td> <td>94.36% <100.00%> (+0.05%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/views/base.py</td> <td>95.41% <100.00%> (+0.39%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/views/database.py</td> <td>97.28% <100.00%> (+0.09%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/views/table.py</td> <td>95.90% <100.00%> (+0.07%)</td> <td>:arrow_up:</td> </tr> <tr> <td>datasette/tracer.py</td> <td>85.05% <0.00%> (+3.44%)</td> <td>:arrow_up:</td> </tr> </tbody> </table>

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b106e1...3bffc35. Read the comment docs.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Re-display user's query with an error message if an error occurs 904537568  
852686827 https://github.com/simonw/datasette/issues/1257#issuecomment-852686827 https://api.github.com/repos/simonw/datasette/issues/1257 MDEyOklzc3VlQ29tbWVudDg1MjY4NjgyNw== simonw 9599 2021-06-02T03:26:27Z 2021-06-02T03:26:27Z OWNER

I tried and failed to get this fix working for tables with double quotes in their name - I couldn't figure out what the double-quote-in-a-table-name version of this code would look like: https://github.com/simonw/datasette/blob/807de378d08752a0f05bb1b980a0a62620a70520/datasette/utils/__init__.py#L538-L548

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Table names containing single quotes break things 828811618  
852681622 https://github.com/simonw/datasette/issues/1257#issuecomment-852681622 https://api.github.com/repos/simonw/datasette/issues/1257 MDEyOklzc3VlQ29tbWVudDg1MjY4MTYyMg== simonw 9599 2021-06-02T03:12:18Z 2021-06-02T03:12:18Z OWNER

Created a test database like this:

% sqlite-utils create-table quote-in-name.db "this'hasquoteinname" id integer name text --pk id
% datasette quote-in-name.db -p 8025 --pdb
INFO:     Started server process [86046]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8025 (Press CTRL+C to quit)
> /Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py(530)detect_fts()
-> rows = conn.execute(detect_fts_sql(table)).fetchall()
(Pdb) c
Traceback (most recent call last):
  File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 1124, in route_path
    response = await view(request, send)
  File "/Users/simon/Dropbox/Development/datasette/datasette/views/base.py", line 147, in view
    return await self.dispatch_request(
  File "/Users/simon/Dropbox/Development/datasette/datasette/views/base.py", line 122, in dispatch_request
    return await handler(request, *args, **kwargs)
  File "/Users/simon/Dropbox/Development/datasette/datasette/views/index.py", line 72, in get
    "fts_table": await db.fts_table(table),
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 279, in fts_table
    return await self.execute_fn(lambda conn: detect_fts(conn, table))
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 155, in execute_fn
    return await asyncio.get_event_loop().run_in_executor(
  File "/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 153, in in_thread
    return fn(conn)
  File "/Users/simon/Dropbox/Development/datasette/datasette/database.py", line 279, in <lambda>
    return await self.execute_fn(lambda conn: detect_fts(conn, table))
  File "/Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py", line 530, in detect_fts
    rows = conn.execute(detect_fts_sql(table)).fetchall()
sqlite3.OperationalError: near "hasquoteinname": syntax error

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Table names containing single quotes break things 828811618  
852673695 https://github.com/simonw/datasette/pull/1352#issuecomment-852673695 https://api.github.com/repos/simonw/datasette/issues/1352 MDEyOklzc3VlQ29tbWVudDg1MjY3MzY5NQ== simonw 9599 2021-06-02T02:52:26Z 2021-06-02T02:52:26Z OWNER

@dependabot recreate

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Bump black from 21.5b1 to 21.5b2 908276134  
852256784 https://github.com/simonw/datasette/issues/263#issuecomment-852256784 https://api.github.com/repos/simonw/datasette/issues/263 MDEyOklzc3VlQ29tbWVudDg1MjI1Njc4NA== simonw 9599 2021-06-01T16:20:09Z 2021-06-01T16:20:09Z OWNER

This is a lot easier to implement now that we have a ?_nofacet=1 option from #1350.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Facets should not execute for ?shape=array|object 323671577  
852256454 https://github.com/simonw/datasette/issues/1350#issuecomment-852256454 https://api.github.com/repos/simonw/datasette/issues/1350 MDEyOklzc3VlQ29tbWVudDg1MjI1NjQ1NA== simonw 9599 2021-06-01T16:19:38Z 2021-06-01T16:19:38Z OWNER

I renamed this to ?_nofacet=1 in #1353.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_nofacets=1 query string argument for disabling facets and suggested facets 906977719  
852255951 https://github.com/simonw/datasette/issues/1353#issuecomment-852255951 https://api.github.com/repos/simonw/datasette/issues/1353 MDEyOklzc3VlQ29tbWVudDg1MjI1NTk1MQ== simonw 9599 2021-06-01T16:18:54Z 2021-06-01T16:18:54Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_nocount=1 for opting out of table counts 908446997  
852254298 https://github.com/simonw/datasette/issues/1354#issuecomment-852254298 https://api.github.com/repos/simonw/datasette/issues/1354 MDEyOklzc3VlQ29tbWVudDg1MjI1NDI5OA== simonw 9599 2021-06-01T16:16:32Z 2021-06-01T16:16:32Z OWNER

Running python update-docs-help.py helps fix this.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Update help in tests for latest Click 908465747  
852238201 https://github.com/simonw/datasette/issues/1353#issuecomment-852238201 https://api.github.com/repos/simonw/datasette/issues/1353 MDEyOklzc3VlQ29tbWVudDg1MjIzODIwMQ== simonw 9599 2021-06-01T15:53:57Z 2021-06-01T15:53:57Z OWNER

I'm going to rename ?_nofacets=1 to ?_nofacet=1 to keep it consistent with the new ?_nocount=1 option (and because I don't like ?_nocounts=1).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 ?_nocount=1 for opting out of table counts 908446997  
852237347 https://github.com/simonw/datasette/issues/1349#issuecomment-852237347 https://api.github.com/repos/simonw/datasette/issues/1349 MDEyOklzc3VlQ29tbWVudDg1MjIzNzM0Nw== simonw 9599 2021-06-01T15:52:50Z 2021-06-01T15:52:50Z OWNER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 CSV ?_stream=on redundantly calculates facets for every page 906385991  
851567204 https://github.com/simonw/datasette/issues/1284#issuecomment-851567204 https://api.github.com/repos/simonw/datasette/issues/1284 MDEyOklzc3VlQ29tbWVudDg1MTU2NzIwNA== mroswell 192568 2021-05-31T15:42:10Z 2021-05-31T15:42:10Z CONTRIBUTOR

I very much want to make:
https://disinfectants.SaferOrToxic.org/disinfectants/listN
have this URL:
https://disinfectants.SaferOrToxic.org/

I'm using only one table page on the site, with no pagination. I'm not using the home page, though when I tried to move my table to the home page as mentioned above, I failed to figure out how.

I am using cloudflare, but I haven't figured out a forwarding or HTML re-write method of doing this, either.

Is there any way I can get a prettier list URL? I'm on Vercel.

(I have a wordpress site on the main domain on a separate host.)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Feature or Documentation Request: Individual table as home page template 845794436  
851133471 https://github.com/simonw/datasette/issues/1351#issuecomment-851133471 https://api.github.com/repos/simonw/datasette/issues/1351 MDEyOklzc3VlQ29tbWVudDg1MTEzMzQ3MQ== simonw 9599 2021-05-31T03:02:59Z 2021-05-31T03:02:59Z OWNER

Since traces only work with text/html and JSON at the moment, the easiest way to do this will be to wrap generated CSV in a HTML page in a textarea if the user specified ?_trace=1:

https://github.com/simonw/datasette/blob/c5ae1197a208e1b034c88882e3ac865813a40980/datasette/tracer.py#L125-L134

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Get `?_trace=1` working with CSV and streaming CSVs 906993731  
851133125 https://github.com/simonw/datasette/issues/1349#issuecomment-851133125 https://api.github.com/repos/simonw/datasette/issues/1349 MDEyOklzc3VlQ29tbWVudDg1MTEzMzEyNQ== simonw 9599 2021-05-31T03:01:48Z 2021-05-31T03:01:48Z OWNER

I think it's worth getting ?_trace=1 to work with streaming CSV - this would have helped me spot this issue a long time ago.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 CSV ?_stream=on redundantly calculates facets for every page 906385991  

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);