html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/pull/1130#issuecomment-861497548,https://api.github.com/repos/simonw/datasette/issues/1130,861497548,MDEyOklzc3VlQ29tbWVudDg2MTQ5NzU0OA==,3243482,2021-06-15T13:27:48Z,2021-06-15T13:27:48Z,CONTRIBUTOR,"There's a workaround: https://css-tricks.com/css-fix-for-100vh-in-mobile-webkit/

and a future fix: https://css-tricks.com/safari-15-new-ui-theme-colors-and-a-css-tricks-cameo/","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",756876238,
https://github.com/simonw/sqlite-utils/issues/269#issuecomment-861103967,https://api.github.com/repos/simonw/sqlite-utils/issues/269,861103967,MDEyOklzc3VlQ29tbWVudDg2MTEwMzk2Nw==,9599,2021-06-15T01:34:10Z,2021-06-15T01:34:10Z,OWNER,"SQLite doesn't have the concept of a boolean column, so there's not much I can do here: https://www.sqlite.org/datatype3.html#boolean_datatype","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919250621,
https://github.com/simonw/sqlite-utils/issues/266#issuecomment-861103684,https://api.github.com/repos/simonw/sqlite-utils/issues/266,861103684,MDEyOklzc3VlQ29tbWVudDg2MTEwMzY4NA==,9599,2021-06-15T01:33:13Z,2021-06-15T01:33:13Z,OWNER,Dupe of #37,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913135723,
https://github.com/simonw/datasette/issues/1377#issuecomment-861089794,https://api.github.com/repos/simonw/datasette/issues/1377,861089794,MDEyOklzc3VlQ29tbWVudDg2MTA4OTc5NA==,9599,2021-06-15T00:53:29Z,2021-06-15T00:53:29Z,OWNER,"Potential hook names:

- `skip_csrf(scope, datasette)`
- ... I can't think of any other ones I would tolerate to be honest","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920884085,
https://github.com/simonw/datasette/issues/1377#issuecomment-861087949,https://api.github.com/repos/simonw/datasette/issues/1377,861087949,MDEyOklzc3VlQ29tbWVudDg2MTA4Nzk0OQ==,9599,2021-06-15T00:49:19Z,2021-06-15T00:49:19Z,OWNER,"The new `skip_if_scope` mechanism in `asgi-csrf` https://github.com/simonw/asgi-csrf/issues/20 is designed to help here.

Now I need to design a plugin hook that allows plugins to have an opinion on whether a specific `scope` should have CSRF skipped.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920884085,
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861087651,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861087651,MDEyOklzc3VlQ29tbWVudDg2MTA4NzY1MQ==,231498,2021-06-15T00:48:37Z,2021-06-15T00:48:37Z,NONE,"@simonw -- i've created an omega-query that fetched most of what was interesting to me for a single user.

found by poking around in the ""Explorer"" tab in https://docs.github.com/en/graphql/overview/explorer

note: pagination is still required via `first` and `last` but it seems to allow unlimited history.

```
query MyQuery {
  __typename
  user(login: ""<user>"") {
    id
    pinnedItems(first: 100) {
      edges {
        node
      }
    }
    pullRequests(first: 100) {
      nodes {
        body
        title
        state
        createdAt
      }
    }
    createdAt
    issues(first: 100) {
      pageInfo {
        endCursor
        startCursor
      }
      nodes {
        title
        url
        createdAt
        body
      }
    }
    issueComments(first: 100) {
      edges {
        node {
          id
          updatedAt
          url
          body
        }
      }
    }
    repositories(first: 100) {
      nodes {
        createdAt
        description
        parent {
          name
        }
        pinnedIssues(first: 100) {
          edges {
            node {
              id
            }
          }
        }
        pinnedDiscussions(first: 100) {
          edges {
            node {
              id
            }
          }
        }
      }
    }
    starredRepositories(first: 100) {
      edges {
        node {
          id
        }
      }
    }
  }
}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216,
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861042050,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861042050,MDEyOklzc3VlQ29tbWVudDg2MTA0MjA1MA==,9599,2021-06-14T22:45:42Z,2021-06-14T22:45:42Z,MEMBER,I'm definitely interested in supporting events in this tool - see #14.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216,
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861041597,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861041597,MDEyOklzc3VlQ29tbWVudDg2MTA0MTU5Nw==,9599,2021-06-14T22:44:54Z,2021-06-14T22:44:54Z,MEMBER,Have you found a way to access events in GraphQL? I can only see way to access a timeline of events for a single issue or a single pull request. See also https://github.community/t/get-event-equivalent-for-v4/13600/2,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216,
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-861035862,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,861035862,MDEyOklzc3VlQ29tbWVudDg2MTAzNTg2Mg==,231498,2021-06-14T22:29:20Z,2021-06-14T22:29:20Z,NONE,"it looks like the v4 GraphQL API is the only way to get data beyond 90 days from GitHub.

this is significant change, but may be worth considering in the future.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216,
https://github.com/dogsheep/github-to-sqlite/issues/64#issuecomment-860895838,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64,860895838,MDEyOklzc3VlQ29tbWVudDg2MDg5NTgzOA==,231498,2021-06-14T18:23:21Z,2021-06-14T21:37:35Z,NONE,"i have a basic working version at https://github.com/khimaros/github-to-sqlite

this can be tested with `github-to-sqlite events.db khimaros/events`

caveat: the GitHub API doesn't seem to provide a complete history of events.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",920636216,
https://github.com/simonw/datasette/issues/1375#issuecomment-860548546,https://api.github.com/repos/simonw/datasette/issues/1375,860548546,MDEyOklzc3VlQ29tbWVudDg2MDU0ODU0Ng==,4068,2021-06-14T09:41:59Z,2021-06-14T09:41:59Z,NONE,"> There is a feature for this at the moment, but it's a little bit hidden: you can use `?_json=col` to tell
> Datasette that you would like a specific column to be exported as nested JSON: https://docs.datasette.io/en/stable/json_api.html#special-json-arguments

Thanks :)
 
> I considered trying to make this automatic - so it detects columns that appear to contain valid JSON and outputs them as nested objects - but the problem with that is that it can lead to inconsistent results - you might hit the API and find that not every column contains valid JSON (compared to the previous day) resulting in the API retuning string instead of the expected dictionary and breaking your code.

If a developer is not sure if the JSON fields are valid, but then retrieves and parse them, it should handle errors too. Handling inconsistent data is necessary due to the nature of SQLite. A global or dataset option to render the data as they have been defined (JSON, boolean, etc.) when requesting JSON could allow the user to download a regular JSON from the browser without having to rely on APIs. I would guess someone could just make a custom template with an extra JSON-parsed download button otherwise :)","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919508498,
https://github.com/simonw/datasette/issues/1376#issuecomment-860230663,https://api.github.com/repos/simonw/datasette/issues/1376,860230663,MDEyOklzc3VlQ29tbWVudDg2MDIzMDY2Mw==,9599,2021-06-13T15:39:37Z,2021-06-13T15:39:37Z,OWNER,Actually it looks like there is a PR open already that addresses this: #1296 ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919822817,
https://github.com/simonw/datasette/issues/1375#issuecomment-860230385,https://api.github.com/repos/simonw/datasette/issues/1375,860230385,MDEyOklzc3VlQ29tbWVudDg2MDIzMDM4NQ==,9599,2021-06-13T15:37:49Z,2021-06-13T15:37:49Z,OWNER,"There is a feature for this at the moment, but it's a little bit hidden: you can use `?_json=col` to tell
Datasette that you would like a specific column to be exported as nested JSON: https://docs.datasette.io/en/stable/json_api.html#special-json-arguments

I considered trying to make this automatic - so it detects columns that appear to contain valid JSON and outputs them as nested objects - but the problem with that is that it can lead to inconsistent results - you might hit the API and find that not every column contains valid JSON (compared to the previous day) resulting in the API retuning  string instead of the expected dictionary and breaking your code.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919508498,
https://github.com/simonw/datasette/pull/1373#issuecomment-857684605,https://api.github.com/repos/simonw/datasette/issues/1373,857684605,MDEyOklzc3VlQ29tbWVudDg1NzY4NDYwNQ==,22429695,2021-06-09T13:15:31Z,2021-06-13T15:34:07Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1373?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1373](https://codecov.io/gh/simonw/datasette/pull/1373?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (d117ba7) into [main](https://codecov.io/gh/simonw/datasette/commit/03418ee037057aa85204f5a3feb2066cbb6a9b3e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (03418ee) will **increase** coverage by `7.65%`.
> The diff coverage is `93.29%`.

> :exclamation: Current head d117ba7 differs from pull request most recent head 51ff366. Consider uploading reports for the commit 51ff366 to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1373/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1373?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)

```diff
@@            Coverage Diff             @@
##             main    #1373      +/-   ##
==========================================
+ Coverage   84.02%   91.68%   +7.65%     
==========================================
  Files          28       34       +6     
  Lines        3774     4340     +566     
==========================================
+ Hits         3171     3979     +808     
+ Misses        603      361     -242     
```


| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1373?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [datasette/inspect.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2luc3BlY3QucHk=) | `36.11% <ø> (ø)` | |
| [datasette/plugins.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3BsdWdpbnMucHk=) | `82.35% <ø> (ø)` | |
| [datasette/publish/common.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3B1Ymxpc2gvY29tbW9uLnB5) | `94.73% <ø> (ø)` | |
| [datasette/cli.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `76.33% <70.64%> (+1.19%)` | :arrow_up: |
| [datasette/filters.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2ZpbHRlcnMucHk=) | `94.35% <77.77%> (ø)` | |
| [datasette/database.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2RhdGFiYXNlLnB5) | `92.93% <89.28%> (ø)` | |
| [datasette/utils/asgi.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL2FzZ2kucHk=) | `90.98% <92.00%> (-0.95%)` | :arrow_down: |
| [datasette/publish/heroku.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3B1Ymxpc2gvaGVyb2t1LnB5) | `87.73% <94.44%> (+1.13%)` | :arrow_up: |
| [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `95.73% <94.52%> (-0.27%)` | :arrow_down: |
| [datasette/utils/\_\_init\_\_.py](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.36% <94.53%> (+0.55%)` | :arrow_up: |
| ... and [29 more](https://codecov.io/gh/simonw/datasette/pull/1373/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1373?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1373?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [e797565...51ff366](https://codecov.io/gh/simonw/datasette/pull/1373?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",916183914,
https://github.com/simonw/datasette/issues/1376#issuecomment-860229397,https://api.github.com/repos/simonw/datasette/issues/1376,860229397,MDEyOklzc3VlQ29tbWVudDg2MDIyOTM5Nw==,9599,2021-06-13T15:31:02Z,2021-06-13T15:31:02Z,OWNER,Alternative fix would be to update that section of the documentation - if the container upgrade proves tricky I can fall back on that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919822817,
https://github.com/simonw/datasette/issues/1376#issuecomment-860229226,https://api.github.com/repos/simonw/datasette/issues/1376,860229226,MDEyOklzc3VlQ29tbWVudDg2MDIyOTIyNg==,9599,2021-06-13T15:29:45Z,2021-06-13T15:29:45Z,OWNER,"Oh good catch - this is a SQLite version issue.

The `fixtures.db` file used on https://latest.datasette.io/ includes a generated column (for testing purposes) which is a feature added in SQLite 3.31.0 on 2020-01-22.

https://latest.datasette.io/-/versions

But... it looks like the packaged Datasette Docker container doesn't have that SQLite version!

I should fix that. I'm renaming this issue.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919822817,
https://github.com/simonw/sqlite-utils/issues/271#issuecomment-860142489,https://api.github.com/repos/simonw/sqlite-utils/issues/271,860142489,MDEyOklzc3VlQ29tbWVudDg2MDE0MjQ4OQ==,9599,2021-06-13T02:53:06Z,2021-06-13T02:53:06Z,OWNER,"Looks like this is the problem: https://github.com/simonw/sqlite-utils/blob/b0f9d1e494c9891ce407e27b0f5c6deeea361d30/sqlite_utils/db.py#L1724-L1742

Note how `set_cols = [col for col in all_columns if col not in pks] ` can potentially return an empty list if ALL of the columns are primary keys - but the next line of code that assigns `sql2` continues regardless, when it should instead be skipped if there are no columns in `set_cols`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919702451,
https://github.com/dogsheep/twitter-to-sqlite/issues/57#issuecomment-860063190,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/57,860063190,MDEyOklzc3VlQ29tbWVudDg2MDA2MzE5MA==,232237,2021-06-12T14:46:44Z,2021-06-12T14:46:44Z,NONE,I'm having the same issue (same versions of python and twitter-to-sqlite). It's the `user-timeline` command. Other commands are working. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907645813,
https://github.com/simonw/datasette/issues/1286#issuecomment-860047794,https://api.github.com/repos/simonw/datasette/issues/1286,860047794,MDEyOklzc3VlQ29tbWVudDg2MDA0Nzc5NA==,4068,2021-06-12T12:36:15Z,2021-06-12T12:36:15Z,NONE,"@mroswell That is a very nice solution. I wonder if custom classes, like `col-columnName-value` could be automatically added to cells when facets on such column are enabled, to allow custom styling without having to modify templates or add custom JavaScript code.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849220154,
https://github.com/simonw/sqlite-utils/issues/269#issuecomment-860031217,https://api.github.com/repos/simonw/sqlite-utils/issues/269,860031217,MDEyOklzc3VlQ29tbWVudDg2MDAzMTIxNw==,4068,2021-06-12T10:01:53Z,2021-06-12T10:01:53Z,NONE,"`sqlite-utils transform` does not allow setting the column type to boolean:
```
Error: Invalid value for '--type': 'bool' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919250621,
https://github.com/simonw/sqlite-utils/issues/270#issuecomment-860031071,https://api.github.com/repos/simonw/sqlite-utils/issues/270,860031071,MDEyOklzc3VlQ29tbWVudDg2MDAzMTA3MQ==,4068,2021-06-12T10:00:24Z,2021-06-12T10:00:24Z,NONE,"Sure, I am sorry if my message hasn't been clear enough. I am also a new user :)

At the beginning, I just call `sqlite-utils insert ""$db"" ""$table"" ""$jsonfile""` to create the database. sqlite-utils convert JSON values into `TEXT`, when it tries to determine the schema automatically. I then try to transform the table to set `JSON` as type:

```
sqlite-utils transform species.sqlite species --type criteria json
Usage: sqlite-utils transform [OPTIONS] PATH TABLE
Try 'sqlite-utils transform --help' for help.

Error: Invalid value for '--type': 'json' is not one of 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919314806,
https://github.com/simonw/sqlite-utils/issues/270#issuecomment-859986489,https://api.github.com/repos/simonw/sqlite-utils/issues/270,859986489,MDEyOklzc3VlQ29tbWVudDg1OTk4NjQ4OQ==,9599,2021-06-12T02:47:12Z,2021-06-12T02:47:12Z,OWNER,"Can you expand on what you'd like to change here? The library and CLI tool already allow JSON data to be stored in columns:

- https://sqlite-utils.datasette.io/en/stable/cli.html#nested-json-values
- https://sqlite-utils.datasette.io/en/stable/python-api.html#storing-json","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919314806,
https://github.com/simonw/sqlite-utils/issues/269#issuecomment-859940977,https://api.github.com/repos/simonw/sqlite-utils/issues/269,859940977,MDEyOklzc3VlQ29tbWVudDg1OTk0MDk3Nw==,4068,2021-06-11T22:33:08Z,2021-06-11T22:33:08Z,NONE,"`true` and `false` json values are cast to integer, which is not optimal.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919250621,
https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859898736,https://api.github.com/repos/simonw/sqlite-utils/issues/268,859898736,MDEyOklzc3VlQ29tbWVudDg1OTg5ODczNg==,9599,2021-06-11T20:37:44Z,2021-06-11T20:37:44Z,OWNER,"From the prototype:
```
% sqlite-utils schema 24ways.db
CREATE TABLE [articles] (
               [title] TEXT  ,
   [contents] TEXT  ,
   [year] TEXT  ,
   [author] TEXT  ,
   [author_slug] TEXT  ,
   [published] TEXT  ,
   [url] TEXT  ,
   [topic] TEXT  
        );
CREATE VIRTUAL TABLE ""articles_fts"" USING FTS5 (
                title, author, contents,
                content=""articles""
            );
CREATE TABLE 'articles_fts_data'(id INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE 'articles_fts_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
CREATE TABLE 'articles_fts_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
CREATE TABLE 'articles_fts_config'(k PRIMARY KEY, v) WITHOUT ROWID;
% sqlite-utils schema 24ways.db | sqlite3 /tmp/boo.db
Error: near line 15: table 'articles_fts_data' already exists
Error: near line 16: table 'articles_fts_idx' already exists
Error: near line 17: table 'articles_fts_docsize' already exists
Error: near line 18: table 'articles_fts_config' already exists
```
The problem here is that the `CREATE VIRTUAL TABLE ""articles_fts""...` line causes those next four tables to be created - but that means that piping the output of this command into `sqlite3` in order to re-create those tables throws errors.

I don't think this matters. I see this tool as more for introspection than for recreating table structures.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919181559,
https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859895540,https://api.github.com/repos/simonw/sqlite-utils/issues/268,859895540,MDEyOklzc3VlQ29tbWVudDg1OTg5NTU0MA==,9599,2021-06-11T20:30:34Z,2021-06-11T20:30:34Z,OWNER,"You can currently see the `sql` on the CLI using:

    % sqlite-utils rows fixtures.db sqlite_master -c name -c sql
    name                                                            sql
    --------------------------------------------------------------  ----------------------------------------------------------------------------------------------------------------------------------------------------------------
    simple_primary_key                                              CREATE TABLE simple_primary_key (
                                                                      id varchar(30) primary key,
                                                                      content text
                                                                    )
    sqlite_autoindex_simple_primary_key_1
    primary_key_multiple_columns                                    CREATE TABLE primary_key_multiple_columns (
                                                                      id varchar(30) primary key,
                                                                      content text,
                                                                      content2 text
                                                                    )
    sqlite_autoindex_primary_key_multiple_columns_1
    primary_key_multiple_columns_explicit_label                     CREATE TABLE primary_key_multiple_columns_explicit_label (
                                                                      id varchar(30) primary key,
                                                                      content text,
                                                                      content2 text
                                                                    )
    sqlite_autoindex_primary_key_multiple_columns_explicit_label_1
    compound_primary_key                                            CREATE TABLE compound_primary_key (
                                                                      pk1 varchar(30),
                                                                      pk2 varchar(30),
                                                                      content text,
                                                                      PRIMARY KEY (pk1, pk2)
                                                                    )
    sqlite_autoindex_compound_primary_key_1
    compound_three_primary_keys                                     CREATE TABLE compound_three_primary_keys (
                                                                      pk1 varchar(30),
                                                                      pk2 varchar(30),
                                                                      pk3 varchar(30),
                                                                      content text,
                                                                      PRIMARY KEY (pk1, pk2, pk3)
                                                                    )
    sqlite_autoindex_compound_three_primary_keys_1
    foreign_key_references                                          CREATE TABLE foreign_key_references (
                                                                      pk varchar(30) primary key,
                                                                      foreign_key_with_label varchar(30),
                                                                      foreign_key_with_no_label varchar(30),
                                                                      FOREIGN KEY (foreign_key_with_label) REFERENCES simple_primary_key(id),
                                                                      FOREIGN KEY (foreign_key_with_no_label) REFERENCES primary_key_multiple_columns(id)
                                                                    )
    sqlite_autoindex_foreign_key_references_1
    sortable                                                        CREATE TABLE sortable (
                                                                      pk1 varchar(30),
                                                                      pk2 varchar(30),
                                                                      content text,
                                                                      sortable integer,
                                                                      sortable_with_nulls real,
                                                                      sortable_with_nulls_2 real,
                                                                      text text,
                                                                      PRIMARY KEY (pk1, pk2)
                                                                    )
    sqlite_autoindex_sortable_1
    no_primary_key                                                  CREATE TABLE no_primary_key (
                                                                      content text,
                                                                      a text,
                                                                      b text,
                                                                      c text
                                                                    )
    123_starts_with_digits                                          CREATE TABLE [123_starts_with_digits] (
                                                                      content text
                                                                    )
    paginated_view                                                  CREATE VIEW paginated_view AS
                                                                        SELECT
                                                                            content,
                                                                            '- ' || content || ' -' AS content_extra
                                                                        FROM no_primary_key
    Table With Space In Name                                        CREATE TABLE ""Table With Space In Name"" (
                                                                      pk varchar(30) primary key,
                                                                      content text
                                                                    )
    sqlite_autoindex_Table With Space In Name_1
    table/with/slashes.csv                                          CREATE TABLE ""table/with/slashes.csv"" (
                                                                      pk varchar(30) primary key,
                                                                      content text
                                                                    )
    sqlite_autoindex_table/with/slashes.csv_1
    complex_foreign_keys                                            CREATE TABLE ""complex_foreign_keys"" (
                                                                      pk varchar(30) primary key,
                                                                      f1 text,
                                                                      f2 text,
                                                                      f3 text,
                                                                      FOREIGN KEY (""f1"") REFERENCES [simple_primary_key](id),
                                                                      FOREIGN KEY (""f2"") REFERENCES [simple_primary_key](id),
                                                                      FOREIGN KEY (""f3"") REFERENCES [simple_primary_key](id)
                                                                    )
    sqlite_autoindex_complex_foreign_keys_1
    custom_foreign_key_label                                        CREATE TABLE ""custom_foreign_key_label"" (
                                                                      pk varchar(30) primary key,
                                                                      foreign_key_with_custom_label text,
                                                                      FOREIGN KEY (""foreign_key_with_custom_label"") REFERENCES [primary_key_multiple_columns_explicit_label](id)
                                                                    )
    sqlite_autoindex_custom_foreign_key_label_1
    units                                                           CREATE TABLE units (
                                                                      pk integer primary key,
                                                                      distance int,
                                                                      frequency int
                                                                    )
    searchable                                                      CREATE TABLE searchable (
                                                                      pk integer primary key,
                                                                      text1 text,
                                                                      text2 text,
                                                                      [name with . and spaces] text
                                                                    )
    searchable_fts                                                  CREATE VIRTUAL TABLE ""searchable_fts""
                                                                        USING FTS3 (text1, text2, [name with . and spaces], content=""searchable"")
    searchable_fts_content                                          CREATE TABLE 'searchable_fts_content'(docid INTEGER PRIMARY KEY, 'c0text1', 'c1text2', 'c2name with . and spaces', 'c3content')
    searchable_fts_segments                                         CREATE TABLE 'searchable_fts_segments'(blockid INTEGER PRIMARY KEY, block BLOB)
    searchable_fts_segdir                                           CREATE TABLE 'searchable_fts_segdir'(level INTEGER,idx INTEGER,start_block INTEGER,leaves_end_block INTEGER,end_block INTEGER,root BLOB,PRIMARY KEY(level, idx))
    sqlite_autoindex_searchable_fts_segdir_1
    select                                                          CREATE TABLE [select] (
                                                                      [group] text,
                                                                      [having] text,
                                                                      [and] text
                                                                    )
    facet_cities                                                    CREATE TABLE facet_cities (
                                                                        id integer primary key,
                                                                        name text
                                                                    )
    simple_view                                                     CREATE VIEW simple_view AS
                                                                        SELECT content, upper(content) AS upper_content FROM simple_primary_key
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919181559,
https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859894105,https://api.github.com/repos/simonw/sqlite-utils/issues/268,859894105,MDEyOklzc3VlQ29tbWVudDg1OTg5NDEwNQ==,9599,2021-06-11T20:28:52Z,2021-06-11T20:28:52Z,OWNER,"Out of interest, here are the rows from that table where `sql` is `null`: https://latest.datasette.io/fixtures?sql=select%0D%0A++*%0D%0Afrom%0D%0A++sqlite_master%0D%0Awhere%0D%0A++sql+is+null

```csv
type,name,tbl_name,rootpage,sql
index,sqlite_autoindex_simple_primary_key_1,simple_primary_key,3,
index,sqlite_autoindex_primary_key_multiple_columns_1,primary_key_multiple_columns,5,
index,sqlite_autoindex_primary_key_multiple_columns_explicit_label_1,primary_key_multiple_columns_explicit_label,7,
index,sqlite_autoindex_compound_primary_key_1,compound_primary_key,9,
index,sqlite_autoindex_compound_three_primary_keys_1,compound_three_primary_keys,11,
index,sqlite_autoindex_foreign_key_references_1,foreign_key_references,14,
index,sqlite_autoindex_sortable_1,sortable,16,
index,sqlite_autoindex_Table With Space In Name_1,Table With Space In Name,20,
index,sqlite_autoindex_table/with/slashes.csv_1,table/with/slashes.csv,22,
index,sqlite_autoindex_complex_foreign_keys_1,complex_foreign_keys,24,
index,sqlite_autoindex_custom_foreign_key_label_1,custom_foreign_key_label,26,
index,sqlite_autoindex_tags_1,tags,31,
index,sqlite_autoindex_searchable_tags_1,searchable_tags,34,
index,sqlite_autoindex_searchable_fts_segdir_1,searchable_fts_segdir,37,
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919181559,
https://github.com/simonw/sqlite-utils/issues/268#issuecomment-859888469,https://api.github.com/repos/simonw/sqlite-utils/issues/268,859888469,MDEyOklzc3VlQ29tbWVudDg1OTg4ODQ2OQ==,9599,2021-06-11T20:26:20Z,2021-06-11T20:26:20Z,OWNER,`sqlite-utils schema data.db` could output the same thing to the console.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",919181559,
https://github.com/simonw/datasette/pull/1374#issuecomment-859572791,https://api.github.com/repos/simonw/datasette/issues/1374,859572791,MDEyOklzc3VlQ29tbWVudDg1OTU3Mjc5MQ==,22429695,2021-06-11T13:12:58Z,2021-06-11T13:12:58Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1374?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1374](https://codecov.io/gh/simonw/datasette/pull/1374?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (0ef0dd5) into [main](https://codecov.io/gh/simonw/datasette/commit/cd7678fde65319d7b6955ce9f4678ba4b9e64b66?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (cd7678f) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1374/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1374?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)

```diff
@@           Coverage Diff           @@
##             main    #1374   +/-   ##
=======================================
  Coverage   91.68%   91.68%           
=======================================
  Files          34       34           
  Lines        4340     4340           
=======================================
  Hits         3979     3979           
  Misses        361      361           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1374?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1374?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [cd7678f...0ef0dd5](https://codecov.io/gh/simonw/datasette/pull/1374?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",918730335,
https://github.com/simonw/datasette/issues/858#issuecomment-858831895,https://api.github.com/repos/simonw/datasette/issues/858,858831895,MDEyOklzc3VlQ29tbWVudDg1ODgzMTg5NQ==,56045588,2021-06-10T17:44:09Z,2021-06-10T17:44:09Z,NONE,"any fixes for that recursive issue with temp file? I get it using both heroku and cloudrun, although it seems to still publish and deploy fine","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642388564,
https://github.com/simonw/datasette/issues/858#issuecomment-858813675,https://api.github.com/repos/simonw/datasette/issues/858,858813675,MDEyOklzc3VlQ29tbWVudDg1ODgxMzY3NQ==,56045588,2021-06-10T17:27:46Z,2021-06-10T17:27:46Z,NONE,shell=True is added to line 56 (I guess it used to be 54) of heroku.py as detailed in the original issue. (for posterity),"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",642388564,
https://github.com/simonw/datasette/issues/1371#issuecomment-858099514,https://api.github.com/repos/simonw/datasette/issues/1371,858099514,MDEyOklzc3VlQ29tbWVudDg1ODA5OTUxNA==,9599,2021-06-09T21:03:49Z,2021-06-09T21:03:49Z,OWNER,I'll release these as an alpha straight away - it makes sense to have plugin hook changes available for people to test as alpha dependencies ASAP.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",915455228,
https://github.com/simonw/datasette/pull/1370#issuecomment-857298526,https://api.github.com/repos/simonw/datasette/issues/1370,857298526,MDEyOklzc3VlQ29tbWVudDg1NzI5ODUyNg==,25778,2021-06-09T01:18:59Z,2021-06-09T01:18:59Z,CONTRIBUTOR,"I'm happy to grab some or all of these in this PR, if you want. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",914130834,
https://github.com/simonw/datasette/pull/1370#issuecomment-857139881,https://api.github.com/repos/simonw/datasette/issues/1370,857139881,MDEyOklzc3VlQ29tbWVudDg1NzEzOTg4MQ==,9599,2021-06-08T20:58:41Z,2021-06-08T20:58:41Z,OWNER,We can remove a bunch of unnecessary `str(path)` calls too - this search finds a bunch of possible candidates: https://ripgrep.datasette.io/-/ripgrep?pattern=str%5C%28.*%28db%7Cpath%29&glob=datasette%2F**%2F*.py,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",914130834,
https://github.com/simonw/datasette/pull/1368#issuecomment-856182547,https://api.github.com/repos/simonw/datasette/issues/1368,856182547,MDEyOklzc3VlQ29tbWVudDg1NjE4MjU0Nw==,2670795,2021-06-07T18:59:47Z,2021-06-07T23:04:25Z,CONTRIBUTOR,"Note that if we went with a ""update_metadata"" hook, the hook signature would look something like this (it would return nothing):

```
update_metadata(
  datasette=self, metadata=metadata, key=key, database=database, table=table,
  fallback=fallback
)
```

The Datasette function `_metadata_recursive_update(self, orig, updated)` would disappear into the plugins. Doing this, though, we'd lose the easy ability to make the local metadata.yaml immutable (since we'd no longer have the recursive update).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913865304,
https://github.com/simonw/sqlite-utils/issues/266#issuecomment-856231119,https://api.github.com/repos/simonw/sqlite-utils/issues/266,856231119,MDEyOklzc3VlQ29tbWVudDg1NjIzMTExOQ==,9599,2021-06-07T20:26:05Z,2021-06-07T20:26:05Z,OWNER,"https://github.com/python/cpython/blob/2ab27c4af4ddf7528e1375e77c787c7fbb09b5e6/Lib/typing.py#L2173-L2195

In Python 3.6 or higher can do this:

```python
class Employee(NamedTuple):
    name: str
    id: int
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913135723,
https://github.com/simonw/datasette/issues/1365#issuecomment-856212136,https://api.github.com/repos/simonw/datasette/issues/1365,856212136,MDEyOklzc3VlQ29tbWVudDg1NjIxMjEzNg==,9599,2021-06-07T19:54:04Z,2021-06-07T19:54:04Z,OWNER,"I've hit this one too. I agree, fixing this in Datasette itself is better than fixing it in the tests across multiple other projects.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913017577,
https://github.com/simonw/datasette/issues/1369#issuecomment-856208637,https://api.github.com/repos/simonw/datasette/issues/1369,856208637,MDEyOklzc3VlQ29tbWVudDg1NjIwODYzNw==,9599,2021-06-07T19:47:23Z,2021-06-07T19:47:23Z,OWNER,No point in showing the IDs twice if the blue label doesn't differ from the gray ID,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913900374,
https://github.com/simonw/datasette/issues/1367#issuecomment-856160770,https://api.github.com/repos/simonw/datasette/issues/1367,856160770,MDEyOklzc3VlQ29tbWVudDg1NjE2MDc3MA==,9599,2021-06-07T18:22:33Z,2021-06-07T18:22:33Z,OWNER,Here's why: https://github.com/simonw/datasette/blob/03ec71193b9545536898a4bc7493274fec48bdd7/datasette/static/app.css#L455-L458,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913823889,
https://github.com/simonw/datasette/issues/1366#issuecomment-856147969,https://api.github.com/repos/simonw/datasette/issues/1366,856147969,MDEyOklzc3VlQ29tbWVudDg1NjE0Nzk2OQ==,9599,2021-06-07T18:03:03Z,2021-06-07T18:03:03Z,OWNER,"Here's an example of a test that uses it. It's necessary because sometimes fixtures that create temporary directories break in unexpected ways:

https://github.com/simonw/datasette/blob/0a7621f96f8ad14da17e7172e8a7bce24ef78966/tests/test_plugins.py#L658-L666","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913809802,
https://github.com/simonw/datasette/issues/1366#issuecomment-856147450,https://api.github.com/repos/simonw/datasette/issues/1366,856147450,MDEyOklzc3VlQ29tbWVudDg1NjE0NzQ1MA==,9599,2021-06-07T18:02:13Z,2021-06-07T18:02:13Z,OWNER,"The hack in question is this fixture, which I've been using in an ad-hoc manner to work around errors while running the tests: https://github.com/simonw/datasette/blob/030deb4b25cda842ff7129ab7c18550c44dd8379/tests/conftest.py#L62-L75

I don't understand the underlying issue well enough to know how to get rid of it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913809802,
https://github.com/simonw/sqlite-utils/issues/266#issuecomment-855611939,https://api.github.com/repos/simonw/sqlite-utils/issues/266,855611939,MDEyOklzc3VlQ29tbWVudDg1NTYxMTkzOQ==,9599,2021-06-07T06:07:41Z,2021-06-07T06:07:41Z,OWNER,"Looks like this is the way to do this:
```python
Point = typing.NamedTuple(
    ""Point"", [('x', int), ('y', int)]
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",913135723,
https://github.com/simonw/datasette/issues/1362#issuecomment-855430317,https://api.github.com/repos/simonw/datasette/issues/1362,855430317,MDEyOklzc3VlQ29tbWVudDg1NTQzMDMxNw==,9599,2021-06-06T17:07:48Z,2021-06-06T17:07:48Z,OWNER,"I guess I can offer a `disable_csp` setting so that people with complex custom templates aren't completely blocked from using them with Datasette, but maybe it would be better not to offer that? Or to offer it as a `datasette-insecure-csp` plugin instead?

I like the idea of very actively encouraging CSP across all Datasette projects, but I'm nervous about making the software unusable for certain edge cases.

Maybe require CSP and wait for someone to complain?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855429111,https://api.github.com/repos/simonw/datasette/issues/1362,855429111,MDEyOklzc3VlQ29tbWVudDg1NTQyOTExMQ==,9599,2021-06-06T16:59:05Z,2021-06-06T17:00:15Z,OWNER,"Twitter conversation: https://twitter.com/simonw/status/1401565566045806594

@dracos provided some really useful code examples there:

> We generate it here: https://github.com/mysociety/fixmystreet/blob/e9fec4e567e7148ed128816e5770c2963be51af6/perllib/FixMyStreet/Cobrand/Default.pm#L89-L90
And use it e.g. https://github.com/mysociety/fixmystreet/blob/ba6788cd25d8f471a4e3308403607627b4d2f4f6/templates/web/base/common_header_tags.html or https://github.com/mysociety/fixmystreet/blob/cb4f2b96364d151988b5c664888468b25cc62240/templates/web/fixmystreet.com/header/css.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855428601,https://api.github.com/repos/simonw/datasette/issues/1362,855428601,MDEyOklzc3VlQ29tbWVudDg1NTQyODYwMQ==,9599,2021-06-06T16:55:33Z,2021-06-06T16:55:33Z,OWNER,"> No, because Vary header is about _request_ headers that cause the response to vary, not response headers.

Hah, of course! Thanks for the correction. So the nonce mechanism would actually be pretty great here, especially for the `extra_body_script()` hook.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855428296,https://api.github.com/repos/simonw/datasette/issues/1362,855428296,MDEyOklzc3VlQ29tbWVudDg1NTQyODI5Ng==,154364,2021-06-06T16:53:20Z,2021-06-06T16:53:20Z,NONE,"> Presumably this would also require adding Content-Security-Policy to the Vary header though, which will have a nasty effect on Cloudflare and Fastly and such like.

No, because Vary header is about *request* headers that cause the response to vary, not response headers.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855427396,https://api.github.com/repos/simonw/datasette/issues/1362,855427396,MDEyOklzc3VlQ29tbWVudDg1NTQyNzM5Ng==,9599,2021-06-06T16:46:17Z,2021-06-06T16:46:17Z,OWNER,"Mind you, since that plugin hook looks like this:

```python
@hookimpl
def extra_body_script():
    return {
        ""module"": True,
        ""script"": ""console.log('Your JavaScript goes here...')""
    }
```
Having it calculate a sha256 hash wouldn't be difficult.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855426750,https://api.github.com/repos/simonw/datasette/issues/1362,855426750,MDEyOklzc3VlQ29tbWVudDg1NTQyNjc1MA==,9599,2021-06-06T16:41:30Z,2021-06-06T16:44:49Z,OWNER,"This is from the current `base.html` template: https://github.com/simonw/datasette/blob/030deb4b25cda842ff7129ab7c18550c44dd8379/datasette/templates/base.html#L62-L66

Which includes this: https://github.com/simonw/datasette/blob/030deb4b25cda842ff7129ab7c18550c44dd8379/datasette/templates/_close_open_menus.html#L1-L16

The `body_scripts` bit is for this `extra_body_script` plugin hook, which is the thing that will be the most affected by implementing CSP: https://docs.datasette.io/en/stable/plugin_hooks.html#extra-body-script-template-database-table-columns-view-name-request-datasette","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855426516,https://api.github.com/repos/simonw/datasette/issues/1362,855426516,MDEyOklzc3VlQ29tbWVudDg1NTQyNjUxNg==,9599,2021-06-06T16:39:34Z,2021-06-06T16:39:34Z,OWNER,The reason Datasette uses small inline scripts right now is to avoid the overhead of an extra HTTP request for a JavaScript file - but these are both inherently cachable and perform much better under HTTP/2 so that's likely a false optimization.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855426314,https://api.github.com/repos/simonw/datasette/issues/1362,855426314,MDEyOklzc3VlQ29tbWVudDg1NTQyNjMxNA==,9599,2021-06-06T16:38:04Z,2021-06-06T16:38:04Z,OWNER,"The other option for inline scripts is the CSP nonce:

    Content-Security-Policy: script-src 'nonce-2726c7f26c'

Then:

    <script nonce=""2726c7f26c"">
      var inline = 1;
    </script>

Since an attacker can't guess what the nonce will be it prevents them from injecting their own script block - this seems easier to make available to plugins than a full hashing mechanism, just make `{{ csp_nonce() }}` available to the template.

That template function can then be smart enough to set a flag which Datasette uses to decide if the `script-src 'nonce-2726c7f26c'` policy should be sent or not.

Presumably this would also require adding `Content-Security-Policy` to the `Vary` header though, which will have a nasty effect on Cloudflare and Fastly and such like.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855418899,https://api.github.com/repos/simonw/datasette/issues/1362,855418899,MDEyOklzc3VlQ29tbWVudDg1NTQxODg5OQ==,9599,2021-06-06T15:42:55Z,2021-06-06T15:42:55Z,OWNER,Another consideration: testing that this works correctly could require adoption of a real browser test environment (probably Cypress or maybe Playwright) to execute tests that will fail if CSP is violated.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855418698,https://api.github.com/repos/simonw/datasette/issues/1362,855418698,MDEyOklzc3VlQ29tbWVudDg1NTQxODY5OA==,9599,2021-06-06T15:41:24Z,2021-06-06T15:41:24Z,OWNER,"I think the best way to answer these questions is with some prototyping - of both Datasette and some of the existing JavaScript plugins.

I can start with a `datasette-experimental-csp` plugin that sets the header (and could even run an optional report URI mechanism).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855418401,https://api.github.com/repos/simonw/datasette/issues/1362,855418401,MDEyOklzc3VlQ29tbWVudDg1NTQxODQwMQ==,9599,2021-06-06T15:39:38Z,2021-06-06T15:39:38Z,OWNER,"The security benefit of forcing all JavaScript plugins to be written as CSP-friendly external scripts is very compelling though.

Other plugin-heavy ecosystems such as WordPress have suffered greatly from insecurely written plugins - could this be a huge security win for the Datasette ecosystem generally?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/1362#issuecomment-855418065,https://api.github.com/repos/simonw/datasette/issues/1362,855418065,MDEyOklzc3VlQ29tbWVudDg1NTQxODA2NQ==,9599,2021-06-06T15:37:11Z,2021-06-06T15:37:11Z,OWNER,"The easiest way to apply CSP is to remove all inline `<script>` blocks (Datasette has a few) and instead serve JavaScript as separate linked files.

It's possible to keep inline script blocks by calculating a hash of their content and adding a `Content-Security-Policy: script-src 'sha256-B2yPHKaXnvFWtRChIbabYmUBFZdVfKKXHbWtWidDVF8='` to the policy. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src

This could be achieved with some Django template tricks, but it feels very risky - and done carelessly could end up calculating a hash of a reflected XSS attack!

The biggest challenge I see around here involves plugins and custom templates. Adopting CSP would require plugins to avoid using any inline scripts, instead keeping their entire implementations in `.js` files.

That's maybe not a bad thing, but it represents a big commitment. It would need to be adopted before Datasette 1.0.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912864936,
https://github.com/simonw/datasette/issues/283#issuecomment-855369819,https://api.github.com/repos/simonw/datasette/issues/283,855369819,MDEyOklzc3VlQ29tbWVudDg1NTM2OTgxOQ==,9599,2021-06-06T09:40:18Z,2021-06-06T09:40:18Z,OWNER,"> One note on using this pragma I got an error on starting datasette `no such table: pragma_database_list`.
> 
> I diagnosed this to an older version of sqlite3 (3.14.2) and upgrading to a newer version (3.34.2) fixed the issue.

That issue is fixed in #1276.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",325958506,
https://github.com/simonw/datasette/issues/1361#issuecomment-855308811,https://api.github.com/repos/simonw/datasette/issues/1361,855308811,MDEyOklzc3VlQ29tbWVudDg1NTMwODgxMQ==,9599,2021-06-05T23:16:21Z,2021-06-05T23:16:21Z,OWNER,That seems to have fixed it. I'd love to get rid of this `restore_working_directory` hack entirely.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912485040,
https://github.com/simonw/datasette/issues/1361#issuecomment-855307292,https://api.github.com/repos/simonw/datasette/issues/1361,855307292,MDEyOklzc3VlQ29tbWVudDg1NTMwNzI5Mg==,9599,2021-06-05T22:59:35Z,2021-06-05T22:59:35Z,OWNER,"That broke things. Here's how `pytest-cov` fixed a similar issue: https://github.com/pytest-dev/pytest-cov/commit/7ccb1783bf8290447df58deeb41eae74296a6d9b

See also https://github.com/nedbat/coveragepy/issues/881 and https://github.com/pytest-dev/pytest-cov/issues/306 and https://github.com/nedbat/coveragepy/issues/824","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912485040,
https://github.com/simonw/datasette/issues/1361#issuecomment-855306497,https://api.github.com/repos/simonw/datasette/issues/1361,855306497,MDEyOklzc3VlQ29tbWVudDg1NTMwNjQ5Nw==,9599,2021-06-05T22:51:01Z,2021-06-05T22:51:01Z,OWNER,I'm going to try removing that `restore_working_directory` fixture entirely.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912485040,
https://github.com/simonw/datasette/issues/1361#issuecomment-855306347,https://api.github.com/repos/simonw/datasette/issues/1361,855306347,MDEyOklzc3VlQ29tbWVudDg1NTMwNjM0Nw==,9599,2021-06-05T22:49:30Z,2021-06-05T22:49:30Z,OWNER,"Stack Overflow: https://stackoverflow.com/a/49367679/6083

> The answer was that `os.chdir()` had been set to the deleted directory by accident. The directory was missing, but the error happened (it seems) at the attempt to get it with `os.getcwd()`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912485040,
https://github.com/simonw/datasette/issues/1360#issuecomment-855303776,https://api.github.com/repos/simonw/datasette/issues/1360,855303776,MDEyOklzc3VlQ29tbWVudDg1NTMwMzc3Ng==,9599,2021-06-05T22:23:23Z,2021-06-05T22:23:23Z,OWNER,"Worth noting that I found this issue myself, and to my knowledge it has not been uncovered by anyone else prior to the patch being released.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912464443,
https://github.com/simonw/datasette/issues/1360#issuecomment-855303649,https://api.github.com/repos/simonw/datasette/issues/1360,855303649,MDEyOklzc3VlQ29tbWVudDg1NTMwMzY0OQ==,9599,2021-06-05T22:22:06Z,2021-06-05T22:22:06Z,OWNER,I've released fixes in both 0.56.1 and 0.57.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912464443,
https://github.com/simonw/datasette/issues/1358#issuecomment-855302339,https://api.github.com/repos/simonw/datasette/issues/1358,855302339,MDEyOklzc3VlQ29tbWVudDg1NTMwMjMzOQ==,9599,2021-06-05T22:08:16Z,2021-06-05T22:08:16Z,OWNER,"Release notes are in, pushing the release now.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912418094,
https://github.com/simonw/datasette/issues/1358#issuecomment-855302320,https://api.github.com/repos/simonw/datasette/issues/1358,855302320,MDEyOklzc3VlQ29tbWVudDg1NTMwMjMyMA==,9599,2021-06-05T22:08:06Z,2021-06-05T22:08:06Z,OWNER,See #1360 for the security fix.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912418094,
https://github.com/simonw/datasette/issues/1358#issuecomment-855288228,https://api.github.com/repos/simonw/datasette/issues/1358,855288228,MDEyOklzc3VlQ29tbWVudDg1NTI4ODIyOA==,9599,2021-06-05T19:57:18Z,2021-06-05T19:57:18Z,OWNER,"There's also a security fix I need to make, so I'm not going to block this on wrapping up the work on the new Docker image testing from #1344 before putting out this release.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912418094,
https://github.com/simonw/datasette/pull/1291#issuecomment-855287200,https://api.github.com/repos/simonw/datasette/issues/1291,855287200,MDEyOklzc3VlQ29tbWVudDg1NTI4NzIwMA==,9599,2021-06-05T19:48:36Z,2021-06-05T19:48:36Z,OWNER,"This is great, thank you.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849582643,
https://github.com/simonw/datasette/pull/1291#issuecomment-812815358,https://api.github.com/repos/simonw/datasette/issues/1291,812815358,MDEyOklzc3VlQ29tbWVudDgxMjgxNTM1OA==,22429695,2021-04-03T05:32:50Z,2021-06-05T19:48:30Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1291?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1291](https://codecov.io/gh/simonw/datasette/pull/1291?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (8789157) into [main](https://codecov.io/gh/simonw/datasette/commit/0a7621f96f8ad14da17e7172e8a7bce24ef78966?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (0a7621f) will **not change** coverage.
> The diff coverage is `n/a`.

> :exclamation: Current head 8789157 differs from pull request most recent head 9bf089f. Consider uploading reports for the commit 9bf089f to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1291/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1291?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)

```diff
@@           Coverage Diff           @@
##             main    #1291   +/-   ##
=======================================
  Coverage   91.51%   91.51%           
=======================================
  Files          34       34           
  Lines        4255     4255           
=======================================
  Hits         3894     3894           
  Misses        361      361           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1291?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1291?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [0a7621f...9bf089f](https://codecov.io/gh/simonw/datasette/pull/1291?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849582643,
https://github.com/simonw/datasette/issues/1356#issuecomment-855282466,https://api.github.com/repos/simonw/datasette/issues/1356,855282466,MDEyOklzc3VlQ29tbWVudDg1NTI4MjQ2Ng==,9599,2021-06-05T19:05:06Z,2021-06-05T19:05:06Z,OWNER,Yeah that's a good point. I avoided making them sub-commands because `datasette serve` already supports the multitude of other arguments they also need... but actually that was just me being lazy - I can easily share arguments between multiple functions like I do in `sqlite-utils` itself: https://github.com/simonw/sqlite-utils/blob/d1a372b3006e6cf7d2017b3ddc484bf5c033e45d/sqlite_utils/cli.py#L46,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/pull/1357#issuecomment-855281774,https://api.github.com/repos/simonw/datasette/issues/1357,855281774,MDEyOklzc3VlQ29tbWVudDg1NTI4MTc3NA==,22429695,2021-06-05T18:59:07Z,2021-06-05T18:59:07Z,NONE,"# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report
> Merging [#1357](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (1b27643) into [main](https://codecov.io/gh/simonw/datasette/commit/6e9b07be92905011211d8df7a872fb7c1f2737b2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (6e9b07b) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1357/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)

```diff
@@           Coverage Diff           @@
##             main    #1357   +/-   ##
=======================================
  Coverage   91.71%   91.72%           
=======================================
  Files          34       34           
  Lines        4333     4336    +3     
=======================================
+ Hits         3974     3977    +3     
  Misses        359      359           
```


| [Impacted Files](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | |
|---|---|---|
| [datasette/utils/testing.py](https://codecov.io/gh/simonw/datasette/pull/1357/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL3Rlc3RpbmcucHk=) | `95.38% <ø> (ø)` | |
| [datasette/app.py](https://codecov.io/gh/simonw/datasette/pull/1357/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `95.71% <100.00%> (+0.01%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [6e9b07b...1b27643](https://codecov.io/gh/simonw/datasette/pull/1357?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",912394511,
https://github.com/simonw/datasette/issues/1238#issuecomment-855278998,https://api.github.com/repos/simonw/datasette/issues/1238,855278998,MDEyOklzc3VlQ29tbWVudDg1NTI3ODk5OA==,9599,2021-06-05T18:37:16Z,2021-06-05T18:37:16Z,OWNER,"Alternative idea: populate `request.scope` with a new `route_path` which is the base-url-stripped version, which we then use for other routing operations.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813899472,
https://github.com/simonw/datasette/issues/1238#issuecomment-855278540,https://api.github.com/repos/simonw/datasette/issues/1238,855278540,MDEyOklzc3VlQ29tbWVudDg1NTI3ODU0MA==,9599,2021-06-05T18:33:25Z,2021-06-05T18:33:25Z,OWNER,"Got the test to pass by ensuring the tests don't accidentally double-rewrite the path.

Ran into a new problem:
```
    @pytest.mark.asyncio
    @pytest.mark.parametrize(
        ""prefix,expected_path"", [(None, ""/asgi-scope""), (""/prefix/"", ""/prefix/asgi-scope"")]
    )
    async def test_client_path(datasette, prefix, expected_path):
        original_base_url = datasette._settings[""base_url""]
        try:
            if prefix is not None:
                datasette._settings[""base_url""] = prefix
            response = await datasette.client.get(""/asgi-scope"")
            path = response.json()[""path""]
>           assert path == expected_path
E           AssertionError: assert '/asgi-scope' == '/prefix/asgi-scope'
E             - /prefix/asgi-scope
E             ? -------
E             + /asgi-scope
```
That test confirms that messing around with the `base_url` doesn't modify the ASGI scope... but the fix I'm using for this issue DOES modify the ASGI scope.

The question raised here is: should the ASGI scope stay unmodified when `base_url` is used?

I think it should. It doesn't make sense to obscure the ""real"" path just to get custom pages to work properly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813899472,
https://github.com/simonw/datasette/issues/1238#issuecomment-855272693,https://api.github.com/repos/simonw/datasette/issues/1238,855272693,MDEyOklzc3VlQ29tbWVudDg1NTI3MjY5Mw==,9599,2021-06-05T17:45:42Z,2021-06-05T17:45:42Z,OWNER,"Applying this fix worked when I manually tested it:
```diff
         base_url = self.ds.setting(""base_url"")
         if base_url != ""/"" and path.startswith(base_url):
             path = ""/"" + path[len(base_url) :]
+            scope = dict(scope, path=path, raw_path=path.encode(""utf-8""))
         request = Request(scope, receive)
```
But... the test I wrote still failed. My hunch is that this is because deep within the test framework requests go through `ds.client` which may be applying its own changes relevant to `base_url`:

https://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/utils/testing.py#L139","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813899472,
https://github.com/simonw/datasette/issues/1238#issuecomment-855270917,https://api.github.com/repos/simonw/datasette/issues/1238,855270917,MDEyOklzc3VlQ29tbWVudDg1NTI3MDkxNw==,9599,2021-06-05T17:32:29Z,2021-06-05T17:32:29Z,OWNER,"This looks like the cause: https://github.com/simonw/datasette/blob/6e9b07be92905011211d8df7a872fb7c1f2737b2/datasette/app.py#L1087-L1092

Note how `path` is modified... but then we create a new `Request()` that uses the old scope, which has unmodified `scope[""path""]` - and then the code later on looks at `request.scope[""path""]` when deciding if the request matches:

https://github.com/simonw/datasette/blob/afed51b1e36cf275c39e71c7cb262d6c5bdbaa31/datasette/app.py#L1154-L1155","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813899472,
https://github.com/simonw/datasette/issues/1356#issuecomment-853895159,https://api.github.com/repos/simonw/datasette/issues/1356,853895159,MDEyOklzc3VlQ29tbWVudDg1Mzg5NTE1OQ==,25778,2021-06-03T14:03:59Z,2021-06-03T14:03:59Z,CONTRIBUTOR,"(Putting thoughts here to keep the conversation in one place.)

I think using datasette for this use-case is the right approach. I usually have both datasette and sqlite-utils installed in the same project, and that's where I'm trying out queries, so it probably makes the most sense to have datasette also manage the output (and maybe the input, too).

It seems like both `--get` and `--query` could work better as subcommands, rather than options, if you're looking at building out a full CLI experience in datasette. It would give a cleaner separation in what you're trying to do and let each have its own dedicated options. So something like this:

```sh
# run an arbitrary query
datasette query covid.db ""select * from ny_times_us_counties limit 1"" --format yaml

# run a canned query
datasette get covid.db some-canned-query --format yaml
```

","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853567861,https://api.github.com/repos/simonw/sqlite-utils/issues/264,853567861,MDEyOklzc3VlQ29tbWVudDg1MzU2Nzg2MQ==,9599,2021-06-03T05:12:21Z,2021-06-03T05:12:21Z,OWNER,I think this is more likely to happen in Datasette than in sqlite-utils - see https://github.com/simonw/datasette/issues/1356 for thoughts on this.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907642546,
https://github.com/simonw/datasette/issues/526#issuecomment-853567413,https://api.github.com/repos/simonw/datasette/issues/526,853567413,MDEyOklzc3VlQ29tbWVudDg1MzU2NzQxMw==,9599,2021-06-03T05:11:27Z,2021-06-03T05:11:27Z,OWNER,"Another potential way to implement this would be to hold the SQLite connection open and execute the full query there.

I've avoided this in the past due to concerns of resource exhaustion - if multiple requests attempt this at the same time all of the connections in the pool will become tied up and the site will be unable to respond to further requests.

But... now that Datasette has authentication there's the possibility of making this feature only available to specific authenticated users - the `--root` user for example. Which avoids the danger while unlocking a super-useful feature.

Not to mention people who are running Datasette privately on their own laptop, or the proposed `--query` CLI feature in #1356.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459882902,
https://github.com/simonw/datasette/issues/1356#issuecomment-853566337,https://api.github.com/repos/simonw/datasette/issues/1356,853566337,MDEyOklzc3VlQ29tbWVudDg1MzU2NjMzNw==,9599,2021-06-03T05:08:32Z,2021-06-03T05:08:32Z,OWNER,Also relevant: CSV streaming for canned queries in #526 - even better if we could stream ANY SQL query.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1062#issuecomment-853566011,https://api.github.com/repos/simonw/datasette/issues/1062,853566011,MDEyOklzc3VlQ29tbWVudDg1MzU2NjAxMQ==,9599,2021-06-03T05:07:42Z,2021-06-03T05:07:42Z,OWNER,Implementing this would make #1356 a whole lot more interesting.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",732674148,
https://github.com/simonw/datasette/issues/1356#issuecomment-853565850,https://api.github.com/repos/simonw/datasette/issues/1356,853565850,MDEyOklzc3VlQ29tbWVudDg1MzU2NTg1MA==,9599,2021-06-03T05:07:21Z,2021-06-03T05:07:21Z,OWNER,"A problem with this is that if you're using `--query` you likely want ALL of the results - at the moment the only Datasette output type that can stream everything is `.csv` and plugin formats can't handle full streams, see #1062 and #1177.

So there's not much point implementing this unless we first make plugins able to add custom streaming formats.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853562891,https://api.github.com/repos/simonw/datasette/issues/1356,853562891,MDEyOklzc3VlQ29tbWVudDg1MzU2Mjg5MQ==,9599,2021-06-03T04:59:40Z,2021-06-03T04:59:40Z,OWNER,"It's weird that `--get` is documented on this page right now: https://docs.datasette.io/en/stable/getting_started.html#datasette-get

If I implement this I should build a new ""Datasette on the command-line"" documentation page to cover both `--get` and `--query`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853560870,https://api.github.com/repos/simonw/datasette/issues/1356,853560870,MDEyOklzc3VlQ29tbWVudDg1MzU2MDg3MA==,9599,2021-06-03T04:53:47Z,2021-06-03T04:53:56Z,OWNER,"This is also interesting when used in conjunction with the proposed `datasette insert` command from #1163 - Datasette would become a plugin-driven CLI tool for both ingesting and outputting data, as a side-effect of its web features.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853560389,https://api.github.com/repos/simonw/datasette/issues/1356,853560389,MDEyOklzc3VlQ29tbWVudDg1MzU2MDM4OQ==,9599,2021-06-03T04:52:13Z,2021-06-03T04:52:13Z,OWNER,I should implement #1355 for more efficient `--csv` streaming as part of this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853560237,https://api.github.com/repos/simonw/datasette/issues/1356,853560237,MDEyOklzc3VlQ29tbWVudDg1MzU2MDIzNw==,9599,2021-06-03T04:51:49Z,2021-06-03T04:51:49Z,OWNER,This feels like a relatively simple feature to implement that unlocks a whole new set of possible uses for Datasette - as described by @eyeseast in https://github.com/simonw/sqlite-utils/issues/264#issue-907642546.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853559915,https://api.github.com/repos/simonw/datasette/issues/1356,853559915,MDEyOklzc3VlQ29tbWVudDg1MzU1OTkxNQ==,9599,2021-06-03T04:50:52Z,2021-06-03T04:50:52Z,OWNER,"What happens if you pass multiple databases? The `--query` would be executed against the first one. And if you pass `--crossdb` it would be executed against the `/_memory` database and would support cross-database joins.

Key thing here is that output plugins are supported (also plugins that add new SQL functions), making many Datasette plugins usable from the command-line.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853558741,https://api.github.com/repos/simonw/sqlite-utils/issues/264,853558741,MDEyOklzc3VlQ29tbWVudDg1MzU1ODc0MQ==,9599,2021-06-03T04:47:19Z,2021-06-03T04:47:19Z,OWNER,"This inspired me to re-examine how `--get` works, hence this issue: https://github.com/simonw/datasette/issues/1355","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907642546,
https://github.com/simonw/datasette/issues/1355#issuecomment-853557439,https://api.github.com/repos/simonw/datasette/issues/1355,853557439,MDEyOklzc3VlQ29tbWVudDg1MzU1NzQzOQ==,9599,2021-06-03T04:43:14Z,2021-06-03T04:43:14Z,OWNER,"It's using `TestClient` at the moment which is a wrapper around `httpx` (as of ) that uses the `@async_to_sync` decorator to hide the async nature.

https://github.com/simonw/datasette/blob/f78ebdc04537a6102316d6dbbf6c887565806078/datasette/utils/testing.py#L102-L156

Maybe the fix here is to switch the `--get` implementation to using `httpx` directly with https://www.python-httpx.org/async/#streaming-responses","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910088936,
https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853553754,https://api.github.com/repos/simonw/sqlite-utils/issues/264,853553754,MDEyOklzc3VlQ29tbWVudDg1MzU1Mzc1NA==,9599,2021-06-03T04:32:42Z,2021-06-03T04:36:36Z,OWNER,"This is a really interesting thought. I've so far resisted the temptation to add plugins to `sqlite-utils`, partly to avoid overlap with Datasette - but I'm open to discussing it.

There's actually a way for you to do what you're describing using `datasette` on the command-line, though it's a little obscure - also Datasette doesn't yet have a GeoJSON output extension, though it really should have one.

Here's an example using [datasette-yaml](https://datasette.io/plugins/datasette-yaml):

```
datasette /tmp/covid.db --get='/covid/ny_times_us_counties.yaml'
- rowid: 1
  date: '2020-01-21'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0
- rowid: 2
  date: '2020-01-22'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0
```
It even works with arbitrary SQL queries, though you might have to apply URL encoding to the `--get` string (this seems to work though):
```
datasette /tmp/covid.db --get='/covid.yaml?sql=select * from ny_times_us_counties limit 2' 
- date: '2020-01-21'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0
- date: '2020-01-22'
  county: Snohomish
  state: Washington
  fips: 53061
  cases: 1
  deaths: 0
```
Here's the documentation for `--get`: https://docs.datasette.io/en/latest/getting_started.html#datasette-get","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907642546,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853554550,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853554550,MDEyOklzc3VlQ29tbWVudDg1MzU1NDU1MA==,9599,2021-06-03T04:34:38Z,2021-06-03T04:34:38Z,OWNER,Documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#listing-indexes,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853548442,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853548442,MDEyOklzc3VlQ29tbWVudDg1MzU0ODQ0Mg==,9599,2021-06-03T04:16:49Z,2021-06-03T04:16:49Z,OWNER,Needs to show the table each index applies to.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853547681,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853547681,MDEyOklzc3VlQ29tbWVudDg1MzU0NzY4MQ==,9599,2021-06-03T04:14:28Z,2021-06-03T04:14:28Z,OWNER,"This looks good:
```
% sqlite-utils indexes /tmp/covid.db -t      
index_name                                                seqno    cid  name                 desc  coll      key
------------------------------------------------------  -------  -----  -----------------  ------  ------  -----
idx_johns_hopkins_csse_daily_reports_combined_key             0     12  combined_key            0  BINARY      1
idx_johns_hopkins_csse_daily_reports_country_or_region        0      1  country_or_region       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_province_or_state        0      2  province_or_state       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_day                      0      0  day                     0  BINARY      1
idx_ny_times_us_counties_date                                 0      0  date                    1  BINARY      1
idx_ny_times_us_counties_fips                                 0      3  fips                    0  BINARY      1
idx_ny_times_us_counties_county                               0      1  county                  0  BINARY      1
idx_ny_times_us_counties_state                                0      2  state                   0  BINARY      1

% sqlite-utils indexes /tmp/covid.db -t --aux
index_name                                                seqno    cid  name                 desc  coll      key
------------------------------------------------------  -------  -----  -----------------  ------  ------  -----
idx_johns_hopkins_csse_daily_reports_combined_key             0     12  combined_key            0  BINARY      1
idx_johns_hopkins_csse_daily_reports_combined_key             1     -1                          0  BINARY      0
idx_johns_hopkins_csse_daily_reports_country_or_region        0      1  country_or_region       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_country_or_region        1     -1                          0  BINARY      0
idx_johns_hopkins_csse_daily_reports_province_or_state        0      2  province_or_state       0  BINARY      1
idx_johns_hopkins_csse_daily_reports_province_or_state        1     -1                          0  BINARY      0
idx_johns_hopkins_csse_daily_reports_day                      0      0  day                     0  BINARY      1
idx_johns_hopkins_csse_daily_reports_day                      1     -1                          0  BINARY      0
idx_ny_times_us_counties_date                                 0      0  date                    1  BINARY      1
idx_ny_times_us_counties_date                                 1     -1                          0  BINARY      0
idx_ny_times_us_counties_fips                                 0      3  fips                    0  BINARY      1
idx_ny_times_us_counties_fips                                 1     -1                          0  BINARY      0
idx_ny_times_us_counties_county                               0      1  county                  0  BINARY      1
idx_ny_times_us_counties_county                               1     -1                          0  BINARY      0
idx_ny_times_us_counties_state                                0      2  state                   0  BINARY      1
idx_ny_times_us_counties_state                                1     -1                          0  BINARY      0
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853546818,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853546818,MDEyOklzc3VlQ29tbWVudDg1MzU0NjgxOA==,9599,2021-06-03T04:11:46Z,2021-06-03T04:11:46Z,OWNER,"By default I won't return auxiliary columns, but I'll offer a `--aux` option to return them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853545743,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853545743,MDEyOklzc3VlQ29tbWVudDg1MzU0NTc0Mw==,9599,2021-06-03T04:08:04Z,2021-06-03T04:08:04Z,OWNER,"Figuring out the right queries:

https://covid-19.datasettes.com/covid?sql=select+sqlite_master.name%2C+i.*+from+sqlite_master%0D%0Ajoin+pragma_index_list%28sqlite_master.name%29+i%0D%0Awhere+type+%3D+%27table%27

This query shows all columns across all indexes across all tables:

```sql
select i.name as index_name, xinfo.* from sqlite_master
join pragma_index_list(sqlite_master.name) i
join pragma_index_xinfo(index_name) xinfo
where sqlite_master.type = 'table'
```
https://covid-19.datasettes.com/covid?sql=select+i.name+as+index_name%2C+xinfo.*+from+sqlite_master%0D%0Ajoin+pragma_index_list%28sqlite_master.name%29+i%0D%0Ajoin+pragma_index_xinfo%28index_name%29+xinfo%0D%0Awhere+sqlite_master.type+%3D+%27table%27","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853544493,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853544493,MDEyOklzc3VlQ29tbWVudDg1MzU0NDQ5Mw==,9599,2021-06-03T04:03:59Z,2021-06-03T04:03:59Z,OWNER,"Here's how `sqlite-utils triggers` works: https://github.com/simonw/sqlite-utils/blob/9c67cb925253cd5ef54a1fe0496e0ff9caeacfd6/sqlite_utils/cli.py#L1266-L1277

Running it from a SQL query makes it easy to support modifiers like `--csv` and `-t`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853541869,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853541869,MDEyOklzc3VlQ29tbWVudDg1MzU0MTg2OQ==,9599,2021-06-03T03:54:14Z,2021-06-03T03:54:14Z,OWNER,Documentation: https://sqlite-utils.datasette.io/en/latest/python-api.html#xindexes,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853535559,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853535559,MDEyOklzc3VlQ29tbWVudDg1MzUzNTU1OQ==,9599,2021-06-03T03:32:47Z,2021-06-03T03:32:47Z,OWNER,"New design:
```python
def test_xindexes(fresh_db):
    fresh_db.executescript(
        """"""
        create table Gosh (c1 text, c2 text, c3 text);
        create index Gosh_c1 on Gosh(c1);
        create index Gosh_c2c3 on Gosh(c2, c3 desc);
    """"""
    )
    assert fresh_db[""Gosh""].xindexes == [
        XIndex(
            name=""Gosh_c2c3"",
            columns=[
                XIndexColumn(seqno=0, cid=1, name=""c2"", desc=0, coll=""BINARY"", key=1),
                XIndexColumn(seqno=1, cid=2, name=""c3"", desc=1, coll=""BINARY"", key=1),
                XIndexColumn(seqno=2, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
            ],
        ),
        XIndex(
            name=""Gosh_c1"",
            columns=[
                XIndexColumn(seqno=0, cid=0, name=""c1"", desc=0, coll=""BINARY"", key=1),
                XIndexColumn(seqno=1, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
            ],
        ),
    ]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853534732,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853534732,MDEyOklzc3VlQ29tbWVudDg1MzUzNDczMg==,9599,2021-06-03T03:30:10Z,2021-06-03T03:30:10Z,OWNER,"I'm going to return `XIndex(name, columns)`  - where `columns` is a list of `XIndexColumn`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853530348,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853530348,MDEyOklzc3VlQ29tbWVudDg1MzUzMDM0OA==,9599,2021-06-03T03:16:33Z,2021-06-03T03:16:33Z,OWNER,"In prototyping this out I realize that I actually want to get back the name of each index, then for each of them the detailed list of index columns. Here's the test from my initial prototype:
```python
def test_xindexes(fresh_db):
    fresh_db.executescript(
        """"""
        create table Gosh (c1 text, c2 text, c3 text);
        create index Gosh_c1 on Gosh(c1);
        create index Gosh_c2c3 on Gosh(c2, c3 desc);
    """"""
    )
    assert fresh_db[""Gosh""].xindexes == [
        (
            ""Gosh_c2c3"",
            [
                XIndex(seqno=0, cid=1, name=""c2"", desc=0, coll=""BINARY"", key=1),
                XIndex(seqno=1, cid=2, name=""c3"", desc=1, coll=""BINARY"", key=1),
                XIndex(seqno=2, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
            ],
        ),
        (
            ""Gosh_c1"",
            [
                XIndex(seqno=0, cid=0, name=""c1"", desc=0, coll=""BINARY"", key=1),
                XIndex(seqno=1, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
            ],
        ),
    ]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853525036,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853525036,MDEyOklzc3VlQ29tbWVudDg1MzUyNTAzNg==,9599,2021-06-03T03:02:22Z,2021-06-03T03:02:22Z,OWNER,"This would be a breaking change - and the fact that it returns auxiliary columns isn't particularly useful for most cases - ""Auxiliary columns are additional columns needed to locate the table entry that corresponds to each index entry"".

Instead, I'm going to add a new property `table.xindexes` which exposes this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/datasette/issues/893#issuecomment-852712469,https://api.github.com/repos/simonw/datasette/issues/893,852712469,MDEyOklzc3VlQ29tbWVudDg1MjcxMjQ2OQ==,9599,2021-06-02T04:29:56Z,2021-06-02T04:29:56Z,OWNER,"I think I fixed this a while ago, though I can't find the commit now. Please re-open if the bug still happens!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",656959584,
https://github.com/simonw/datasette/issues/1127#issuecomment-852712106,https://api.github.com/repos/simonw/datasette/issues/1127,852712106,MDEyOklzc3VlQ29tbWVudDg1MjcxMjEwNg==,9599,2021-06-02T04:28:55Z,2021-06-02T04:28:55Z,OWNER,This became resizable in #1236.,"{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",756818250,
https://github.com/simonw/datasette/pull/1306#issuecomment-852710117,https://api.github.com/repos/simonw/datasette/issues/1306,852710117,MDEyOklzc3VlQ29tbWVudDg1MjcxMDExNw==,9599,2021-06-02T04:24:33Z,2021-06-02T04:24:33Z,OWNER,I have a test - I'll merge this and then add that test.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",864979486,
https://github.com/simonw/datasette/pull/1306#issuecomment-852708574,https://api.github.com/repos/simonw/datasette/issues/1306,852708574,MDEyOklzc3VlQ29tbWVudDg1MjcwODU3NA==,9599,2021-06-02T04:20:18Z,2021-06-02T04:20:18Z,OWNER,https://github.com/simonw/datasette/blob/115332ce76c0e867d9936406aaf4bcee6b1ef3cb/datasette/views/index.py#L77 helps here - looks like we can pass `?_sort=relationships` to trigger the bug.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",864979486,
https://github.com/simonw/datasette/pull/1306#issuecomment-852707676,https://api.github.com/repos/simonw/datasette/issues/1306,852707676,MDEyOklzc3VlQ29tbWVudDg1MjcwNzY3Ng==,9599,2021-06-02T04:17:41Z,2021-06-02T04:17:41Z,OWNER,We need to add a test that illustrates the bug in #1305 so we can prove that this definitely fixes it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",864979486,