# issues

node_id: I_kwDOBm6k_c5Cb-CO
title: Improvements to help make Datasette a better tool for learning SQL
state: open
created_at: 2022-01-26T04:56:07Z
updated_at: 2022-01-26T16:41:46Z
author_association: OWNER
body: Tracking issue for the general goal of making Datasette a better tool for learning SQL.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1613/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5CcApS
title: `sqlite-utils bulk` progress bar
state: closed
created_at: 2022-01-26T05:14:49Z
updated_at: 2022-01-26T05:17:20Z
closed_at: 2022-01-26T05:16:51Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It can easily have a progress bar because it works by looping through an iterator: https://github.com/simonw/sqlite-utils/blob/a9fca7efa4184fbb2a65ca1275c326950ed9d3c1/sqlite_utils/cli.py#L1014-L1018

  Should also support the `--silent` option if I add this.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/391/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5CcA7l
title: `sqlite-utils bulk --batch-size` option
state: closed
created_at: 2022-01-26T05:17:11Z
updated_at: 2022-01-26T18:17:59Z
closed_at: 2022-01-26T18:17:59Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Could add support for `--batch-size` as seen in `insert`/`upsert` too - causing it to break the list up into batches and commit for each one.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/391#issuecomment-1021876055_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/392/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CfDIQ
title: Try again with SQLite codemirror support
state: open
created_at: 2022-01-26T20:05:20Z
updated_at: 2022-12-23T21:27:10Z
author_association: OWNER
type: issue
body:
  I tried and failed to implement autocomplete a while ago. Relevant code:

  https://github.com/codemirror/legacy-modes/blob/8f36abca5f55024258cd23d9cfb0203d8d244f0d/mode/sql.js#L335

  Sounds like upgrading to CodeMirror 6 ASAP would be worthwhile since it has better accessibility and touch screen support: https://codemirror.net/6/

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1614/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5ClhfF
title: Potential simplified publishing mechanism
state: closed
created_at: 2022-01-28T08:34:50Z
updated_at: 2022-02-02T07:34:21Z
closed_at: 2022-02-02T07:34:17Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi,

  Forewarning: this idea is one I've only been thinking about for a while and it's not fully fleshed-out yet.

  I love Datasette and what it stands for. I was thinking about how we could make it accessible to more people, especially those without access to credit cards required for a lot of hosting options. Or they might not feel comfortable signing up for said services.

  So I was thinking I might create a service that hosts Datasette instances for folks. I'd probably stick it on AWS Lambda and limit requests to something like n/month to avoid bankrupting myself. If I did build such a hypothetical service, I was thinking I would rely on GitHub Actions to do the heavy lifting. 

  E.g. user `johndoe` creates a repo `my-animals` with a couple of files: `dogs.csv`, `cats.csv` and the following GitHub Actions workflow:

  ```yaml
  # .github/workflows/push.yml
  on:
    push

  # this allows the publish action to use OIDC to authenticate johndoe/my-animals
  permissions:
    id-token: write
    contents: read

  jobs:
    publish:
      runs-on: ubuntu-latest
      steps:
        - uses: actions/setup-python@v2

        - run: pip install sqlite-utils

        - uses: actions/checkout@v2

        - run: |
            set -eux
            
            sqlite-utils create-database animals.db          
            sqlite-utils insert animals.db dogs dogs.csv --csv          
            sqlite-utils insert animals.db cats cats.csv --csv

        - uses: datasette-hub/publish@v1
          with:
            db: animals.db
            metadata: meta.yml

          # this step is helpful for debugging why the
          # generated sqlite db was rejected
        - uses: actions/upload-artifact@v2
          if: failure()
          with:
            path: animals.db
            retention-days: 1
  ```

  This would then cause a Datasette instance to be available at `https://johndoe-my-animals.datasette-hub.test/`. It feels like this could significantly reduce the friction to someone being able to go from data set to Datasette.

  What do you think? Does this address a real need? Or am I perhaps misunderstanding the main friction points? As a bonus: it feels like this would pair well with [git scraping](https://simonwillison.net/2020/Oct/9/git-scraping/).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1615/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5CrEJJ
title: Better documentation for insert-replace
state: closed
created_at: 2022-01-30T15:40:23Z
updated_at: 2022-02-03T22:13:24Z
closed_at: 2022-02-03T22:13:24Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Currently: https://sqlite-utils.datasette.io/en/stable/python-api.html#insert-replacing-data

  > If you want to insert a record or replace an existing record with the same primary key, using the replace=True argument to .insert() or .insert_all():

  Should describe the exception you get first, then how to use replace to avoid it.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/393/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4x1kCu
title: Bump black from 21.12b0 to 22.1.0
state: closed
created_at: 2022-01-31T13:13:46Z
updated_at: 2022-02-02T22:23:52Z
closed_at: 2022-02-02T22:23:51Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1616
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 21.12b0 to 22.1.0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>22.1.0</h2>
  <p>At long last, Black is no longer a beta product! This is the first non-beta release and the first release covered by our new stability policy.</p>
  <h3>Highlights</h3>
  <ul>
  <li>Remove Python 2 support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2740">#2740</a>)</li>
  <li>Introduce the <code>--preview</code> flag (<a href="https://github-redirect.dependabot.com/psf/black/issues/2752">#2752</a>)</li>
  </ul>
  <h3>Style</h3>
  <ul>
  <li>Deprecate <code>--experimental-string-processing</code> and move the functionality under <code>--preview</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2789">#2789</a>)</li>
  <li>For stubs, one blank line between class attributes and methods is now kept if there's at least one pre-existing blank line (<a href="https://github-redirect.dependabot.com/psf/black/issues/2736">#2736</a>)</li>
  <li>Black now normalizes string prefix order (<a href="https://github-redirect.dependabot.com/psf/black/issues/2297">#2297</a>)</li>
  <li>Remove spaces around power operators if both operands are simple (<a href="https://github-redirect.dependabot.com/psf/black/issues/2726">#2726</a>)</li>
  <li>Work around bug that causes unstable formatting in some cases in the presence of the magic trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2807">#2807</a>)</li>
  <li>Use parentheses for attribute access on decimal float and int literals (<a href="https://github-redirect.dependabot.com/psf/black/issues/2799">#2799</a>)</li>
  <li>Don't add whitespace for attribute access on hexadecimal, binary, octal, and complex literals (<a href="https://github-redirect.dependabot.com/psf/black/issues/2799">#2799</a>)</li>
  <li>Treat blank lines in stubs the same inside top-level if statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2820">#2820</a>)</li>
  <li>Fix unstable formatting with semicolons and arithmetic expressions (<a href="https://github-redirect.dependabot.com/psf/black/issues/2817">#2817</a>)</li>
  <li>Fix unstable formatting around magic trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2572">#2572</a>)</li>
  </ul>
  <h3>Parser</h3>
  <ul>
  <li>Fix mapping cases that contain as-expressions, like <code>case {&quot;key&quot;: 1 | 2 as password}</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2686">#2686</a>)</li>
  <li>Fix cases that contain multiple top-level as-expressions, like <code>case 1 as a, 2 as b</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2716">#2716</a>)</li>
  <li>Fix call patterns that contain as-expressions with keyword arguments, like <code>case Foo(bar=baz as quux)</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2749">#2749</a>)</li>
  <li>Tuple unpacking on <code>return</code> and <code>yield</code> constructs now implies 3.8+ (<a href="https://github-redirect.dependabot.com/psf/black/issues/2700">#2700</a>)</li>
  <li>Unparenthesized tuples on annotated assignments (e.g <code>values: Tuple[int, ...] = 1, 2, 3</code>) now implies 3.8+ (<a href="https://github-redirect.dependabot.com/psf/black/issues/2708">#2708</a>)</li>
  <li>Fix handling of standalone <code>match()</code> or <code>case()</code> when there is a trailing newline or a comment inside of the parentheses. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2760">#2760</a>)</li>
  <li><code>from __future__ import annotations</code> statement now implies Python 3.7+ (<a href="https://github-redirect.dependabot.com/psf/black/issues/2690">#2690</a>)</li>
  </ul>
  <h3>Performance</h3>
  <ul>
  <li>Speed-up the new backtracking parser about 4X in general (enabled when <code>--target-version</code> is set to 3.10 and higher). (<a href="https://github-redirect.dependabot.com/psf/black/issues/2728">#2728</a>)</li>
  <li>Black is now compiled with mypyc for an overall 2x speed-up. 64-bit Windows, MacOS, and Linux (not including musl) are supported. (<a href="https://github-redirect.dependabot.com/psf/black/issues/1009">#1009</a>, <a href="https://github-redirect.dependabot.com/psf/black/issues/2431">#2431</a>)</li>
  </ul>
  <h3>Configuration</h3>
  <ul>
  <li>Do not accept bare carriage return line endings in pyproject.toml (<a href="https://github-redirect.dependabot.com/psf/black/issues/2408">#2408</a>)</li>
  <li>Add configuration option (<code>python-cell-magics</code>) to format cells with custom magics in Jupyter Notebooks (<a href="https://github-redirect.dependabot.com/psf/black/issues/2744">#2744</a>)</li>
  <li>Allow setting custom cache directory on all platforms with environment variable <code>BLACK_CACHE_DIR</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2739">#2739</a>).</li>
  <li>Enable Python 3.10+ by default, without any extra need to specify -<code>-target-version=py310</code>. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2758">#2758</a>)</li>
  <li>Make passing <code>SRC</code> or <code>--code</code> mandatory and mutually exclusive (<a href="https://github-redirect.dependabot.com/psf/black/issues/2804">#2804</a>)</li>
  </ul>
  <h3>Output</h3>
  <ul>
  <li>Improve error message for invalid regular expression (<a href="https://github-redirect.dependabot.com/psf/black/issues/2678">#2678</a>)</li>
  <li>Improve error message when parsing fails during AST safety check by embedding the underlying SyntaxError (<a href="https://github-redirect.dependabot.com/psf/black/issues/2693">#2693</a>)</li>
  <li>No longer color diff headers white as it's unreadable in light themed terminals (<a href="https://github-redirect.dependabot.com/psf/black/issues/2691">#2691</a>)</li>
  <li>Text coloring added in the final statistics (<a href="https://github-redirect.dependabot.com/psf/black/issues/2712">#2712</a>)</li>
  <li>Verbose mode also now describes how a project root was discovered and which paths will be formatted. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2526">#2526</a>)</li>
  </ul>
  <h3>Packaging</h3>
  <ul>
  <li>All upper version bounds on dependencies have been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2718">#2718</a>)</li>
  <li><code>typing-extensions</code> is no longer a required dependency in Python 3.10+ (<a href="https://github-redirect.dependabot.com/psf/black/issues/2772">#2772</a>)</li>
  <li>Set <code>click</code> lower bound to <code>8.0.0</code> as <em>Black</em> crashes on <code>7.1.2</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2791">#2791</a>)</li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>22.1.0</h2>
  <p>At long last, <em>Black</em> is no longer a beta product! This is the first non-beta release
  and the first release covered by our new stability policy.</p>
  <h3>Highlights</h3>
  <ul>
  <li><strong>Remove Python 2 support</strong> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2740">#2740</a>)</li>
  <li>Introduce the <code>--preview</code> flag (<a href="https://github-redirect.dependabot.com/psf/black/issues/2752">#2752</a>)</li>
  </ul>
  <h3>Style</h3>
  <ul>
  <li>Deprecate <code>--experimental-string-processing</code> and move the functionality under
  <code>--preview</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2789">#2789</a>)</li>
  <li>For stubs, one blank line between class attributes and methods is now kept if there's
  at least one pre-existing blank line (<a href="https://github-redirect.dependabot.com/psf/black/issues/2736">#2736</a>)</li>
  <li>Black now normalizes string prefix order (<a href="https://github-redirect.dependabot.com/psf/black/issues/2297">#2297</a>)</li>
  <li>Remove spaces around power operators if both operands are simple (<a href="https://github-redirect.dependabot.com/psf/black/issues/2726">#2726</a>)</li>
  <li>Work around bug that causes unstable formatting in some cases in the presence of the
  magic trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2807">#2807</a>)</li>
  <li>Use parentheses for attribute access on decimal float and int literals (<a href="https://github-redirect.dependabot.com/psf/black/issues/2799">#2799</a>)</li>
  <li>Don't add whitespace for attribute access on hexadecimal, binary, octal, and complex
  literals (<a href="https://github-redirect.dependabot.com/psf/black/issues/2799">#2799</a>)</li>
  <li>Treat blank lines in stubs the same inside top-level <code>if</code> statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2820">#2820</a>)</li>
  <li>Fix unstable formatting with semicolons and arithmetic expressions (<a href="https://github-redirect.dependabot.com/psf/black/issues/2817">#2817</a>)</li>
  <li>Fix unstable formatting around magic trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2572">#2572</a>)</li>
  </ul>
  <h3>Parser</h3>
  <ul>
  <li>Fix mapping cases that contain as-expressions, like <code>case {&quot;key&quot;: 1 | 2 as password}</code>
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/2686">#2686</a>)</li>
  <li>Fix cases that contain multiple top-level as-expressions, like <code>case 1 as a, 2 as b</code>
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/2716">#2716</a>)</li>
  <li>Fix call patterns that contain as-expressions with keyword arguments, like
  <code>case Foo(bar=baz as quux)</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2749">#2749</a>)</li>
  <li>Tuple unpacking on <code>return</code> and <code>yield</code> constructs now implies 3.8+ (<a href="https://github-redirect.dependabot.com/psf/black/issues/2700">#2700</a>)</li>
  <li>Unparenthesized tuples on annotated assignments (e.g
  <code>values: Tuple[int, ...] = 1, 2, 3</code>) now implies 3.8+ (<a href="https://github-redirect.dependabot.com/psf/black/issues/2708">#2708</a>)</li>
  <li>Fix handling of standalone <code>match()</code> or <code>case()</code> when there is a trailing newline or a
  comment inside of the parentheses. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2760">#2760</a>)</li>
  <li><code>from __future__ import annotations</code> statement now implies Python 3.7+ (<a href="https://github-redirect.dependabot.com/psf/black/issues/2690">#2690</a>)</li>
  </ul>
  <h3>Performance</h3>
  <ul>
  <li>Speed-up the new backtracking parser about 4X in general (enabled when
  <code>--target-version</code> is set to 3.10 and higher). (<a href="https://github-redirect.dependabot.com/psf/black/issues/2728">#2728</a>)</li>
  <li><em>Black</em> is now compiled with <a href="https://github.com/mypyc/mypyc">mypyc</a> for an overall 2x
  speed-up. 64-bit Windows, MacOS, and Linux (not including musl) are supported. (<a href="https://github-redirect.dependabot.com/psf/black/issues/1009">#1009</a>,
  <a href="https://github-redirect.dependabot.com/psf/black/issues/2431">#2431</a>)</li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/psf/black/commits/22.1.0">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.12b0&new-version=22.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1616/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4x6zZ5
title: Ensure template_path always uses "/" to match jinja
state: closed
created_at: 2022-02-01T17:20:30Z
updated_at: 2022-02-03T01:58:35Z
closed_at: 2022-02-03T01:58:35Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1617
type: pull
body:
  This PR shoudl fix #1545 

  The existing code substituted / for \, assuming this was the right behaviour for windows. But on Windows, Jinja still uses / for the template list - See https://github.com/pallets/jinja/blob/896a62135bcc151f2997e028c5125bec2cb2431f/src/jinja2/loaders.py#L225

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1617/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C0vQZ
title: Reconsider policy on blocking queries containing the string "pragma"
state: open
created_at: 2022-02-01T19:39:46Z
updated_at: 2022-02-02T19:42:03Z
author_association: NONE
type: issue
body:
  First of all, thanks for creating this cool project, and also supporting publishing to various hosting services out of the box.

  While testing out, I noticed legitimate queries such as 
  ```
  select * from books where title like 'Pragmatic%'
  ```
  or
  ```
  select * from books where title = 'The Pragmatic Programmer'
  ```
  are blocked, due to the regular expression check here:
  https://github.com/simonw/datasette/blob/main/datasette/utils/__init__.py#L185

  Example as seen from a Datasette instance:
  https://fivethirtyeight.datasettes.com/polls?sql=select+*+from+books+where+title+like+%27Pragmatic%25%27%0D%0A

  I'd propose a regular expression like 
  ```
  re.compile(f"pragma_(?!({'|'.join(allowed_pragmas)}))"),
  ```
  instead of
  ```
  re.compile(f"pragma(?!_({'|'.join(allowed_pragmas)}))"),
  ```

  I can create a pull request with this change, unless the maintainers think it would allow unwanted queries to be executed.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1618/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C2gE2
title: JSON link on row page is 404 if base_url setting is used
state: open
created_at: 2022-02-02T07:09:53Z
updated_at: 2023-03-24T15:38:04Z
author_association: OWNER
type: issue
body:
  On my local environment:

      datasette fixtures.db -p 3344 --setting base_url /foo/bar/

  Then hit http://127.0.0.1:3344/foo/bar/fixtures/table%2Fwith%2Fslashes.csv/3

  <img width="434" alt="image" src="https://user-images.githubusercontent.com/9599/152109515-64e2a0f8-989c-4e90-9755-65c60da677ad.png">

  But... that `json` link goes here, which is a 404:

  http://127.0.0.1:3344/foo/bar/foo/bar/fixtures/table%2Fwith%2Fslashes.csv/3?_format=json

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1619/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C2oh5
title: Link: rel="alternate" to JSON for queries too
state: closed
created_at: 2022-02-02T08:02:42Z
updated_at: 2022-02-02T21:53:02Z
closed_at: 2022-02-02T21:33:00Z
author_association: OWNER
body:
  Following:
  - #1533

  I implemented it for tables and rows but I should have done queries as well.

type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1620/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C5qyX
title: Test against Python 3.11 dev version
state: closed
created_at: 2022-02-02T21:38:57Z
updated_at: 2022-03-19T04:04:49Z
closed_at: 2022-02-02T21:58:54Z
author_association: OWNER
type: issue
state_reason: completed
body:
  To avoid another surprise like we got with 3.10: https://simonwillison.net/2021/Oct/9/finding-and-reporting-a-bug/

  From a quick GitHub code search it looks like `3.11-dev` should work: https://cs.github.com/urllib3/urllib3/blob/7bec77e81aa0a194c98381053225813f5347c9d2/.github/workflows/ci.yml#L60

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1621/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4x_evE
title: Test against Python 3.11-dev
state: closed
created_at: 2022-02-02T21:39:38Z
updated_at: 2022-02-02T21:58:53Z
closed_at: 2022-02-02T21:58:53Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1622
body: Refs #1621
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1622/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C5rkX
title: /-/patterns returns link: alternate JSON header to 404
state: closed
created_at: 2022-02-02T21:42:49Z
updated_at: 2022-03-19T04:04:49Z
closed_at: 2022-02-02T21:48:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Bug from:
  - #1620

  ```
  % curl -s -I 'https://latest.datasette.io/-/patterns' | grep link
  link: https://latest.datasette.io/-/patterns.json; rel="alternate"; type="application/json+datasette"
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1623/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C5uG5
title: Index page `/` has no CORS headers
state: open
created_at: 2022-02-02T21:56:10Z
updated_at: 2022-09-28T16:54:22Z
author_association: OWNER
type: issue
body:
  Compare the following:
  ```
  % curl -I 'https://latest.datasette.io/fixtures'
  HTTP/1.1 200 OK
  link: https://latest.datasette.io/fixtures.json; rel="alternate"; type="application/json+datasette"
  cache-control: max-age=5
  referrer-policy: no-referrer
  access-control-allow-origin: *
  access-control-allow-headers: Authorization
  access-control-expose-headers: Link
  content-type: text/html; charset=utf-8
  x-databases: _memory, _internal, fixtures, extra_database
  Date: Wed, 02 Feb 2022 21:55:49 GMT
  Server: Google Frontend
  Transfer-Encoding: chunked

  % curl -I 'https://latest.datasette.io/'        
  HTTP/1.1 200 OK
  link: https://latest.datasette.io/.json; rel="alternate"; type="application/json+datasette"
  content-type: text/html; charset=utf-8
  x-databases: _memory, _internal, fixtures, extra_database
  Date: Wed, 02 Feb 2022 21:55:52 GMT
  Server: Google Frontend
  Transfer-Encoding: chunked
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1624/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5C5y1l
title: Test against Python 3.11-dev
state: open
created_at: 2022-02-02T22:21:03Z
updated_at: 2022-02-03T21:06:35Z
author_association: OWNER
body:
  Same as:
  - https://github.com/simonw/datasette/issues/1621

type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/394/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C5zwU
title: Try running tests against macOS and Windows in addition to Ubuntu
state: open
created_at: 2022-02-02T22:25:57Z
updated_at: 2022-02-02T22:25:57Z
author_association: OWNER
type: issue
body:
  I already do this for `sqlite-utils`: https://github.com/simonw/sqlite-utils/blob/3.22.1/.github/workflows/test.yml

  Related:
  - #1617
  - #1545

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1625/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4x_mXy
title: Try test suite against macOS and Windows
state: open
created_at: 2022-02-02T22:26:51Z
updated_at: 2022-02-03T01:22:44Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1626
body: Refs #1625
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1626/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5C6NxS
title: Get the tests passing against Windows
state: open
created_at: 2022-02-03T01:23:06Z
updated_at: 2022-02-03T01:23:32Z
author_association: OWNER
type: issue
body:
  > OK, the tests do NOT pass against Windows! https://github.com/simonw/datasette/runs/5044105941
  >
  > <img width="767" alt="image" src="https://user-images.githubusercontent.com/9599/152265501-5b5af51b-0fae-4aa7-b86d-7b61aba24b3c.png">

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1626#issuecomment-1028515161_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1627/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODFE5qs5C9aEl
title: sqlite3.OperationalError: no such table: main.my_activity
state: open
created_at: 2022-02-03T17:59:29Z
updated_at: 2022-03-20T02:38:07Z
author_association: NONE
type: issue
body:
  Hello,
  When i run the command `google-takeout-to-sqlite my-activity db.db takeout-20220203T174446Z-001.zip`, i get this error :
  ```
  Traceback (most recent call last):
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\runpy.py", line 197, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "C:\Users\julie\AppData\Local\Programs\Python\Python39-32\Scripts\google-takeout-to-sqlite.exe\__main__.py", line 7, in <module>
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\click\core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\google_takeout_to_sqlite\cli.py", line 31, in my_activity
      utils.save_my_activity(db, zf)
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\google_takeout_to_sqlite\utils.py", line 19, in save_my_activity
      db["my_activity"].create_index(["time"])
    File "c:\users\julie\appdata\local\programs\python\python39-32\lib\site-packages\sqlite_utils\db.py", line 629, in create_index
      self.db.conn.execute(sql)
  sqlite3.OperationalError: no such table: main.my_activity
  ```
  Thank you for your help
  Sorry for my bad English
  EDIT: i used the json format

reactions:
  {"url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5C_JQ-
title: "apt-get: command not found" error on macOS
state: closed
created_at: 2022-02-04T06:03:42Z
updated_at: 2022-02-04T06:10:58Z
closed_at: 2022-02-04T06:10:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Yeah, `apt-get` isn't a thing on macOS so 4a2a3e2fd0d5534f446b3f1fee34cb165e4d86d2 (to test #79 against real SpatiaLite) broke.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/395/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5C_J2q
title: mypy failure, sqlite_utils/utils.py:56
state: closed
created_at: 2022-02-04T06:08:09Z
updated_at: 2022-02-04T06:10:33Z
closed_at: 2022-02-04T06:10:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/simonw/sqlite-utils/runs/5062725880?check_suite_focus=true

  > `sqlite_utils/utils.py:56: error: Incompatible return value type (got "None", expected "str")`

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/396/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5C_Wmv
title: Support IF NOT EXISTS for table creation
state: closed
created_at: 2022-02-04T07:41:15Z
updated_at: 2022-02-06T01:30:46Z
closed_at: 2022-02-06T01:29:01Z
author_association: NONE
type: issue
state_reason: completed
body:
  Currently, I have a bunch of code that looks like this:

  ```python
  subjects = db["subjects"] if db["subjects"].exists() else db["subjects"].create({
      ...
  })
  ```
  It would be neat if sqlite-utils could simplify that by supporting `CREATE TABLE IF NOT EXISTS`, so that I'd be able to write, e.g.

  ```python
  subjects = db["subjects"].create({...}, if_not_exists=True)
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/397/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4yFTCp
title: Update pytest requirement from <6.3.0,>=5.2.2 to >=5.2.2,<7.1.0
state: closed
created_at: 2022-02-04T13:14:10Z
updated_at: 2022-03-06T01:30:06Z
closed_at: 2022-03-06T01:30:06Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1629
type: pull
body:
  Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest/releases">pytest's releases</a>.</em></p>
  <blockquote>
  <h2>7.0.0</h2>
  <h1>pytest 7.0.0 (2022-02-03)</h1>
  <p>(<strong>Please see the full set of changes for this release also in the 7.0.0rc1 notes below</strong>)</p>
  <h2>Deprecations</h2>
  <ul>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9488">#9488</a>: If custom subclasses of nodes like <code>pytest.Item</code>{.interpreted-text role=&quot;class&quot;} override the
  <code>__init__</code> method, they should take <code>**kwargs</code>. See
  <code>uncooperative-constructors-deprecated</code>{.interpreted-text role=&quot;ref&quot;} for details.</p>
  <p>Note that a deprection warning is only emitted when there is a conflict in the
  arguments pytest expected to pass. This deprecation was already part of pytest
  7.0.0rc1 but wasn't documented.</p>
  </li>
  </ul>
  <h2>Bug Fixes</h2>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9355">#9355</a>: Fixed error message prints function decorators when using assert in Python 3.8 and above.</li>
  <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9396">#9396</a>: Ensure <code>pytest.Config.inifile</code>{.interpreted-text role=&quot;attr&quot;} is available during the <code>pytest_cmdline_main &lt;_pytest.hookspec.pytest_cmdline_main&gt;</code>{.interpreted-text role=&quot;func&quot;} hook (regression during <code>7.0.0rc1</code>).</li>
  </ul>
  <h2>Improved Documentation</h2>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9404">#9404</a>: Added extra documentation on alternatives to common misuses of [pytest.warns(None)]{.title-ref} ahead of its deprecation.</li>
  <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9505">#9505</a>: Clarify where the configuration files are located. To avoid confusions documentation mentions
  that configuration file is located in the root of the repository.</li>
  </ul>
  <h2>Trivial/Internal Changes</h2>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9521">#9521</a>: Add test coverage to assertion rewrite path.</li>
  </ul>
  <h1>pytest 7.0.0rc1 (2021-12-06)</h1>
  <h2>Breaking Changes</h2>
  <ul>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/7259">#7259</a>: The <code>Node.reportinfo() &lt;non-python tests&gt;</code>{.interpreted-text role=&quot;ref&quot;} function first return value type has been expanded from [py.path.local | str]{.title-ref} to [os.PathLike[str] | str]{.title-ref}.</p>
  <p>Most plugins which refer to [reportinfo()]{.title-ref} only define it as part of a custom <code>pytest.Item</code>{.interpreted-text role=&quot;class&quot;} implementation.
  Since [py.path.local]{.title-ref} is a [os.PathLike[str]]{.title-ref}, these plugins are unaffacted.</p>
  <p>Plugins and users which call [reportinfo()]{.title-ref}, use the first return value and interact with it as a [py.path.local]{.title-ref}, would need to adjust by calling [py.path.local(fspath)]{.title-ref}.
  Although preferably, avoid the legacy [py.path.local]{.title-ref} and use [pathlib.Path]{.title-ref}, or use [item.location]{.title-ref} or [item.path]{.title-ref}, instead.</p>
  <p>Note: pytest was not able to provide a deprecation period for this change.</p>
  </li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest/commit/3554b833c24f541af5c7272e47cfa52b89d8138d"><code>3554b83</code></a> Add note to changelog</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/6ea7f990a040cacfbc671a8c29f2d76533d9d523"><code>6ea7f99</code></a> Prepare release version 7.0.0</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/737b220516870b3f90e87109c2c6657856dca506"><code>737b220</code></a> [7.0.x] releasing: Add template for major releases (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9597">#9597</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/7fa397296339773ee7a94df8684573fe7bdf7072"><code>7fa3972</code></a> [7.0.x] releasing: Always set doc_version (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9590">#9590</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/b304499925c4ce39a6a45dcc3394d5d0f0aa6b94"><code>b304499</code></a> [7.0.x] Make 'warnings' and 'deselected' in assert_outcomes optional (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9566">#9566</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/f17525df26e65b8de304dad919f66aaeb61f1f5c"><code>f17525d</code></a> [7.0.x] doc: Add ellipsis to warning usecase list (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9562">#9562</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/0a7be971d2b37907099e870095e49a31dc25d194"><code>0a7be97</code></a> ci: Bump up timeout (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9565">#9565</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/c17908cdb3566508d8ac896552ac7cec092be24d"><code>c17908c</code></a> [7.0.x] doc: Recategorize 7.0.0 changelog items (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9564">#9564</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/ab549bba8a9ee7f55f3f9a2dc884f47cca34f6c4"><code>ab549bb</code></a> [7.0.x] Add missing cooperative constructor changelog (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9563">#9563</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/4b1707ff7041256381b95b5647ee6cc8336b54d2"><code>4b1707f</code></a> [7.0.x] Autouse linearization graph (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9557">#9557</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest/compare/5.2.2...7.0.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1629/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DAn7V
title: Add SpatiaLite helpers to CLI
state: closed
created_at: 2022-02-04T14:01:28Z
updated_at: 2022-02-16T01:02:29Z
closed_at: 2022-02-16T00:58:07Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Now that #385 is merged, add CLI versions of those methods.

  ```sh
  # init spatialite
  sqlite-utils init-spatialite database.db

  # or maybe/also
  sqlite-utils create database.db --enable-wal --spatialite

  # add geometry columns
  # needs a database, table, geometry column name, type, with optional SRID and not-null
  # this needs to create a table if it doesn't already exist
  sqlite-utils add-geometry-column database.db table-name geometry --srid 4326 --not-null

  # spatial index an existing table/column
  sqlite-utils create-spatial-index database.db table-name geometry
  ```

  Should be mostly straightforward. The one thing worth highlighting in docs is that geometry columns can only be added to existing tables. Trying to add a geometry column to a table that doesn't exist yet might mean you have a schema like `{"rowid": int, "geometry": bytes}`. Might be worth nudging people to explicitly create a table first, then add geometry columns.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/398/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DCgpI
title: Make it easier to insert geometries, with documentation and maybe code
state: open
created_at: 2022-02-05T00:11:26Z
updated_at: 2023-05-16T03:11:52Z
author_association: OWNER
type: issue
body:
  In playing with the new SpatiaLite helpers from #385 I noticed that actually populating geometry columns is still a little bit tricky. Here's what I ended up doing:

  ```python
  import httpx, sqlite_utils
  db = sqlite_utils.Database("/tmp/spatial.db")
  attractions = httpx.get("https://latest.datasette.io/fixtures/roadside_attractions.json?_shape=array").json()
  db["attractions"].insert_all(attractions, pk="pk")

  # Schema of that table is now:
  # CREATE TABLE [attractions] (
  #    [pk] INTEGER PRIMARY KEY,
  #    [name] TEXT,
  #    [address] TEXT,
  #    [latitude] FLOAT,
  #    [longitude] FLOAT
  # )

  db.init_spatialite()
  db["attractions"].add_geometry_column("point", "POINT")

  db.execute("""
      update attractions set point = GeomFromText(
        'POINT(' || longitude || ' ' || latitude || ')', 4326
      )
  """)
  ```
  That last line took some figuring out - especially the need for the SRID of `4326`, without which I got this error:

  > `IntegrityError: attractions.point violates Geometry constraint [geom-type or SRID not allowed]`

  It would be good to both document this in more detail, but ideally also to come up with a more obvious pattern for inserting common types of spatial data.

  Also related:
  - #398
  - #79

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/399/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DD1BH
title: `sqlite-utils create-table` ... `--if-not-exists`
state: closed
created_at: 2022-02-06T01:32:53Z
updated_at: 2022-02-06T01:34:53Z
closed_at: 2022-02-06T01:34:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Inspired by:
  - #397

  To match the option on `create-index`: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#create-index

  ```
    --if-not-exists        Ignore if index already exists
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/400/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DD2Io
title: Update SpatiaLite example in the documentation
state: closed
created_at: 2022-02-06T02:02:07Z
updated_at: 2022-02-06T02:05:03Z
closed_at: 2022-02-06T02:03:24Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This one here: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions

  It should take advantage of the new methods from:
  - #79

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/401/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DEq5J
title: Advanced class-based `conversions=` mechanism
state: open
created_at: 2022-02-06T19:47:41Z
updated_at: 2022-02-16T10:18:55Z
author_association: OWNER
type: issue
body:
  The `conversions=` parameter works like this at the moment: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions

  ```python
  db["places"].insert(
      {"name": "Wales", "geometry": wkt},
      conversions={"geometry": "GeomFromText(?, 4326)"},
  )
  ```
  This proposal is to support values in that dictionary that are objects, not strings, which can represent more complex conversions - spun out from #399.

  New proposed mechanism:
  ```python
  from sqlite_utils.utils import LongitudeLatitude

  db["places"].insert(
      {
          "name": "London",
          "point": (-0.118092, 51.509865)
      },
      conversions={"point": LongitudeLatitude},
  )
  ```
  Here `LongitudeLatitude` is a magical value which does TWO things: it sets up the `GeomFromText(?, 4326)` SQL function, and it handles converting the `(51.509865, -0.118092)` tuple into a `POINT({} {})` string.

  This would involve a change to the `conversions=` contract - where it usually expects a SQL string fragment, but it can also take an object which combines that SQL string fragment with a Python conversion function.

  Best of all... this resolves the `lat, lon` v.s. `lon, lat` dilemma because you can use `from sqlite_utils.utils import LongitudeLatitude` OR `from sqlite_utils.utils import LatitudeLongitude` depending on which you prefer!

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/399#issuecomment-1030739566_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/402/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5DFu9f
title: Review datasette.utils and decide which functions should be documented for 1.0
state: open
created_at: 2022-02-07T06:39:52Z
updated_at: 2022-02-07T06:39:52Z
author_association: OWNER
body:
  Follows:
  - #1176

type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1630/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4yK44E
title: Update pytest-asyncio requirement from <0.17,>=0.10 to >=0.10,<0.19
state: closed
created_at: 2022-02-07T13:13:19Z
updated_at: 2022-03-06T01:29:54Z
closed_at: 2022-03-06T01:29:53Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1631
type: pull
body:
  Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-asyncio/releases">pytest-asyncio's releases</a>.</em></p>
  <blockquote>
  <h2>pytest-asyncio 0.18.0</h2>
  <hr />
  <h2>title: 'pytest-asyncio: pytest support for asyncio'</h2>
  <p><a href="https://pypi.python.org/pypi/pytest-asyncio"><img src="https://img.shields.io/pypi/v/pytest-asyncio.svg" alt="image" /></a></p>
  <p><a href="https://github.com/pytest-dev/pytest-asyncio/actions?workflow=CI"><img src="https://github.com/pytest-dev/pytest-asyncio/workflows/CI/badge.svg" alt="image" /></a></p>
  <p><a href="https://codecov.io/gh/pytest-dev/pytest-asyncio"><img src="https://codecov.io/gh/pytest-dev/pytest-asyncio/branch/master/graph/badge.svg" alt="image" /></a></p>
  <p><a href="https://github.com/pytest-dev/pytest-asyncio"><img src="https://img.shields.io/pypi/pyversions/pytest-asyncio.svg" alt="Supported Python versions" /></a></p>
  <p><a href="https://github.com/ambv/black"><img src="https://img.shields.io/badge/code%20style-black-000000.svg" alt="image" /></a></p>
  <p>pytest-asyncio is an Apache2 licensed library, written in Python, for
  testing asyncio code with pytest.</p>
  <p>asyncio code is usually written in the form of coroutines, which makes
  it slightly more difficult to test using normal testing tools.
  pytest-asyncio provides useful fixtures and markers to make testing
  easier.</p>
  <pre lang="{.sourceCode" data-meta=".python}"><code>@pytest.mark.asyncio
  async def test_some_asyncio_code():
      res = await library.do_something()
      assert b&quot;expected result&quot; == res
  </code></pre>
  <p>pytest-asyncio has been strongly influenced by
  <a href="https://github.com/eugeniy/pytest-tornado">pytest-tornado</a>.</p>
  <h1>Features</h1>
  <ul>
  <li>fixtures for creating and injecting versions of the asyncio event
  loop</li>
  <li>fixtures for injecting unused tcp/udp ports</li>
  <li>pytest markers for treating tests as asyncio coroutines</li>
  <li>easy testing with non-default event loops</li>
  <li>support for [async def]{.title-ref} fixtures and async generator
  fixtures</li>
  <li>support <em>auto</em> mode to handle all async fixtures and tests
  automatically by asyncio; provide <em>strict</em> mode if a test suite
  should work with different async frameworks simultaneously, e.g.
  <code>asyncio</code> and <code>trio</code>.</li>
  </ul>
  <h1>Installation</h1>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/07e9922f1dc9cb84f0948e6c84ad9921c3662969"><code>07e9922</code></a> Prepare release of v0.18.0 (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/282">#282</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/a92d8947ab57c7d88082dff9dcded686622ed47b"><code>a92d894</code></a> Relax dependency on typing-extensions (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/284">#284</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/e78b59d5d8f8fed7d3aaf39c184a7d4a337676de"><code>e78b59d</code></a> Fix setuptools_scm configuration (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/279">#279</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/d8efa640f0aa1ba8856b908ba486150588018209"><code>d8efa64</code></a> Support parametrized event_loop fixture (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/278">#278</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/dab3b5184c0bdd0bd93dddeb6968de7f565fb8db"><code>dab3b51</code></a> Raise a warning if <a href="https://github.com/pytest"><code>@​pytest</code></a>.mark.asyncio is applied to non-async function (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/275">#275</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/048a6edc56e7d2c4a5f1283b5d2dd93a1270bfce"><code>048a6ed</code></a> Code cleanup: drop 'request' argument from async fixtures (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/270">#270</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/435332735305417413c04ecf29056243137d1261"><code>4353327</code></a> Update changelog</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/c919fd7be35b7f10ba47c953a336baa73cfa8653"><code>c919fd7</code></a> Remove unconditional test item attribute access (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/268">#268</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/39c6d0ce88868d8398606ea10541befa7d7241cd"><code>39c6d0c</code></a> Update README</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/547933dabaa56b3d9c244c8302252f3b638463d2"><code>547933d</code></a> Require typing-extensions on py&lt;3.8 only (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/269">#269</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-asyncio/compare/v0.10.0...v0.18.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1631/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5DJp2i
title: datasette one.db one.db opens database twice, as one and one_2
state: closed
created_at: 2022-02-07T23:14:47Z
updated_at: 2022-03-19T04:04:49Z
closed_at: 2022-02-07T23:50:01Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > ```
  > % mkdir /tmp/data
  > % cp ~/Dropbox/Development/datasette/fixtures.db /tmp/data 
  > % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852
  > ...
  > INFO:     Uvicorn running on http://127.0.0.1:8852 (Press CTRL+C to quit)
  > ^CINFO:     Shutting down
  > % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852
  > ...
  > INFO:     127.0.0.1:49533 - "GET / HTTP/1.1" 200 OK
  > ```
  > The first time I ran Datasette I got two databases - `fixtures` and `created`
  > 
  > BUT... when I ran Datasette the second time it looked like this:
  > 
  > <img width="697" alt="image" src="https://user-images.githubusercontent.com/9599/152887735-5e2b4f21-001d-4e7a-a23a-16a32fdcf554.png">
  > 
  > This is the same result you get if you run:
  > 
  >     datasette /tmp/data/fixtures.db /tmp/data/created.db /tmp/data/created.db
  > 
  > This is caused by this Datasette issue:
  > - https://github.com/simonw/datasette/issues/509
  > 
  > So... either I teach Datasette to de-duplicate multiple identical file paths passed to the command, or I can't use `/data/*.db` in the `Dockerfile` here and I need to go back to other solutions for the challenge described in this comment: https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1031971831

  _Originally posted by @simonw in https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1032029874_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1632/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DJ_Ti
title: Document how to add a primary key to a rowid table using `sqlite-utils transform --pk`
state: closed
created_at: 2022-02-08T01:39:40Z
updated_at: 2022-02-09T04:22:43Z
closed_at: 2022-02-08T19:33:59Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  *Original title: Add option for adding a new, serial, primary key*

  sometimes we have tables that don't have primary keys, but ought to have them. we *can* use rowid for that, but it would often be nicer to have an explicit primary key. using the current value of rowid would be fine.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/403/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DPcCD
title: Add example of `--convert` to the help for `sqlite-utils insert`
state: closed
created_at: 2022-02-09T06:49:09Z
updated_at: 2022-02-09T06:56:35Z
closed_at: 2022-02-09T06:55:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://sqlite-utils.datasette.io/en/3.23/cli-reference.html#insert would be more useful if it included an example of `--convert` in action.

  I can maybe use an example from https://simonwillison.net/2022/Jan/11/sqlite-utils/

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/404/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DPgpv
title: `Database(memory_name="name")` constructor argument
state: closed
created_at: 2022-02-09T07:15:03Z
updated_at: 2022-02-16T01:23:16Z
closed_at: 2022-02-16T01:23:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  SQLite in-memory databases can be named, in which case multiple connections can be opened to a shared in-memory database running within the same process.

  Datasette supports this - SQLite could support it too.

  https://docs.datasette.io/en/0.60.2/internals.html#database-ds-path-none-is-mutable-false-is-memory-false-memory-name-none

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/405/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5DQwbC
title: Creating tables with custom datatypes
state: open
created_at: 2022-02-09T12:16:31Z
updated_at: 2022-09-15T18:13:50Z
author_association: NONE
type: issue
body:
  Via https://stackoverflow.com/a/18622264/454773 I note the ability to register custom handlers for novel datatypes that can map into and out of things like sqlite `BLOB`s.

  From a quick look and a quick play, I didn't spot a way to do this in `sqlite_utils`?

  For example:

  ```python
  # Via https://stackoverflow.com/a/18622264/454773
  import sqlite3
  import numpy as np
  import io

  def adapt_array(arr):
      """
      http://stackoverflow.com/a/31312102/190597 (SoulNibbler)
      """
      out = io.BytesIO()
      np.save(out, arr)
      out.seek(0)
      return sqlite3.Binary(out.read())

  def convert_array(text):
      out = io.BytesIO(text)
      out.seek(0)
      return np.load(out)


  # Converts np.array to TEXT when inserting
  sqlite3.register_adapter(np.ndarray, adapt_array)

  # Converts TEXT to np.array when selecting
  sqlite3.register_converter("array", convert_array)
  ```

  ```python
  from sqlite_utils import Database
  db = Database('test.db')

  # Reset the database connection to used the parsed datatype
  # sqlite_utils doesn't seem to support eg:
  #  Database('test.db', detect_types=sqlite3.PARSE_DECLTYPES)
  db.conn = sqlite3.connect(db_name, detect_types=sqlite3.PARSE_DECLTYPES)

  # Create a table the old fashioned way
  # but using the new custom data type
  vector_table_create = """
  CREATE TABLE dummy 
      (title TEXT, vector array );
  """

  cur = db.conn.cursor()
  cur.execute(vector_table_create)


  # sqlite_utils doesn't appear to support custom types (yet?!)
  # The following errors on the "array" datatype
  """
  db["dummy"].create({
      "title": str,
      "vector": "array",
  })
  """
  ```

  We can then add / retrieve records from the database where the datatype of the `vector` field is a custom registered `array` type (which is to say, a `numpy` array):

  ```python
  import numpy as np

  db["dummy"].insert({'title':"test1", 'vector':np.array([1,2,3])})

  for row in db.query("SELECT * FROM dummy"):
      print(row['title'], row['vector'], type(row['vector']))

  """
  test1 [1 2 3] <class 'numpy.ndarray'>
  """
  ```

  It would be handy to be able to do this idiomatically in `sqlite_utils`.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/406/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5DS_gM
title: base_url or prefix does not work with _exact match
state: open
created_at: 2022-02-09T21:45:07Z
updated_at: 2022-04-28T09:12:56Z
author_association: NONE
type: issue
body:
  When i hit "Apply" button to search with "_exact" for a column syntax the URL prefix is removed from the url.

  ![image](https://user-images.githubusercontent.com/6613091/153293758-0b757d55-5757-4987-992e-9426e69a7956.png)

  And the result is:
  ![image](https://user-images.githubusercontent.com/6613091/153294672-87be7809-bb7b-455d-bf1a-41e90bbfa4ae.png)

  If I add the marked row to url_builder.py it seams to work:
  ![image](https://user-images.githubusercontent.com/6613091/153295231-bdd52e37-efcf-4b21-9d37-69f182a922f4.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1633/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5DbjFU
title: Update Dockerfile generated by `datasette publish`
state: open
created_at: 2022-02-11T00:07:26Z
updated_at: 2022-03-11T17:38:08Z
author_association: OWNER
type: issue
body:
  The generated `Dockerfile` currently looks something like this:
  ```Dockerfile
  FROM python:3.8
  COPY . /app
  WORKDIR /app

  ENV DATASETTE_SECRET 'edab49cbc5d5f6f33238f54852037e3fee710821960b73edd2ce743454182ae2'
  RUN pip install -U datasette datasette-auth-passwords datasette-tiddlywiki datasette-graphql
  RUN datasette inspect fixtures.db other.db --inspect-file inspect-data.json
  ENV PORT 8080
  EXPOSE 8080
  CMD datasette serve --host 0.0.0.0 -i fixtures.db -i other.db --cors --inspect-file inspect-data.json --metadata metadata.json --create --port $PORT /data/*.db
  ```
  This is still on Python 3.8, and it generates a pretty large image compared to the `Dockerfile` used for https://hub.docker.com/datasetteproject/datasette - https://github.com/simonw/datasette/blob/0.60.2/Dockerfile

  Here's the code that generates it: https://github.com/simonw/datasette/blob/7d24fd405f3c60e4c852c5d746c91aa2ba23cf5b/datasette/utils/__init__.py#L389-L400

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1634/reactions", "total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 2, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5D1J_q
title: "permissions" propery in metadata for configuring arbitrary permissions
state: closed
created_at: 2022-02-15T00:25:59Z
updated_at: 2022-12-13T02:40:50Z
closed_at: 2022-12-13T02:40:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The `"allow"` block mechanism can already be used to configure various default permissions. When adding permissions to `datasette-tiddlywiki` I realized it would be good to be able to configure arbitrary permissions such as `edit-tiddlywiki` there too.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1636/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4y3yW0
title: Add SpatiaLite helpers to CLI
state: closed
created_at: 2022-02-15T16:50:17Z
updated_at: 2022-02-16T01:49:40Z
closed_at: 2022-02-16T00:58:08Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/407
type: pull
body:
  Closes #398 

  This adds SpatiaLite helpers to the CLI. 

  ```sh
  # init spatialite when creating a database
  sqlite-utils create database.db --enable-wal --init-spatialite

  # add geometry columns
  # needs a database, table, geometry column name, type, with optional SRID and not-null
  # this will throw an error if the table doesn't already exist
  sqlite-utils add-geometry-column database.db table-name geometry --srid 4326 --not-null

  # spatial index an existing table/column
  # this will throw an error it the table and column don't exist
  sqlite-utils create-spatial-index database.db table-name geometry
  ```

  Docs and tests are included. 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/407/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5EEy8V
title: `filters_from_request` plugin hook docs should mention that returning an async function is allowed
state: open
created_at: 2022-02-18T00:08:26Z
updated_at: 2022-02-18T00:08:26Z
author_association: OWNER
type: issue
body:
  https://docs.datasette.io/en/stable/plugin_hooks.html#filters-from-request-request-database-table-datasette doesn't mention that you can return an `async` function - but you can, and in fact Datasette itself uses that here: https://github.com/simonw/datasette/blob/aa7f0037a46eb76ae6fe9bf2a1f616c58738ecdf/datasette/filters.py#L43-L47

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1638/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5ETMfS
title: `deterministic=True` fails on versions of SQLite prior to 3.8.3
state: closed
created_at: 2022-02-21T14:36:43Z
updated_at: 2022-03-13T16:54:09Z
closed_at: 2022-03-02T00:38:11Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi, love your work.

  I am unable to lookup indexes in a database using sqlite-utils:

  `
  sqlite-utils indexes city_spec.db --table`

  or

  `sqlite-utils indexes city_spec.db MyTable
  `

  **Software**
  sqlite-utils, version 3.24
  sqlite3 --version: 3.36.0 

  **Output:**

  Traceback (most recent call last):
    File "/opt/app-root/bin/sqlite-utils", line 8, in <module>
      sys.exit(cli())
    File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/opt/app-root/lib64/python3.8/site-packages/click/decorators.py", line 26, in new_func
      return f(get_current_context(), *args, **kwargs)
    File "/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/cli.py", line 2123, in indexes
      ctx.invoke(
    File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/cli.py", line 1624, in query
      db.register_fts4_bm25()
    File "/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py", line 403, in register_fts4_bm25
      self.register_function(rank_bm25, deterministic=True)
    File "/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py", line 399, in register_function
      register(fn)
    File "/opt/app-root/lib64/python3.8/site-packages/sqlite_utils/db.py", line 392, in register
      self.conn.create_function(name, arity, fn, **kwargs)
  sqlite3.NotSupportedError: deterministic=True requires SQLite 3.8.3 or higher

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/408/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5EdtaU
title: Make datasette-redirect-forbidden unneccessary
state: open
created_at: 2022-02-23T22:18:46Z
updated_at: 2022-02-23T22:18:46Z
author_association: OWNER
type: issue
body:
  I wrote `datasette-redirect-forbidden` today because I needed 403 errors to redirect to `/-/login` and it was the quickest way to solve that problem.

  This should be a feature of Datasette core.

  - https://github.com/simonw/datasette-redirect-forbidden/issues/2

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1639/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5EeCp0
title: Support static assets where file length may change, e.g. logs
state: open
created_at: 2022-02-24T00:34:42Z
updated_at: 2022-03-05T01:19:25Z
author_association: NONE
type: issue
body:
  This is a bit of an oxymoron.

  I am serving a log.txt file for a background process using the Datasette --static CLI. This is useful as I can observe a background process from the web UI to see any errors that occur (instead of spelunking the logs via docker exec/ssh etc).

  I get this error, which I think is because Datasette assumes that the size of the content does not change (but appending new log lines means the content length changes).


  ```python
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/datasette/app.py", line 1181, in route_path
      response = await view(request, send)
    File "/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py", line 305, in inner_static
      await asgi_send_file(send, full_path, chunk_size=chunk_size)
    File "/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py", line 280, in asgi_send_file
      await send(
    File "/usr/local/lib/python3.9/site-packages/asgi_csrf.py", line 104, in wrapped_send
      await send(event)
    File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 460, in send
      output = self.conn.send(event)
    File "/usr/local/lib/python3.9/site-packages/h11/_connection.py", line 468, in send
      data_list = self.send_with_data_passthrough(event)
    File "/usr/local/lib/python3.9/site-packages/h11/_connection.py", line 501, in send_with_data_passthrough
      writer(event, data_list.append)
    File "/usr/local/lib/python3.9/site-packages/h11/_writers.py", line 58, in __call__
      self.send_data(event.data, write)
    File "/usr/local/lib/python3.9/site-packages/h11/_writers.py", line 78, in send_data
      raise LocalProtocolError("Too much data for declared Content-Length")
  h11._util.LocalProtocolError: Too much data for declared Content-Length
  ERROR:    Exception in ASGI application
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/datasette/app.py", line 1181, in route_path
      response = await view(request, send)
    File "/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py", line 305, in inner_static
      await asgi_send_file(send, full_path, chunk_size=chunk_size)
    File "/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py", line 280, in asgi_send_file
      await send(
    File "/usr/local/lib/python3.9/site-packages/asgi_csrf.py", line 104, in wrapped_send
      await send(event)
    File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 460, in send
      output = self.conn.send(event)
    File "/usr/local/lib/python3.9/site-packages/h11/_connection.py", line 468, in send
      data_list = self.send_with_data_passthrough(event)
    File "/usr/local/lib/python3.9/site-packages/h11/_connection.py", line 501, in send_with_data_passthrough
      writer(event, data_list.append)
    File "/usr/local/lib/python3.9/site-packages/h11/_writers.py", line 58, in __call__
      self.send_data(event.data, write)
    File "/usr/local/lib/python3.9/site-packages/h11/_writers.py", line 78, in send_data
      raise LocalProtocolError("Too much data for declared Content-Length")
  h11._util.LocalProtocolError: Too much data for declared Content-Length
  ```

  Thanks, I am finding Datasette very useful.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1640/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5EgRX4
title: Tweak mobile keyboard settings
state: open
created_at: 2022-02-24T13:47:10Z
updated_at: 2022-02-24T13:49:26Z
author_association: OWNER
type: issue
body:
  https://developer.apple.com/library/archive/documentation/StringsTextFonts/Conceptual/TextAndWebiPhoneOS/KeyboardManagement/KeyboardManagement.html#//apple_ref/doc/uid/TP40009542-CH5-SW12

  `autocorrect="off"` is worth experimenting with.

  Twitter: https://twitter.com/forestgregg/status/1496842959563726852

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1641/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODFdgUs4zaUta
title: scrape-dependents: enable paging through package menu option if present
state: open
created_at: 2022-02-24T15:07:25Z
updated_at: 2022-02-24T15:07:25Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/github-to-sqlite/pulls/70
type: pull
body:
  Some repos organize network dependents by a Package toggle. This PR adds the ability to page through those options and scrape underlying dependents.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/70/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5EhnEx
title: `with db:` for transactions
state: open
created_at: 2022-02-24T19:22:06Z
updated_at: 2022-10-01T03:42:50Z
author_association: OWNER
body: This can be a documented wrapper around `with db.conn:`.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/409/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4zbaJy
title: Correct spelling mistakes (found with codespell)
state: closed
created_at: 2022-02-24T20:44:18Z
updated_at: 2022-03-06T08:48:29Z
closed_at: 2022-03-01T21:05:29Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/410
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/410/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Eqzlb
title: Dependency issue with asgiref and uvicorn
state: closed
created_at: 2022-02-26T18:00:35Z
updated_at: 2022-03-05T01:11:27Z
closed_at: 2022-03-05T01:11:17Z
author_association: OWNER
type: issue
state_reason: completed
body:
  ```
  ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

  We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

  datasette 0.60.2 requires asgiref<3.5.0,>=3.2.10, but you'll have asgiref 3.5.0 which is incompatible.
  ```
  That's after I forced an upgrade of `uvicorn` due to this warning:
  ```
  ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

  We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

  uvicorn 0.13.1 requires click==7.*, but you'll have click 8.0.4 which is incompatible.
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1642/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Ezr5h
title: Sensible `cache-control` headers for static assets, including those served by plugins
state: open
created_at: 2022-02-28T18:12:03Z
updated_at: 2022-03-08T02:59:29Z
author_association: NONE
type: issue
body:
  ## What I'm seeing

  With `default_cache_ttl = 86400`, I see the following:

  A table view returns `Cache-control: max-age=86400`:

  ![Screenshot_20220228_190000](https://user-images.githubusercontent.com/697092/156034352-4d64683e-39c8-49af-81df-0217a5957bbd.png)

  A static asset returns no `Cache-control` header:

  ![Screenshot_20220228_185933](https://user-images.githubusercontent.com/697092/156034363-d0b03cc2-5889-4ed2-b601-8c1846b8469a.png)

  ## What I expected to see

  I expected the static asset to return a `Cache-control` header indicating that this response can be cached.

  ## Why this matters

  I'm productionising a Datasette deployment right now and was looking into putting it behind a Varnish instance. I was surprised to see requests for static assets being served from Datasette rather than Varnish, this is what led me to look more closely at the response headers.

  While Datasette serves those static assets pretty quickly, I don't see why Datasette should serve them. By their nature, static assets like images and JS files are very cacheable, so it should be easy to serve them from a cache like Varnish.

  (Note that Varnish can easily be configured to override this header, enabling caching for static assets. But it would be better if this override was not necessary.)

  ## Discussion

  It seems clear to me that serving static assets without a `Cache-control` header is not ideal.

  I see two options here:

  A. Static assets use the same logic as table / SQL views to set the `Cache-control` header based on `default_cache_ttl`.
  B. An additional setting for static assets is introduced (`default_static_cache_ttl`, say).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1645/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5E-TMu
title: Configuration directory mode does not pick up other file extensions than .db
state: closed
created_at: 2022-03-02T13:15:23Z
updated_at: 2022-10-07T23:06:17Z
closed_at: 2022-10-07T23:03:35Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hello, I've been trying to run Datasette with the [configuration directory mode](https://docs.datasette.io/en/stable/settings.html#configuration-directory-mode) with a structure such as this one:

  ```plain
  some-directory/
      example.sqlite3
      another-example.db
      one-more.custom
      [...]
  ```

  (In my scenario I can't just change the filename extension without other problems arising)

  Now databases with the `.sqlite3` or the custom filename extension are ignored by Datasette in this case. I'm aware that the docs state that a `.db` extension is required, but I was wondering if there is a reason for restricting this or any workaround available? When I run `datasette example.sqlite3` or `datasette one-more.custom` the databases are served by Datasette without a problem. 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1646/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5FJLi4
title: Support for generated columns
state: open
created_at: 2022-03-04T20:41:33Z
updated_at: 2022-03-11T22:32:43Z
author_association: CONTRIBUTOR
type: issue
body:
  This is a fairly new feature -- SQLite version 3.31.0 (2020-01-22) -- that I, admittedly, haven't gotten to work yet. But it looks _incredibly_ useful: https://dgl.cx/2020/06/sqlite-json-support

  I'm not sure if this is an option on `add-column` or a separate command like `add-generated-column`. Either way, it needs an argument to populate it. It could be something like this:

  ```sh
  sqlite-utils add-column data.db table-name generated --as 'json_extract(data, "$.field")' --virtual
  ```

  More here: https://www.sqlite.org/gencol.html

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/411/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5FJvvw
title: Optional Pandas integration
state: open
created_at: 2022-03-05T01:49:27Z
updated_at: 2022-06-14T15:36:29Z
author_association: OWNER
type: issue
body:
  It would be neat if there was a way to use this more seamlessly with Pandas, in particular Pandas dataframes - but without making Pandas a required dependency.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/412/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODEm0Qs4z_V3w
title: Update Twitter dev link, clarify apps vs projects
state: open
created_at: 2022-03-05T11:56:08Z
updated_at: 2022-03-05T11:56:08Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/twitter-to-sqlite/pulls/65
type: pull
body:
  Twitter pushes you heavily towards v2 projects instead of v1 apps – I know the README mentions v1 API compatibility at the top, but I still nearly got turned around here.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/65/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FKmgf
title: Test failures with SQLite 3.37.0+ due to column affinity case
state: closed
created_at: 2022-03-05T17:37:46Z
updated_at: 2022-03-05T19:56:28Z
closed_at: 2022-03-05T19:47:04Z
author_association: OWNER
type: issue
state_reason: completed
body:
  These three tests are failing on my local machine:
  ```
  FAILED tests/test_internals_database.py::test_table_column_details[facetable-expected0] - AssertionError: assert [Column(cid=0, name='pk', type='INTEGER', no...
  FAILED tests/test_internals_database.py::test_table_column_details[sortable-expected1] - AssertionError: assert [Column(cid=0, name='pk1', type='varchar(30)'...
  FAILED tests/test_table_html.py::test_sort_links - AssertionError: assert [{'a_href': None,\n  'attrs': {'class': ['col-Link'],\n            'data-column': '...
  ```
  I ran `pytest --lf -vv` and the output had things like this in it:
  ```
  E         -  Column(cid=1, name='created', type='text', notnull=0, default_value=None, is_pk=0, hidden=0),
  E         ?                                      ^^^^
  E         +  Column(cid=1, name='created', type='TEXT', notnull=0, default_value=None, is_pk=0, hidden=0),
  ...
  E            {'a_href': '/fixtures/sortable?_sort=sortable_with_nulls_2',
  E             'attrs': {'class': ['col-sortable_with_nulls_2'],
  E                       'data-column': 'sortable_with_nulls_2',
  E                       'data-column-not-null': '0',
  E         -             'data-column-type': 'real',
  E         ?                                  ^^^^
  E         +             'data-column-type': 'REAL',
  E         ?                                  ^^^^
  ```
  Something is causing column types to come back in uppercase where previously they were lowercase.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1647/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4z_p6S
title: Use dash encoding for table names and row primary keys in URLs
state: closed
created_at: 2022-03-05T19:50:45Z
updated_at: 2022-03-07T15:38:30Z
closed_at: 2022-03-07T15:38:30Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1648
type: pull
body:
  Refs #1439.

  - [x] Build `dash_encode` / `dash_decode` functions
  - [x] Use dash encoding for row primary keys
  - [x] Use dash encoding for `?_next=` pagination tokens
  - [x] Use dash encoding for table names in URLs
  - [x] Use dash encoding for database name
  - ~~Implement redirects from previous `%` URLs that replace those with `-`~~ - separate issue: #1650

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1648/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c40AW_v
title: Add /opt/homebrew to where spatialite extension can be found
state: closed
created_at: 2022-03-06T18:09:35Z
updated_at: 2022-03-06T22:46:00Z
closed_at: 2022-03-06T19:39:15Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1649
type: pull
body:
  Helps homebrew on Apple Silicon setups find spatialite without needing
  a full path.

  Similar to #1114

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1649/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FL6Z5
title: Implement redirects from old % encoding to new dash encoding
state: closed
created_at: 2022-03-06T23:40:02Z
updated_at: 2022-03-07T19:26:15Z
closed_at: 2022-03-07T19:26:14Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > One big advantage to this scheme is that redirecting old links to `%2F` pages (e.g. https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators) is easy - if you see a `%` in the `raw_path`, redirect to that page with the `%` replaced by `-`.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1439#issuecomment-1060044007_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1650/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FPF9M
title: Get rid of the no-longer necessary ?_format=json hack for tables called x.json
state: closed
created_at: 2022-03-07T15:40:42Z
updated_at: 2022-03-19T04:04:50Z
closed_at: 2022-03-15T18:25:42Z
author_association: OWNER
body:
  Tidy up from:
  - #1439

type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1651/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FQcCx
title: Mechanism to default a table to sorting by multiple columns
state: open
created_at: 2022-03-07T21:20:11Z
updated_at: 2022-03-07T21:23:39Z
author_association: OWNER
type: issue
body:
  ### Discussed in https://github.com/simonw/datasette/discussions/1652

  <div type='discussions-op-text'>

  <sup>Originally posted by **zaneselvans** March  7, 2022</sup>
  It's easy to tell datasette to sort tables using a single column, as [described in the docs](https://docs.datasette.io/en/stable/metadata.html#setting-a-default-sort-order):

  ```yaml
  databases:
    ferc1:
      tables:
        f1_edcfu_epda:
          sort: created_time
  ```

  But is there some way to tell it to sort using a composite key, like you would in an `ORDER BY` clause instead? For example, the way it's being done **[in this query](https://data.catalyst.coop/ferc1?sql=select%0D%0A++rowid%2C%0D%0A++respondent_id%2C%0D%0A++report_year%2C%0D%0A++spplmnt_num%2C%0D%0A++row_number%2C%0D%0A++row_seq%2C%0D%0A++row_prvlg%2C%0D%0A++acct_num%2C%0D%0A++depr_plnt_base%2C%0D%0A++est_avg_srvce_lf%2C%0D%0A++net_salvage%2C%0D%0A++apply_depr_rate%2C%0D%0A++mrtlty_crv_typ%2C%0D%0A++avg_remaining_lf%2C%0D%0A++report_prd%0D%0Afrom%0D%0A++f1_edcfu_epda%0D%0Awhere%0D%0A++respondent_id+%3D+210%0D%0A++AND+report_year+%3D+2020%0D%0Aorder+by%0D%0A++report_year%2C+report_prd%2C+respondent_id%2C+spplmnt_num%2C+row_number%0D%0Alimit%0D%0A++1000)** on our Datasette?

  ```sql
  SELECT
    respondent_id,
    report_year,
    spplmnt_num,
    row_number,
    row_seq,
    row_prvlg,
    acct_num,
    depr_plnt_base,
    est_avg_srvce_lf,
    net_salvage,
    apply_depr_rate,
    mrtlty_crv_typ,
    avg_remaining_lf,
    report_prd
  FROM
    f1_edcfu_epda
  WHERE
    respondent_id = 210
    AND report_year = 2020
  ORDER BY
    report_year, report_prd, respondent_id, spplmnt_num, row_number
  LIMIT
    1000
  ```

  The problem here is that by default it's using `rowid` (the SQLite assigned autoincrementing integer key) to order the records, but the table **should** have a natural composite primary key, but the original database that this data is being migrated from doesn't enforce unique primary keys, so there are dupes, and we don't want to drop those rows, and the records are somehow getting jumbled in the database (the `rowid` ordering isn't lined up with the expected ordering based on the composite primary key, though it's close) and this jumbling is confusing to users that expect to see the data ordered based on the natural primary key.

  I've tried setting the `sort` metadata parameter to a list of column names, a tuple of column names, a quoted string of comma-separated column names, a quoted string of a tuple of column names...

  ```yaml
  databases:
    ferc1:
      tables:
        f1_edcfu_epda:
          sort: "(report_year, report_prd, respondent_id, spplmnt_num, row_number)"
  ```

  and they all give me server errors like:

  ```
  Cannot sort table by (report_year, report_prd, respondent_id, spplmnt_num, row_number)
  ```</div>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1653/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FQkDj
title: Adopt a code of conduct
state: closed
created_at: 2022-03-07T22:00:24Z
updated_at: 2022-03-07T22:19:35Z
closed_at: 2022-03-07T22:19:35Z
author_association: OWNER
body: This is long overdue, especially given the size of the project now.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1654/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FV5wr
title: query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data
state: open
created_at: 2022-03-09T00:56:40Z
updated_at: 2023-10-17T21:53:17Z
author_association: CONTRIBUTOR
type: issue
body:
  [this page](https://labordata.bunkum.us/opdr-8335ea3?sql=with+most_recent_lu+as+%28%0D%0A++select%0D%0A++++*%0D%0A++from%0D%0A++++%28%0D%0A++++++select%0D%0A++++++++*%0D%0A++++++from%0D%0A++++++++lm_data%0D%0A++++++order+by%0D%0A++++++++f_num%2C%0D%0A++++++++receive_date+desc%0D%0A++++%29+t%0D%0A++group+by%0D%0A++++f_num%0D%0A%29%0D%0Aselect%0D%0A++aff_abbr+%7C%7C+coalesce%28%27+local+%27+%7C%7C+desig_num%2C+%27+%27+%7C%7C+unit_name%29+as+abbr_local_name%2C%0D%0A++coalesce%28%0D%0A++++regexp_match%28%27%28.*%3F%29%28%2C%3F+AFL-CIO%24%29%27%2C+union_name%29%2C%0D%0A++++regexp_match%28%27%28.*%3F%29%28+IND%24%29%27%2C+union_name%29%2C%0D%0A++++union_name%0D%0A++%29+%7C%7C+coalesce%28%27+local+%27+%7C%7C+desig_num%2C+%27+%27+%7C%7C+unit_name%29+as+full_local_name%2C%0D%0A++*%0D%0Afrom%0D%0A++most_recent_lu%0D%0Awhere+%28desig_num+IS+NOT+NULL+OR+unit_name+IS+NOT+NULL%29+AND+desig_name+%21%3D+%27HQ%27%0D%0Alimit%0D%0A++5000+offset+0)

  is using about 400 mb in firefox 97 on mac os x. if you download the html for the page, it's about 11mb and if you get the csv for the data its about 1mb.

  it's using over a 1G on chrome 99.

  i found this because, i was trying to figure out why editing the SQL was getting very slow.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1655/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5FiLSg
title: Display autodoc type information more legibly
state: closed
created_at: 2022-03-11T15:58:20Z
updated_at: 2022-03-11T18:07:10Z
closed_at: 2022-03-11T18:07:10Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://sqlite-utils.datasette.io/en/3.25/reference.html#sqlite_utils.db.Table.insert looks like this at the moment:

  <img width="703" alt="image" src="https://user-images.githubusercontent.com/9599/157902622-368935a8-93f2-42e9-98ad-94a45c818e80.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/413/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Fiuhh
title: I forgot to include the changelog in the 3.25.1 release
state: closed
created_at: 2022-03-11T18:32:36Z
updated_at: 2022-03-11T18:40:39Z
closed_at: 2022-03-11T18:40:39Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I pushed a release for https://github.com/simonw/sqlite-utils/releases/tag/3.25.1 but forgot to include the release notes in `docs/changelog.rst`

  This means https://sqlite-utils.datasette.io/en/stable/changelog.html isn't showing them.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/414/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c40ZRDA
title: Update pytest requirement from <7.1.0,>=5.2.2 to >=5.2.2,<7.2.0
state: closed
created_at: 2022-03-14T13:11:53Z
updated_at: 2022-03-15T18:03:03Z
closed_at: 2022-03-15T18:03:02Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1656
type: pull
body:
  Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest/releases">pytest's releases</a>.</em></p>
  <blockquote>
  <h2>7.1.0</h2>
  <h1>pytest 7.1.0 (2022-03-13)</h1>
  <h2>Breaking Changes</h2>
  <ul>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/8838">#8838</a>: As per our policy, the following features have been deprecated in the 6.X series and are now
  removed:</p>
  <ul>
  <li><code>pytest._fillfuncargs</code> function.</li>
  <li><code>pytest_warning_captured</code> hook - use <code>pytest_warning_recorded</code> instead.</li>
  <li><code>-k -foobar</code> syntax - use <code>-k 'not foobar'</code> instead.</li>
  <li><code>-k foobar:</code> syntax.</li>
  <li><code>pytest.collect</code> module - import from <code>pytest</code> directly.</li>
  </ul>
  <p>For more information consult
  <a href="https://docs.pytest.org/en/latest/deprecations.html">Deprecations and Removals</a> in the docs.</p>
  </li>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9437">#9437</a>: Dropped support for Python 3.6, which reached <a href="https://devguide.python.org/#status-of-python-branches">end-of-life</a> at 2021-12-23.</p>
  </li>
  </ul>
  <h2>Improvements</h2>
  <ul>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/5192">#5192</a>: Fixed test output for some data types where <code>-v</code> would show less information.</p>
  <p>Also, when showing diffs for sequences, <code>-q</code> would produce full diffs instead of the expected diff.</p>
  </li>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9362">#9362</a>: pytest now avoids specialized assert formatting when it is detected that the default <code>__eq__</code> is overridden in <code>attrs</code> or <code>dataclasses</code>.</p>
  </li>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9536">#9536</a>: When <code>-vv</code> is given on command line, show skipping and xfail reasons in full instead of truncating them to fit the terminal width.</p>
  </li>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9644">#9644</a>: More information about the location of resources that led Python to raise <code>ResourceWarning</code>{.interpreted-text role=&quot;class&quot;} can now
  be obtained by enabling <code>tracemalloc</code>{.interpreted-text role=&quot;mod&quot;}.</p>
  <p>See <code>resource-warnings</code>{.interpreted-text role=&quot;ref&quot;} for more information.</p>
  </li>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9678">#9678</a>: More types are now accepted in the <code>ids</code> argument to <code>@pytest.mark.parametrize</code>.
  Previously only [str]{.title-ref}, [float]{.title-ref}, [int]{.title-ref} and [bool]{.title-ref} were accepted;
  now [bytes]{.title-ref}, [complex]{.title-ref}, [re.Pattern]{.title-ref}, [Enum]{.title-ref} and anything with a [__name__]{.title-ref} are also accepted.</p>
  </li>
  <li>
  <p><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9692">#9692</a>: <code>pytest.approx</code>{.interpreted-text role=&quot;func&quot;} now raises a <code>TypeError</code>{.interpreted-text role=&quot;class&quot;} when given an unordered sequence (such as <code>set</code>{.interpreted-text role=&quot;class&quot;}).</p>
  <p>Note that this implies that custom classes which only implement <code>__iter__</code> and <code>__len__</code> are no longer supported as they don't guarantee order.</p>
  </li>
  </ul>
  <h2>Bug Fixes</h2>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/8242">#8242</a>: The deprecation of raising <code>unittest.SkipTest</code>{.interpreted-text role=&quot;class&quot;} to skip collection of
  tests during the pytest collection phase is reverted - this is now a supported
  feature again.</li>
  <li><a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9493">#9493</a>: Symbolic link components are no longer resolved in conftest paths.
  This means that if a conftest appears twice in collection tree, using symlinks, it will be executed twice.</li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest/commit/1dbffcc0b4d822b87ad9f90595ffab6d9beee769"><code>1dbffcc</code></a> [pre-commit.ci] auto fixes from pre-commit.com hooks</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/d53a5fb37194faf63ee5d74606cc883138879bc4"><code>d53a5fb</code></a> Prepare release version 7.1.0</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/d306ec0a7e1711d936e5bb98352a430bff758eae"><code>d306ec0</code></a> Update upcoming trainings (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9744">#9744</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/3e4c14bfaa046bcb5b75903470accf83d93f01ce"><code>3e4c14b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9751">#9751</a> from fabianegli/main</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/7f924b13a50a05b8dc894418fa7faf779201e129"><code>7f924b1</code></a> Fix typo in deprecation documentation</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/4a8f8ada431974f2837260af3ed36299fd382814"><code>4a8f8ad</code></a> build(deps): Bump django from 4.0.2 to 4.0.3 in /testing/plugins_integration ...</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/c0fd2d883940f1292d5e8234803beaacd08315e6"><code>c0fd2d8</code></a> build(deps): Bump pytest-asyncio from 0.18.1 to 0.18.2 in /testing/plugins_in...</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/843e01824c257c3190792a9df430289c3abe349d"><code>843e018</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9732">#9732</a> from nicoddemus/9730-toml-failure</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/bc43d66b47b917d43a22e0c703ecfe4eea342263"><code>bc43d66</code></a> [automated] Update plugin list (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest/issues/9733">#9733</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest/commit/e38d1cac489e42f4bdbecbb50f9f25dc9c36c19f"><code>e38d1ca</code></a> Improve error message for malformed pyproject.toml files</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest/compare/5.2.2...7.1.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1656/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FrXWs
title: Tilde encoding: use ~ instead of - for dash-encoding
state: closed
created_at: 2022-03-14T22:55:17Z
updated_at: 2022-03-15T18:25:11Z
closed_at: 2022-03-15T18:01:58Z
author_association: OWNER
body: Refs #1439
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1657/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Fulod
title: Revert main to version that passes tests
state: closed
created_at: 2022-03-15T15:37:02Z
updated_at: 2022-03-19T04:04:50Z
closed_at: 2022-03-15T15:42:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I've made a real mess of this. I'm going to revert Datasette`main` back to the last commit that passed the tests and try this again in a branch.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1657#issuecomment-1068125636_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1658/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c40eW7C
title: Tilde encoding
state: closed
created_at: 2022-03-15T16:19:07Z
updated_at: 2022-03-15T18:01:58Z
closed_at: 2022-03-15T18:01:57Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1659
body: Refs #1657
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1659/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Fvv5v
title: Refactor and simplify Datasette routing and views
state: closed
created_at: 2022-03-15T19:56:56Z
updated_at: 2022-03-21T19:19:12Z
closed_at: 2022-03-21T19:19:01Z
author_association: OWNER
type: issue
state_reason: completed
body:
  While working on:
  - https://github.com/simonw/datasette/issues/1657
  - https://github.com/simonw/datasette/issues/1439

  It became very clear that the least maintainable part of Datasette at the moment is the way routing to the database, table and row views work - in particular the subclassing mechanism with BaseView and DataView, but also the complex variety of ways in which the URL routes capture different named regular expression groups.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1660/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FwjY-
title: Remove Hashed URL mode
state: closed
created_at: 2022-03-15T23:13:56Z
updated_at: 2022-03-19T00:37:37Z
closed_at: 2022-03-19T00:37:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It's now handled by a plugin instead:
  - #647
  - https://github.com/simonw/datasette-hashed-urls/issues/3

  https://github.com/simonw/datasette-hashed-urls

  Sub-tasks:

  - [x] Remove hashed URL mode implementation
  - [x] Update documentation
  - [x] Ensure `--setting hash_urls 1` shows a useful message

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1661/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FxGBd
title: [feature request] Publish to fully static website
state: closed
created_at: 2022-03-16T03:32:28Z
updated_at: 2022-03-19T00:42:23Z
closed_at: 2022-03-19T00:42:23Z
author_association: NONE
type: issue
state_reason: completed
body:
  It seems currently all datasette publish requires a real backend server which is able to query the database and send results back to the frontend. There are a few projects to on-demand download a portion of data from the  database from a sqlite lite database url, and present it directly to the user. These methods leverages web assembly under the hood. I think datasette is a perfect use case for this technology. Below are a few examples of querying sqlite database from frontend directly.

  * [Using sqlite3 as a notekeeping document graph with automatic reference indexing](https://epilys.github.io/bibliothecula/notekeeping.html)
  * [Hosting SQLite databases on Github Pages - (or any static file hoster) - phiresky's blog](https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/)
  * [Static torrent website with peer-to-peer queries over BitTorrent on 2M records](https://boredcaveman.xyz/post/0x2_static-torrent-website-p2p-queries.html)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1662/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5FxUBf
title: Document the internals that were used in datasette-hashed-urls
state: closed
created_at: 2022-03-16T05:17:08Z
updated_at: 2022-03-19T04:04:50Z
closed_at: 2022-03-17T21:32:38Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The https://github.com/simonw/datasette-hashed-urls used a couple of currently undocumented features:
  - `db.hash`
  - `Datasette(..., immutables=[...])`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1663/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5F1TIC
title: Convert with `--multi` and `--dry-run` flag does not work
state: closed
created_at: 2022-03-16T21:59:46Z
updated_at: 2022-03-21T04:18:24Z
closed_at: 2022-03-21T04:18:24Z
author_association: NONE
type: issue
state_reason: completed
body:
  It's not possible to combine `--multi` and `--dry-run` flag in the `convert` command.

  Let's first create a simple database from JSON string

  ```console
  $ echo '[{"foo": "abc"}]' | sqlite-utils insert demo.db demo -
  $ sqlite-utils query demo.db "SELECT * FROM demo"             
  [{"foo": "abc"}]
  ```

  and then try to convert the "foo" column with a static value "bar" (see docs [Converting a column into multiple columns](https://sqlite-utils.datasette.io/en/stable/cli.html#converting-a-column-into-multiple-columns))

  ```console
  $ sqlite-utils convert demo.db demo foo '{"foo": "bar"}' --multi --dry-run
  Traceback (most recent call last):
    File "/home/dotcs/anaconda3/envs/tools/bin/sqlite-utils", line 8, in <module>
      sys.exit(cli())
    File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/sqlite_utils/cli.py", line 2686, in convert
      for row in db.conn.execute(sql, where_args).fetchall():
  sqlite3.OperationalError: user-defined function raised exception
  ```

  But without the `--dry-run` flag it does work as expected:

  ```console
  $ sqlite-utils convert demo.db demo foo '{"foo": "bar"}' --multi
  $ sqlite-utils query demo.db "SELECT * FROM demo"               
  [{"foo": "bar"}]
  ```

  ```console
  $ sqlite-utils --version
  sqlite-utils, version 3.25.1
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/415/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c40oRq-
title: Remove hashed URL mode
state: closed
created_at: 2022-03-17T23:19:10Z
updated_at: 2022-03-19T00:12:04Z
closed_at: 2022-03-19T00:12:04Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1664
body: Refs #1661.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1664/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5F6uoo
title: Options for how `r.parsedate()` should handle invalid dates
state: closed
created_at: 2022-03-17T23:29:55Z
updated_at: 2022-05-03T21:36:49Z
closed_at: 2022-03-21T04:01:39Z
author_association: NONE
type: issue
state_reason: completed
body:
  Exceptions are normal expected behavior when typecasting an invalid format. However, r.parsedate() is really just re-formatting strings and keeping the type as text. So it may be better to print-and-pass on exception so the user can see a complete list of invalid values -- while also allowing for the parser to reformat the remaining valid values. 
  ```
  sqlite-utils convert idfpr.db license "Expiration Date" "r.parsedate(value)"
    [#######-----------------------------]   21%  00:01:57Traceback (most recent call last):
    File "/usr/local/lib/python3.9/dist-packages/sqlite_utils/db.py", line 2336, in convert_value
      return fn(v)
    File "<string>", line 2, in fn
    File "/usr/local/lib/python3.9/dist-packages/sqlite_utils/recipes.py", line 8, in parsedate
      parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).date().isoformat()
    File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", line 1374, in parse
      return DEFAULTPARSER.parse(timestr, **kwargs)
    File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", line 652, in parse
      raise ParserError("String does not contain a date: %s", timestr)
  dateutil.parser._parser.ParserError: String does not contain a date:   /  /    
  ```
  In this case, I had just one variation of an invalid date: '  /  /  '. But theoretically there could be many values that would have to be fixed one at a time with the current exception handling. 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/416/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c40q1eP
title: Pin setup-gcloud to v0 instead of master
state: closed
created_at: 2022-03-18T17:17:22Z
updated_at: 2022-03-23T19:31:10Z
closed_at: 2022-03-23T17:55:39Z
author_association: NONE
pull_request: simonw/datasette/pulls/1665
type: pull
body:
  setup-gcloud will be updating the branch name from master to main in
  a future release. Even though GitHub will establish redirects, this
  will break any GitHub Actions workflows that pin to master. This PR
  updates your GitHub Actions workflows to pin to v0, which is the
  recommended best practice.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1665/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5F_E1d
title: Refactor URL routing to enable testing
state: closed
created_at: 2022-03-19T03:52:29Z
updated_at: 2022-03-19T16:32:03Z
closed_at: 2022-03-19T16:32:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I ran into some bugs earlier with URL routing - having more robust testing around this (especially since they are defined using regular expressions) would be really useful.

  - A utility function that resolves a path against a list of reflexes and returns the match
  - Make the routes and regular expressions available from a private Datasette method
  - Add tests that exercise them

  Related:
  - #1660

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1666/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5F_nES
title: Make route matched pattern groups more consistent
state: closed
created_at: 2022-03-19T16:32:35Z
updated_at: 2022-03-19T20:37:42Z
closed_at: 2022-03-19T20:37:41Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > ... highlights how inconsistent the way the capturing works is. Especially `as_format` which can be `None` or `""` or `.json` or `json` or not used at all in the case of `TableView`.

  https://github.com/simonw/datasette/blob/764738dfcb16cd98b0987d443f59d5baa9d3c332/tests/test_routes.py#L12-L36

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1666#issuecomment-1073039670_

  Part of:
  - #1660

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1667/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5F_n1q
title: Introduce concept of a database `route`, separate from its name
state: closed
created_at: 2022-03-19T16:48:28Z
updated_at: 2022-03-20T16:43:16Z
closed_at: 2022-03-20T16:43:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Some issues came up in the new `datasette-hashed-urls` plugin relating to the way it renames databases on startup to achieve unique URLs that depend on the database SHA-256 content:

  - https://github.com/simonw/datasette-hashed-urls/issues/10
  - https://github.com/simonw/datasette-hashed-urls/issues/9
  - https://github.com/simonw/datasette-hashed-urls/issues/8

  All three of these could be addressed by making the "path" concept for a database (the `/foo` bit where it is served) work independently of the database's name, which would be used for default display and also as the alias when configuring cross-database aliases.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1668/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5F__4n
title: Release 0.61 alpha
state: closed
created_at: 2022-03-20T00:35:35Z
updated_at: 2022-03-20T01:24:36Z
closed_at: 2022-03-20T01:24:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I'm going to release this as a 0.61 alpha so I can more easily depend on it from `datasette-hashed-urls`.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1669/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GAEgQ
title: Ship Datasette 0.61
state: closed
created_at: 2022-03-20T02:47:54Z
updated_at: 2022-03-23T18:32:32Z
closed_at: 2022-03-23T18:32:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Let the alpha bake for a while, since #1668 is a big last-minute change.

  After shipping, release a new `datasette-hashed-urls` that depends on it, also this:

  - https://github.com/simonw/datasette-hashed-urls/issues/11

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1670/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GA9DT
state: open
created_at: 2022-03-20T19:17:24Z
updated_at: 2022-03-22T17:43:12Z
author_association: NONE
type: issue
title:
  Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply

body:
  I found a strange behavior, and I'm not sure if it's related to views and boolean values perhaps, or if there's something else weird going on here, but I'll provide an example that may help show what I'm seeing happen.

  ```bash
  #!/bin/bash

  echo "\"id\",\"expiration_date\"
  0,2018-01-04
  1,2019-01-05
  2,2020-01-06
  3,2021-01-07
  4,2022-01-08
  5,2023-01-09
  6,2024-01-10
  7,2025-01-11
  8,2026-01-12
  9,2027-01-13
  " > test.csv
  csvs-to-sqlite test.csv test.db
  sqlite-utils create-view --replace test.db test_view "select id, expiration_date, case when julianday('NOW') >= julianday(expiration_date) then 1 else 0 end as has_expired FROM test"
  ```

  ```bash
  datasette test.db
  ```
  ![image](https://user-images.githubusercontent.com/9308268/159178745-9c6152f7-eac6-4bf9-bef5-a2d63d3ee13f.png)

  ![image](https://user-images.githubusercontent.com/9308268/159178824-c8952137-270c-42a4-ad1c-f6ad2c51e499.png)

  ![image](https://user-images.githubusercontent.com/9308268/159178877-23e00b36-443a-43ef-83e5-e0bdddd3fdcd.png)

  ![image](https://user-images.githubusercontent.com/9308268/159178918-65922cc7-2514-4735-a72d-4904b99976d4.png)

  Thanks again and let me know if you want me to provide anything else!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1671/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GBHS4
title: Refactor CSV handling code out of DataView
state: open
created_at: 2022-03-20T21:47:00Z
updated_at: 2022-03-20T21:52:39Z
author_association: OWNER
type: issue
body:
  > I think the way to get rid of most of the remaining complexity in `DataView` is to refactor how CSV stuff works - pulling it in line with other export factors and extracting the streaming mechanism. Opening a fresh issue for that.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1073355032_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1672/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GBKCX
title: Streaming CSV spends a lot of time in `table_column_details`
state: open
created_at: 2022-03-20T22:25:28Z
updated_at: 2022-03-20T22:34:06Z
author_association: OWNER
type: issue
body:
  At least I think it does. I tried running `py-spy top -p $PID` against a Datasette process that was trying to do:

      datasette covid.db --get '/covid/ny_times_us_counties.csv?_size=10&_stream=on'

  While investigating:
  - #1355

  And spotted this:
  ```
  datasette covid.db --get /covid/ny_times_us_counties.csv?_size=10&_stream=on' (python v3.10.2)
  Total Samples 5800
  GIL: 71.00%, Active: 98.00%, Threads: 4

    %Own   %Total  OwnTime  TotalTime  Function (filename:line)                                                                                                                                            
    8.00%   8.00%    4.32s     4.38s   sql_operation_in_thread (datasette/database.py:212)
    5.00%   5.00%    3.77s     3.93s   table_column_details (datasette/utils/__init__.py:614)
    6.00%   6.00%    3.72s     3.72s   _worker (concurrent/futures/thread.py:81)
    7.00%   7.00%    2.98s     2.98s   _read_from_self (asyncio/selector_events.py:120)
    5.00%   6.00%    2.35s     2.49s   detect_fts (datasette/utils/__init__.py:571)
    4.00%   4.00%    1.34s     1.34s   _write_to_self (asyncio/selector_events.py:140)
  ```
  Relevant code: https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/__init__.py#L609-L625

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1673/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GBMNn
title: Tweak design of /.json
state: open
created_at: 2022-03-20T22:58:01Z
updated_at: 2022-03-20T22:58:40Z
author_association: OWNER
type: issue
body:
  https://latest.datasette.io/.json

  Currently:
  ```json
  {
    "_memory": {
      "name": "_memory",
      "hash": null,
      "color": "a6c7b9",
      "path": "/_memory",
      "tables_and_views_truncated": [],
      "tables_and_views_more": false,
      "tables_count": 0,
      "table_rows_sum": 0,
      "show_table_row_counts": false,
      "hidden_table_rows_sum": 0,
      "hidden_tables_count": 0,
      "views_count": 0,
      "private": false
    },
    "fixtures": {
      "name": "fixtures",
      "hash": "645005884646eb941c89997fbd1c0dd6be517cb1b493df9816ae497c0c5afbaa",
      "color": "645005",
      "path": "/fixtures",
      "tables_and_views_truncated": [
        {
          "name": "compound_three_primary_keys",
          "columns": [
            "pk1",
            "pk2",
            "pk3",
            "content"
          ],
          "primary_keys": [
            "pk1",
            "pk2",
            "pk3"
          ],
          "count": 1001,
          "hidden": false,
          "fts_table": null,
          "num_relationships_for_sorting": 0,
          "private": false
        },
  ```
  As-of this issue the `"path"` key is confusing, it doesn't match what https://latest.datasette.io/-/databases returns:

  - #1668

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1674/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GEvjF
title: Extract out `check_permissions()` from `BaseView
state: closed
created_at: 2022-03-21T16:39:46Z
updated_at: 2022-03-21T17:14:31Z
closed_at: 2022-03-21T17:13:21Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I'm going to refactor this stuff out and document it so it can be easily used by plugins:

  https://github.com/simonw/datasette/blob/4a4164b81191dec35e423486a208b05a9edc65e4/datasette/views/base.py#L69-L103

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1074136176_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1675/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GE5tW
title: Reconsider ensure_permissions() logic, can it be less confusing?
state: open
created_at: 2022-03-21T17:14:57Z
updated_at: 2022-12-02T01:23:40Z
author_association: OWNER
type: issue
body:
  > Updated documentation: https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/docs/internals.rst#await-ensure_permissionsactor-permissions
  >
  >> This method allows multiple permissions to be checked at onced. It raises a `datasette.Forbidden` exception if any of the checks are denied before one of them is explicitly granted.
  >> 
  >> This is useful when you need to check multiple permissions at once. For example, an actor should be able to view a table if either one of the following checks returns `True` or not a single one of them returns `False`:
  >
  > That's pretty hard to understand! I'm going to open a separate issue to reconsider if this is a useful enough abstraction given how confusing it is.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1675#issuecomment-1074177827_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1676/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GE6uo
title: Remove `check_permission()` from `BaseView`
state: closed
created_at: 2022-03-21T17:18:18Z
updated_at: 2022-03-21T18:45:04Z
closed_at: 2022-03-21T18:45:03Z
author_association: OWNER
body:
  Follow-on from:
  - #1675

  Refs:
  - #1660

type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1677/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GFACU
title: Make `check_visibility()` a documented API
state: closed
created_at: 2022-03-21T17:30:34Z
updated_at: 2022-03-21T19:04:03Z
closed_at: 2022-03-21T19:01:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Spotted this while working on:
  - #1677

  https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/datasette/utils/__init__.py#L1005-L1021

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1678/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5GFHCO
title: insert fails on JSONL with whitespace
state: closed
created_at: 2022-03-21T17:58:14Z
updated_at: 2022-03-25T21:19:06Z
closed_at: 2022-03-25T21:17:13Z
author_association: NONE
type: issue
state_reason: completed
body:
  Any JSON that is newline-delimited and has whitespace (newlines) between the start of a JSON object and an attribute fails due to a parse error.

  e.g. given the valid JSONL:

  ```{
    "attribute": "value"
  }
  {
    "attribute": "value2"
  }
  ```

  I would expect that `sqlite-utils insert --nl my.db mytable file.jsonl` would properly import the data into `mytable`. However, the following error is thrown instead:

  `json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)`

  It makes sense that since the file is intended to be newline separated, the thing being parsed is "{" (which obviously fails), however the default newline-separated output of `jq` isn't compact. Using `jq -c` avoids this problem, but the fix is unintuitive and undocumented.

  Proposed solutions:
  1. Default to a "loose" newline-separated parse; this could be implemented internally as [the equivalent of] a `jq -c` filter ahead of the insert step.
  2. Catch the JSONDecodeError (or pre-empt it in the case of a record === "{\n") and give the user a "it looks like your json isn't _actually_ newline-delimited; try running it through `jq -c` instead" error message.

  It might just have been too early in the morning when I was playing with this, but running pipes of data through sqlite-utils without the 'knack' of it led to some false starts.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/417/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GFh-G
title: Research: how much overhead does the n=1 time limit have?
state: closed
created_at: 2022-03-21T19:27:46Z
updated_at: 2022-03-21T21:55:57Z
closed_at: 2022-03-21T21:55:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/simonw/datasette/blob/1a7750eb29fd15dd2eea3b9f6e33028ce441b143/datasette/utils/__init__.py#L181-L200

  ```python
  @contextmanager
  def sqlite_timelimit(conn, ms):
      deadline = time.perf_counter() + (ms / 1000)
      # n is the number of SQLite virtual machine instructions that will be
      # executed between each check. It's hard to know what to pick here.
      # After some experimentation, I've decided to go with 1000 by default and
      # 1 for time limits that are less than 50ms
      n = 1000
      if ms < 50:
          n = 1

      def handler():
          if time.perf_counter() >= deadline:
              return 1

      conn.set_progress_handler(handler, n)
      try:
          yield
      finally:
          conn.set_progress_handler(None, n)
  ```
  How often do I set a time limit of 50 or less? How much slower does it go thanks to this code?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1679/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GFrty
title: Consider simplifying permissions for 1.0
state: open
created_at: 2022-03-21T20:17:29Z
updated_at: 2022-03-21T20:17:29Z
author_association: OWNER
type: issue
body:
  Permission checks right now can express one of three opinions:

  - `False` means "so not grant this permisson"
  - `True` means "grant this permission"
  - `None` means "I have no opinion"

  But... there's also a concept of a "default" for a given permission check, which might be `False` or `True`.

  I worry this is too complicated. Could this be simplified before 1.0? In particular the default concept.

  See also:
  - #1676 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1680/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODFdgUs5GKICZ
title: Store commit parents
state: closed
created_at: 2022-03-22T17:06:48Z
updated_at: 2022-04-22T12:44:04Z
closed_at: 2022-04-22T12:44:04Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi @simonw 👋 

  Currently, stored commit data doesn't quite give me the information I'm needing... 

  Committer date and author date are not 100% reliable for dividing a commit history up by release or branch. 
  A PR created before a release but merged after can have earlier dates… — this can be quite frustrating if you're trying to pin down commits for a release: _It should be there!_, but then isn't. (This gets worse using release branches.) 

  Would you be open to adding the `sha` of a `parent` of a commit to the commit table? (As an FK? 🤔 — likely not feasible.)

  It's part of the [response body](https://docs.github.com/en/rest/reference/commits#get-a-commit): 

  ```
    "parents": [
      {
        "url": "https://api.github.com/repos/octocat/Hello-World/commits/6dcb09b5b57875f334f61aebed695e2e4193db5e",
        "sha": "6dcb09b5b57875f334f61aebed695e2e4193db5e"
      }
    ],
  ```

  I think this list should only have a single entry. (🤔 — not sure why it's a list then...)

  With this it would be possible to build/reconstruct a chain of commits from the history, that I don't **think** is available as yet (unless you know a better way). 

  It is certainly possible to get sequential lists of commits out of git directly, so the same would be possible combining tools, but wondering if a single tool could do it. 

  What do you think? 
  Thanks! 🏅 

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/71/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GKSWB
title: Potential bug in numeric handling where_clause for filters
state: open
created_at: 2022-03-22T17:43:50Z
updated_at: 2022-03-22T17:49:09Z
author_association: OWNER
type: issue
body:
  > Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like `>`:
  >
  > https://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/datasette/filters.py#L203-L212
  > 
  > Though... it looks like there's a bug in that? It doesn't account for `float` values - `"3.5".isdigit()` return `False` - probably for the best, because `int(3.5)` would break that value anyway.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1681/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5GPdLa
title: Add generated files to .gitignore
state: closed
created_at: 2022-03-23T17:48:12Z
updated_at: 2022-03-24T21:01:44Z
closed_at: 2022-03-24T21:01:44Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  I end up with these in my local directory:

  	.hypothesis/
  	Pipfile
  	Pipfile.lock
  	pyproject.toml

  Might as well gitignore them.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/418/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM405rPe
title: Ignore common generated files
state: closed
created_at: 2022-03-23T18:06:22Z
updated_at: 2022-03-24T21:01:44Z
closed_at: 2022-03-24T21:01:44Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/419
type: pull
body:
  Closes #418 

  This adds four files to `.gitignore`:

  	.hypothesis/
  	Pipfile
  	Pipfile.lock
  	pyproject.toml

  Those are all generated in the course of development and testing.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/419/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GPs-p
title: SQL queries against databases with different routes are broken
state: closed
created_at: 2022-03-23T18:42:57Z
updated_at: 2022-03-23T18:48:16Z
closed_at: 2022-03-23T18:48:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  500 error on https://datasette-hashed-urls-preview.vercel.app/fixtures-09f8f95?sql=select+*+from+facetable

  Here's the trace:
  ```
    File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py", line 54, in data
      return await QueryView(self.ds).data(
    File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py", line 232, in data
      self.ds.get_database(database), sql
    File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/app.py", line 401, in get_database
      return self.databases[name]
  KeyError: 'fixtures-aa7318b'
  ```
  It looks like this is a Datasette bug, which is frustrating because I just shipped Datasette 0.61 five minutes ago!

  _Originally posted by @simonw in https://github.com/simonw/datasette-hashed-urls/issues/13#issuecomment-1076693667_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1682/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5GPzKu
title: Document how to use a `--convert` function that runs initialization code first
state: closed
created_at: 2022-03-23T19:07:36Z
updated_at: 2022-08-28T11:34:37Z
closed_at: 2022-03-25T20:07:33Z
author_association: NONE
type: issue
state_reason: completed
body:
  When I have an insert command with transform like this:

  ```
  cat items.json | jq '.data' | sqlite-utils insert listings.db listings - --convert '
  d = enchant.Dict("en_US")
  row["is_dictionary_word"] = d.check(row["name"])
  '  --import=enchant --ignore
  ```

  I noticed as the number of rows increases the operation becomes quite slow, likely due to the creation of the `d = enchant.Dict("en_US")` object for each row. Is there a way to share that instance `d` between transform function calls, like a shared context?

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/420/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GVEe-
title: allow_facet: False should be respected by column cog menu
state: closed
created_at: 2022-03-24T19:05:06Z
updated_at: 2022-03-24T19:16:36Z
closed_at: 2022-03-24T19:16:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The column cog menu currently shows "Facet by this" even if faceting is disabled for the Datasette instance.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1683/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GVVd3
title: Mechanism for disabling faceting on large tables only
state: open
created_at: 2022-03-24T20:06:11Z
updated_at: 2022-03-24T20:13:19Z
author_association: OWNER
type: issue
body:
  Forest turned off faceting on https://labordata.bunkum.us/ because it was causing performance problems on some of the huge tables - but it would be nice if it could still be an option on smaller tables such as https://labordata.bunkum.us/voluntary_recognitions-4421085/voluntary_recognitions

  One option: a new setting that automatically disables faceting (and facet suggestion) for tables that have either more than X rows or that are so big that the count could not be completed within the time limit.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1684/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5GW-YQ
title: "Error: near "(": syntax error" when using sqlite-utils indexes CLI
state: closed
created_at: 2022-03-25T07:12:51Z
updated_at: 2022-04-13T22:41:59Z
closed_at: 2022-04-13T22:41:59Z
author_association: NONE
type: issue
state_reason: completed
body:
  This bug relates to https://github.com/simonw/sqlite-utils/issues/408#issuecomment-1066139147

  **New error when using CLI: "sqlite-utils indexes global.db --table"**

  ```
  (app-root) sqlite-utils indexes global.db --table
  Error: near "(": syntax error
  (app-root) sqlite-utils --version
  sqlite-utils, version 3.25.1
  (app-root) sqlite3 --version
  3.36.0 2021-06-18 18:36:39
  (app-root) python --version
  Python 3.8.11
  ```


  Dockerfile
  ```
  FROM centos/python-38-centos7

  USER root

  RUN yum update -y
  RUN yum upgrade -y


  # epel
  RUN yum -y install epel-release && yum clean all

  # SQLite
  RUN yum -y install zlib-devel geos geos-devel proj proj-devel freexl freexl-devel libxml2-devel 

  WORKDIR /build/
  COPY sqlite-autoconf-3360000.tar.gz ./
  RUN tar -zxf sqlite-autoconf-3360000.tar.gz
  WORKDIR /build/sqlite-autoconf-3360000
  RUN ./configure
  RUN make
  RUN make install

  # 
  RUN /opt/app-root/bin/python3.8 -m pip install --upgrade pip
  RUN pip install sqlite-utils
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/421/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c41BFWj
title: Update jinja2 requirement from <3.1.0,>=2.10.3 to >=2.10.3,<3.2.0
state: closed
created_at: 2022-03-25T13:12:13Z
updated_at: 2022-09-05T18:36:49Z
closed_at: 2022-09-05T18:36:48Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1685
type: pull
body:
  Updates the requirements on [jinja2](https://github.com/pallets/jinja) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/pallets/jinja/releases">jinja2's releases</a>.</em></p>
  <blockquote>
  <h2>3.1.0</h2>
  <p>This is a feature release, which includes new features and removes previously deprecated features. The 3.1.x branch is now the supported bugfix branch, the 3.0.x branch has become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all dependencies and control upgrades. We also encourage upgrading to MarkupSafe 2.1.1, the latest version at this time.</p>
  <ul>
  <li>Changes: <a href="https://jinja.palletsprojects.com/en/3.1.x/changes/#version-3-1-0">https://jinja.palletsprojects.com/en/3.1.x/changes/#version-3-1-0</a></li>
  <li>Milestone: <a href="https://github.com/pallets/jinja/milestone/8?closed=1">https://github.com/pallets/jinja/milestone/8?closed=1</a></li>
  <li>MarkupSafe changes: <a href="https://markupsafe.palletsprojects.com/en/2.1.x/changes/#version-2-1-1">https://markupsafe.palletsprojects.com/en/2.1.x/changes/#version-2-1-1</a></li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pallets/jinja/blob/main/CHANGES.rst">jinja2's changelog</a>.</em></p>
  <blockquote>
  <h2>Version 3.1.0</h2>
  <p>Released 2022-03-24</p>
  <ul>
  <li>
  <p>Drop support for Python 3.6. :pr:<code>1534</code></p>
  </li>
  <li>
  <p>Remove previously deprecated code. :pr:<code>1544</code></p>
  <ul>
  <li><code>WithExtension</code> and <code>AutoEscapeExtension</code> are built-in now.</li>
  <li><code>contextfilter</code> and <code>contextfunction</code> are replaced by
  <code>pass_context</code>. <code>evalcontextfilter</code> and
  <code>evalcontextfunction</code> are replaced by <code>pass_eval_context</code>.
  <code>environmentfilter</code> and <code>environmentfunction</code> are replaced
  by <code>pass_environment</code>.</li>
  <li><code>Markup</code> and <code>escape</code> should be imported from MarkupSafe.</li>
  <li>Compiled templates from very old Jinja versions may need to be
  recompiled.</li>
  <li>Legacy resolve mode for <code>Context</code> subclasses is no longer
  supported. Override <code>resolve_or_missing</code> instead of
  <code>resolve</code>.</li>
  <li><code>unicode_urlencode</code> is renamed to <code>url_quote</code>.</li>
  </ul>
  </li>
  <li>
  <p>Add support for native types in macros. :issue:<code>1510</code></p>
  </li>
  <li>
  <p>The <code>{% trans %}</code> tag can use <code>pgettext</code> and <code>npgettext</code> by
  passing a context string as the first token in the tag, like
  <code>{% trans &quot;title&quot; %}</code>. :issue:<code>1430</code></p>
  </li>
  <li>
  <p>Update valid identifier characters from Python 3.6 to 3.7.
  :pr:<code>1571</code></p>
  </li>
  <li>
  <p>Filters and tests decorated with <code>@async_variant</code> are pickleable.
  :pr:<code>1612</code></p>
  </li>
  <li>
  <p>Add <code>items</code> filter. :issue:<code>1561</code></p>
  </li>
  <li>
  <p>Subscriptions (<code>[0]</code>, etc.) can be used after filters, tests, and
  calls when the environment is in async mode. :issue:<code>1573</code></p>
  </li>
  <li>
  <p>The <code>groupby</code> filter is case-insensitive by default, matching
  other comparison filters. Added the <code>case_sensitive</code> parameter to
  control this. :issue:<code>1463</code></p>
  </li>
  <li>
  <p>Windows drive-relative path segments in template names will not
  result in <code>FileSystemLoader</code> and <code>PackageLoader</code> loading from
  drive-relative paths. :pr:<code>1621</code></p>
  </li>
  </ul>
  <h2>Version 3.0.3</h2>
  <p>Released 2021-11-09</p>
  <ul>
  <li>Fix traceback rewriting internals for Python 3.10 and 3.11.
  :issue:<code>1535</code></li>
  <li>Fix how the native environment treats leading and trailing spaces
  when parsing values on Python 3.10. :pr:<code>1537</code></li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pallets/jinja/commit/84c0e2cf4e58655032930aeb200b979e3c6f5f32"><code>84c0e2c</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/jinja/issues/1625">#1625</a> from pallets/release-3.1.0</li>
  <li><a href="https://github.com/pallets/jinja/commit/7b0c47fe1cd836e19242738fc53f40855c1d7971"><code>7b0c47f</code></a> release version 3.1.0</li>
  <li><a href="https://github.com/pallets/jinja/commit/ede0f98725786911031e33b70fcfbb802ae0c825"><code>ede0f98</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/jinja/issues/1621">#1621</a> from pallets/template-safe-path</li>
  <li><a href="https://github.com/pallets/jinja/commit/040088a39527825c03107f17695d13a1752cc2f3"><code>040088a</code></a> use <code>posixpath.join</code> when loading template names</li>
  <li><a href="https://github.com/pallets/jinja/commit/a2920752fd111c2d52c88eb57487667b3cf0ea7b"><code>a292075</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/jinja/issues/1620">#1620</a> from janfilips/patch-1</li>
  <li><a href="https://github.com/pallets/jinja/commit/6e4df02773465d7a80c75cad93289d24f70bfa74"><code>6e4df02</code></a> Fix formatting in tricks.rst</li>
  <li><a href="https://github.com/pallets/jinja/commit/3a050b122ed8c7b3b6bf0116cb91bd9d5eac7d8f"><code>3a050b1</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/jinja/issues/1617">#1617</a> from pallets/docs-prose</li>
  <li><a href="https://github.com/pallets/jinja/commit/4b63cd8f763f58fa6e5bb43a9af0bc77e93bc987"><code>4b63cd8</code></a> rewrite include statement section</li>
  <li><a href="https://github.com/pallets/jinja/commit/a98d482186935cc535873ed3a9002bca837cc2e5"><code>a98d482</code></a> clean up faq, move technical discussions</li>
  <li><a href="https://github.com/pallets/jinja/commit/9de99f85603e3546055764fc13b1e077e1d2a7b3"><code>9de99f8</code></a> clean up engine comparisons</li>
  <li>Additional commits viewable in <a href="https://github.com/pallets/jinja/compare/2.10.3...3.1.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1685/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GZTLd
title: heroku bails if app name specifed in datasette publish is the same as existing app
state: open
created_at: 2022-03-25T17:10:34Z
updated_at: 2022-03-25T17:10:34Z
author_association: NONE
type: issue
body:
  Seem that `heroku` does not accept an app overwrite triggered by specifying the app name using `datasette publish`, as below:

  ```
  datasette publish heroku some.db --name "jazzy-name" 
  ```

  The resulting error has the below traceback:

  ```
  Creating jazzy-name... !
   ▸    Name jazzy-name is already taken
  Traceback (most recent call last):
    File "/opt/homebrew/bin/datasette", line 33, in <module>
      sys.exit(load_entry_point('datasette==0.60.1', 'console_scripts', 'datasette')())
    File "/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/datasette/publish/heroku.py", line 127, in heroku
      create_output = check_output(cmd).decode("utf8")
    File "/opt/homebrew/Cellar/python@3.10/3.10.2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 420, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/opt/homebrew/Cellar/python@3.10/3.10.2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['heroku', 'apps:create', 'jazzy-name', '--json']' returned non-zero exit status 1.
  ```

  It's a solid failsafe, but does `datasette publish` have a way to force an overwrite?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1686/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5GaDvN
title: Reconsider not running convert functions against null values
state: open
created_at: 2022-03-25T20:22:40Z
updated_at: 2022-03-25T20:23:21Z
author_association: OWNER
type: issue
body:
  I just got caught out by the fact that `None` values are not processed by the `.convert()` mechanism https://github.com/simonw/sqlite-utils/blob/0b7b80bd40fe86e4d66a04c9f607d94991c45c0b/sqlite_utils/db.py#L2504-L2510

  I had run this code while working on #420 and I wasn't sure why it didn't work:

  ```
  $ sqlite-utils add-column content.db articles score float
  $ sqlite-utils convert content.db articles score '
  import random
  random.seed(10)

  def convert(value):
      global random
      return random.random()
  '
  ```
  The reason it didn't work is that the newly added `score` column was full of `null` values.

  I fixed it by doing this instead:

      $ sqlite-utils add-column content.db articles score float --not-null-default 1.0

  But this indicates to me that the design of `convert()` here may be incorrect.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/422/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Gai9L
title: Make show_json.html or a similar mechanism stable for plugins
state: open
created_at: 2022-03-25T23:42:45Z
updated_at: 2022-03-25T23:42:45Z
author_association: OWNER
type: issue
body:
  I used `show_json.html` in the new `datasette-packages` plugin, which means it will break if that template changes:
  - https://github.com/simonw/datasette-packages/issues/3

  It would be useful if it (or something like it) was documented and stable for plugins to use.

  Also relevant:
  - #878

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1687/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Gazsw
state: closed
created_at: 2022-03-26T01:17:44Z
updated_at: 2022-03-27T01:01:14Z
closed_at: 2022-03-26T21:34:47Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
title:
  [plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins?

body:
  I'm trying to make a small plugin that depends on static assets, by following the guide [here](https://docs.datasette.io/en/stable/writing_plugins.html#writing-one-off-plugins). I made a `plugins/` directory with `datasette_nteract_data_explorer.py`. 

  I am trying to follow the example of `datasette_vega`, and serving static assets. I created a `statics/` directory within `plugins/` to serve my JS and CSS.

  https://github.com/simonw/datasette-vega/blob/00de059ab1ef77394ba9f9547abfacf966c479c4/datasette_vega/__init__.py#L13

  Unfortunately, datasette doesn't seem to be able to find my assets.

  Input:

  ```bash
  datasette ~/Library/Safari/History.db  --plugins-dir=plugins/
  ```
  ![Image 2022-03-25 at 9 18 17 PM](https://user-images.githubusercontent.com/9020979/160218979-a3ff474b-5255-4a76-85d1-6f90ab2e3b44.jpg)

  Output:

  ![Image 2022-03-25 at 9 11 00 PM](https://user-images.githubusercontent.com/9020979/160218733-ca5144cf-f23f-43d8-a8d3-e3a871e57f3a.jpg)




  I suspect this issue might go away if I move away from "one-off" plugin mode, but it's been a while since I created a new python package so I'm not sure how much work there is to go between "one off" and "packaged for PyPI". I'd like to try to avoid needing to repackage a new `tar.gz` file and or reinstall my library repeatedly when developing new python code.

  1. Is there a way to serve a static assets when using the `plugins/` directory method instead of installing plugins as a new python package?
  2. If not, is there a way I can work on developing a plugin without creating and repackaging tar.gz files after every change, or is that the recommended path? 

  Thanks for your help!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1688/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GdOPQ
title: datasette.add_message() documentation is incorrect
state: closed
created_at: 2022-03-26T20:49:42Z
updated_at: 2022-03-26T21:35:57Z
closed_at: 2022-03-26T20:51:21Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://docs.datasette.io/en/0.61.1/internals.html#add-message-request-message-message-type-datasette-info says:

  `.add_message(request, message, message_type=datasette.INFO)`

  But in the code it's:

  https://github.com/simonw/datasette/blob/6b99e4a66ba0ed8fca8ee41ceb7206928b60d5d1/datasette/app.py#L582

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1689/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Gdg1B
title: Idea: `datasette.set_actor_cookie(response, actor)`
state: open
created_at: 2022-03-26T22:41:52Z
updated_at: 2022-03-26T22:43:00Z
author_association: OWNER
type: issue
body:
  I just wrote this code in a plugin and it felt like it could benefit from an abstraction: https://github.com/simonw/datasette-auth0/blob/152e6eb21e96e9b73bd9c205f9749a1297d0ef0b/datasette_auth0/__init__.py#L79-L92

  ```python
      redirect_response = Response.redirect("/")
      expires_at = int(time.time()) + (24 * 60 * 60)
      redirect_response.set_cookie(
          "ds_actor",
          datasette.sign(
              {
                  "a": profile_response.json(),
                  "e": baseconv.base62.encode(expires_at),
              },
              "actor",
          ),
      )
      return redirect_response
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1690/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GdhWX
title: Bug in pytest-httpx example
state: closed
created_at: 2022-03-26T22:45:30Z
updated_at: 2022-03-26T22:46:09Z
closed_at: 2022-03-26T22:46:09Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://docs.datasette.io/en/0.61.1/testing_plugins.html#testing-outbound-http-calls-with-pytest-httpx says:

  ```python
  async def test_outbound_http_call(httpx_mock):
      httpx_mock.add_response(
          url='https://www.example.com/',
          data='Hello world',
      )
  ```
  That's wrong - `data=` should be `text=`.

  https://github.com/Colin-b/pytest_httpx/blob/v0.20.0/README.md#reply-with-custom-body

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1691/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Gd1sL
title: [plugins][feature request]: Support additional script tag attributes when loading custom JS
state: open
created_at: 2022-03-27T01:16:03Z
updated_at: 2022-03-30T06:14:51Z
author_association: CONTRIBUTOR
type: issue
body:
  ## Motivation

  - The build system for my new [plugin](https://github.com/hydrosquall/datasette-nteract-data-explorer) has two output JS files, one for browsers that support ES modules, one for browsers that don't. At present, I'm only passing one of them into Datasette.
  - I'd like to specify the non-es-module script as a fallback for older browsers. I don't want to load it by default, because browsers will only need one, and it's heavy, so for now I'm only supporting modern browsers. 

  To be able to support legacy browsers without slowing down users with modern browsers, I would like to be able to set additional HTML attributes on the tag fallback script, `nomodule` and `defer`. My injected scripts should look something like this:

  ```html
  <script type="module" src="/index.my-es-module-bundle.js"></script>
  <script src="/index.my-legacy-fallback-bundle.js" nomodule="" defer></script>
  ```

  ## Proposal

  To achieve this, I propose additional optional properties to the API accepted by the `extra_js_urls` hook and custom JS field the `metadata.json` [described here](https://docs.datasette.io/en/stable/custom_templates.html#custom-css-and-javascript). 

  Under this API, I'd write something like this to get the above HTML rendered in Datasette.

  ```json
  {
      "extra_js_urls": [
          {
              "url": "/index.my-es-module-bundle.js",
              "module": true,
          },
          {
              "url": "/index.my-legacy-fallback-bundle.js",
              "nomodule": "",
              "defer": true
          }
      ]
  }
  ```

  ## Resources

  - [MDN on the script tag](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script)
    - There may be other properties that could be added that are potentially valuable, like `async` or `referrerpolicy`, but I don't have an immediate need for those.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1692/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c41OrSL
title: Bump black from 22.1.0 to 22.3.0
state: closed
created_at: 2022-03-29T13:11:09Z
updated_at: 2022-06-28T13:11:38Z
closed_at: 2022-06-28T13:11:36Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1693
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 22.1.0 to 22.3.0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>22.3.0</h2>
  <h3>Preview style</h3>
  <ul>
  <li>Code cell separators <code>#%%</code> are now standardised to <code># %%</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2919">#2919</a>)</li>
  <li>Remove unnecessary parentheses from <code>except</code> statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2939">#2939</a>)</li>
  <li>Remove unnecessary parentheses from tuple unpacking in <code>for</code> loops (<a href="https://github-redirect.dependabot.com/psf/black/issues/2945">#2945</a>)</li>
  <li>Avoid magic-trailing-comma in single-element subscripts (<a href="https://github-redirect.dependabot.com/psf/black/issues/2942">#2942</a>)</li>
  </ul>
  <h3>Configuration</h3>
  <ul>
  <li>Do not format <code>__pypackages__</code> directories by default (<a href="https://github-redirect.dependabot.com/psf/black/issues/2836">#2836</a>)</li>
  <li>Add support for specifying stable version with <code>--required-version</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2832">#2832</a>).</li>
  <li>Avoid crashing when the user has no homedir (<a href="https://github-redirect.dependabot.com/psf/black/issues/2814">#2814</a>)</li>
  <li>Avoid crashing when md5 is not available (<a href="https://github-redirect.dependabot.com/psf/black/issues/2905">#2905</a>)</li>
  <li>Fix handling of directory junctions on Windows (<a href="https://github-redirect.dependabot.com/psf/black/issues/2904">#2904</a>)</li>
  </ul>
  <h3>Documentation</h3>
  <ul>
  <li>Update pylint config documentation (<a href="https://github-redirect.dependabot.com/psf/black/issues/2931">#2931</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Move test to disable plugin in Vim/Neovim, which speeds up loading (<a href="https://github-redirect.dependabot.com/psf/black/issues/2896">#2896</a>)</li>
  </ul>
  <h3>Output</h3>
  <ul>
  <li>In verbose, mode, log when <em>Black</em> is using user-level config (<a href="https://github-redirect.dependabot.com/psf/black/issues/2861">#2861</a>)</li>
  </ul>
  <h3>Packaging</h3>
  <ul>
  <li>Fix Black to work with Click 8.1.0 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2966">#2966</a>)</li>
  <li>On Python 3.11 and newer, use the standard library's <code>tomllib</code> instead of <code>tomli</code>
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/2903">#2903</a>)</li>
  <li><code>black-primer</code>, the deprecated internal devtool, has been removed and copied to a
  <a href="https://github.com/cooperlees/black-primer">separate repository</a> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2924">#2924</a>)</li>
  </ul>
  <h3>Parser</h3>
  <ul>
  <li>Black can now parse starred expressions in the target of <code>for</code> and <code>async for</code>
  statements, e.g <code>for item in *items_1, *items_2: pass</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2879">#2879</a>).</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>22.3.0</h2>
  <h3>Preview style</h3>
  <ul>
  <li>Code cell separators <code>#%%</code> are now standardised to <code># %%</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2919">#2919</a>)</li>
  <li>Remove unnecessary parentheses from <code>except</code> statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2939">#2939</a>)</li>
  <li>Remove unnecessary parentheses from tuple unpacking in <code>for</code> loops (<a href="https://github-redirect.dependabot.com/psf/black/issues/2945">#2945</a>)</li>
  <li>Avoid magic-trailing-comma in single-element subscripts (<a href="https://github-redirect.dependabot.com/psf/black/issues/2942">#2942</a>)</li>
  </ul>
  <h3>Configuration</h3>
  <ul>
  <li>Do not format <code>__pypackages__</code> directories by default (<a href="https://github-redirect.dependabot.com/psf/black/issues/2836">#2836</a>)</li>
  <li>Add support for specifying stable version with <code>--required-version</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2832">#2832</a>).</li>
  <li>Avoid crashing when the user has no homedir (<a href="https://github-redirect.dependabot.com/psf/black/issues/2814">#2814</a>)</li>
  <li>Avoid crashing when md5 is not available (<a href="https://github-redirect.dependabot.com/psf/black/issues/2905">#2905</a>)</li>
  <li>Fix handling of directory junctions on Windows (<a href="https://github-redirect.dependabot.com/psf/black/issues/2904">#2904</a>)</li>
  </ul>
  <h3>Documentation</h3>
  <ul>
  <li>Update pylint config documentation (<a href="https://github-redirect.dependabot.com/psf/black/issues/2931">#2931</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Move test to disable plugin in Vim/Neovim, which speeds up loading (<a href="https://github-redirect.dependabot.com/psf/black/issues/2896">#2896</a>)</li>
  </ul>
  <h3>Output</h3>
  <ul>
  <li>In verbose, mode, log when <em>Black</em> is using user-level config (<a href="https://github-redirect.dependabot.com/psf/black/issues/2861">#2861</a>)</li>
  </ul>
  <h3>Packaging</h3>
  <ul>
  <li>Fix Black to work with Click 8.1.0 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2966">#2966</a>)</li>
  <li>On Python 3.11 and newer, use the standard library's <code>tomllib</code> instead of <code>tomli</code>
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/2903">#2903</a>)</li>
  <li><code>black-primer</code>, the deprecated internal devtool, has been removed and copied to a
  <a href="https://github.com/cooperlees/black-primer">separate repository</a> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2924">#2924</a>)</li>
  </ul>
  <h3>Parser</h3>
  <ul>
  <li>Black can now parse starred expressions in the target of <code>for</code> and <code>async for</code>
  statements, e.g <code>for item in *items_1, *items_2: pass</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2879">#2879</a>).</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/psf/black/commit/ae2c0758c9e61a385df9700dc9c231bf54887041"><code>ae2c075</code></a> Prepare release 22.3.0 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2968">#2968</a>)</li>
  <li><a href="https://github.com/psf/black/commit/e9681a40dcb3d38b56b301d811bb1c55201fd97e"><code>e9681a4</code></a> Fix _unicodefun patch code for Click 8.1.0 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2966">#2966</a>)</li>
  <li><a href="https://github.com/psf/black/commit/ac7402cbf6a0deb5c74e9abcffc5bd7b1148fda5"><code>ac7402c</code></a> Bump sphinx from 4.4.0 to 4.5.0 in /docs (<a href="https://github-redirect.dependabot.com/psf/black/issues/2959">GH-2959</a>)</li>
  <li><a href="https://github.com/psf/black/commit/f239d227c003c52126239e1b9a37c36c2b2b8305"><code>f239d22</code></a> Enforce no formatting changes for PRs via CI (<a href="https://github-redirect.dependabot.com/psf/black/issues/2951">GH-2951</a>)</li>
  <li><a href="https://github.com/psf/black/commit/bd1e98034907463f5d86f4d87e89202dc6c34dd4"><code>bd1e980</code></a> Remove unnecessary parentheses from <code>except</code> clauses (<a href="https://github-redirect.dependabot.com/psf/black/issues/2939">#2939</a>)</li>
  <li><a href="https://github.com/psf/black/commit/14d84ba2e96c5ca1351b8fe4d0d415cc148f4117"><code>14d84ba</code></a> Resolve new flake8-bugbear errors (B020) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2950">GH-2950</a>)</li>
  <li><a href="https://github.com/psf/black/commit/14e5ce5412efa53438df0180e735b3834df3b579"><code>14e5ce5</code></a> Remove unnecessary parentheses from tuple unpacking in <code>for</code> loops (<a href="https://github-redirect.dependabot.com/psf/black/issues/2945">#2945</a>)</li>
  <li><a href="https://github.com/psf/black/commit/3800ebd81df6a1c31d1eac8cc15899537b9cbb61"><code>3800ebd</code></a> Avoid magic-trailing-comma in single-element subscripts (<a href="https://github-redirect.dependabot.com/psf/black/issues/2942">#2942</a>)</li>
  <li><a href="https://github.com/psf/black/commit/062b54931dc3ea35f673e755893fe28ff1f5a889"><code>062b549</code></a> Github now supports .git-blame-ignore-revs (<a href="https://github-redirect.dependabot.com/psf/black/issues/2948">GH-2948</a>)</li>
  <li><a href="https://github.com/psf/black/commit/5379d4f3f460ec9b7063dd1cc10f437b0edf9ae3"><code>5379d4f</code></a> stub style: remove some possible future changes (<a href="https://github-redirect.dependabot.com/psf/black/issues/2940">#2940</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/psf/black/compare/22.1.0...22.3.0">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.1.0&new-version=22.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1693/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c41OrWq
title: Update click requirement from <8.1.0,>=7.1.1 to >=7.1.1,<8.2.0
state: closed
created_at: 2022-03-29T13:11:23Z
updated_at: 2022-04-08T23:05:10Z
closed_at: 2022-04-08T23:05:09Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1694
type: pull
body:
  Updates the requirements on [click](https://github.com/pallets/click) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/pallets/click/releases">click's releases</a>.</em></p>
  <blockquote>
  <h2>8.1.0</h2>
  <p>This is a feature release, which includes new features and removes previously deprecated features. The 8.1.x branch is now the supported bugfix branch, the 8.0.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as <a href="https://pypi.org/project/pip-tools/">pip-tools</a> to pin all dependencies and control upgrades.</p>
  <ul>
  <li>Changes: <a href="https://click.palletsprojects.com/en/8.1.x/changes/#version-8-1-0">https://click.palletsprojects.com/en/8.1.x/changes/#version-8-1-0</a></li>
  <li>Milestone: <a href="https://github.com/pallets/click/milestone/9?closed=1">https://github.com/pallets/click/milestone/9?closed=1</a></li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pallets/click/blob/main/CHANGES.rst">click's changelog</a>.</em></p>
  <blockquote>
  <h2>Version 8.1.0</h2>
  <p>Released 2022-03-28</p>
  <ul>
  <li>
  <p>Drop support for Python 3.6. :pr:<code>2129</code></p>
  </li>
  <li>
  <p>Remove previously deprecated code. :pr:<code>2130</code></p>
  <ul>
  <li><code>Group.resultcallback</code> is renamed to <code>result_callback</code>.</li>
  <li><code>autocompletion</code> parameter to <code>Command</code> is renamed to
  <code>shell_complete</code>.</li>
  <li><code>get_terminal_size</code> is removed, use
  <code>shutil.get_terminal_size</code> instead.</li>
  <li><code>get_os_args</code> is removed, use <code>sys.argv[1:]</code> instead.</li>
  </ul>
  </li>
  <li>
  <p>Rely on :pep:<code>538</code> and :pep:<code>540</code> to handle selecting UTF-8 encoding
  instead of ASCII. Click's locale encoding detection is removed.
  :issue:<code>2198</code></p>
  </li>
  <li>
  <p>Single options boolean flags with <code>show_default=True</code> only show
  the default if it is <code>True</code>. :issue:<code>1971</code></p>
  </li>
  <li>
  <p>The <code>command</code> and <code>group</code> decorators can be applied with or
  without parentheses. :issue:<code>1359</code></p>
  </li>
  <li>
  <p>The <code>Path</code> type can check whether the target is executable.
  :issue:<code>1961</code></p>
  </li>
  <li>
  <p><code>Command.show_default</code> overrides <code>Context.show_default</code>, instead
  of the other way around. :issue:<code>1963</code></p>
  </li>
  <li>
  <p>Parameter decorators and <code>@group</code> handles <code>cls=None</code> the same as
  not passing <code>cls</code>. <code>@option</code> handles <code>help=None</code> the same as
  not passing <code>help</code>. :issue:<code>[#1959](https://github.com/pallets/click/issues/1959)</code></p>
  </li>
  <li>
  <p>A flag option with <code>required=True</code> requires that the flag is
  passed instead of choosing the implicit default value. :issue:<code>1978</code></p>
  </li>
  <li>
  <p>Indentation in help text passed to <code>Option</code> and <code>Command</code> is
  cleaned the same as using the <code>@option</code> and <code>@command</code>
  decorators does. A command's <code>epilog</code> and <code>short_help</code> are also
  processed. :issue:<code>1985</code></p>
  </li>
  <li>
  <p>Store unprocessed <code>Command.help</code>, <code>epilog</code> and <code>short_help</code>
  strings. Processing is only done when formatting help text for
  output. :issue:<code>2149</code></p>
  </li>
  <li>
  <p>Allow empty str input for <code>prompt()</code> when
  <code>confirmation_prompt=True</code> and <code>default=&quot;&quot;</code>. :issue:<code>2157</code></p>
  </li>
  <li>
  <p>Windows glob pattern expansion doesn't fail if a value is an invalid
  pattern. :issue:<code>2195</code></p>
  </li>
  <li>
  <p>It's possible to pass a list of <code>params</code> to <code>@command</code>. Any
  params defined with decorators are appended to the passed params.
  :issue:<code>2131</code>.</p>
  </li>
  <li>
  <p><code>@command</code> decorator is annotated as returning the correct type if
  a <code>cls</code> argument is used. :issue:<code>2211</code></p>
  </li>
  <li>
  <p>A <code>Group</code> with <code>invoke_without_command=True</code> and <code>chain=False</code>
  will invoke its result callback with the group function's return
  value. :issue:<code>2124</code></p>
  </li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pallets/click/commit/e4aceee8d2bf7fe9461915b0a21c4359ddcb8dc2"><code>e4aceee</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/click/issues/2224">#2224</a> from pallets/release-8.1.0</li>
  <li><a href="https://github.com/pallets/click/commit/f8d811e5d5644aca8d32eebff196bf7c659ebf45"><code>f8d811e</code></a> release version 8.1.0</li>
  <li><a href="https://github.com/pallets/click/commit/20c88f02788586a80e6d867854c8313eaba5ad6e"><code>20c88f0</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/click/issues/2223">#2223</a> from pallets/env-var</li>
  <li><a href="https://github.com/pallets/click/commit/8d7f03dac8739afed890af0c0921965786c5e83c"><code>8d7f03d</code></a> treat empty auto_envvar as None</li>
  <li><a href="https://github.com/pallets/click/commit/ef11be6e49e19a055fe7e5a89f0f1f4062c68dba"><code>ef11be6</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/click/issues/2041">#2041</a> from spanglerco/shell-completion-option-values</li>
  <li><a href="https://github.com/pallets/click/commit/f2e579ab187ca8fdfbe6ce86de08f0e9f62fe4ae"><code>f2e579a</code></a> shell completion prioritizes option values over new options</li>
  <li><a href="https://github.com/pallets/click/commit/d251cb0abc9b0dbda2402d4d831c18718cfb51bf"><code>d251cb0</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/click/issues/2219">#2219</a> from pallets/paramtype-name</li>
  <li><a href="https://github.com/pallets/click/commit/e0033315510510acdba14e880a361ccca091fb4b"><code>e003331</code></a> fix ParamType.to_info_dict() with no name</li>
  <li><a href="https://github.com/pallets/click/commit/19be092b6db4e4300e31906498e354ec0adf870c"><code>19be092</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pallets/click/issues/2217">#2217</a> from pallets/group-return</li>
  <li><a href="https://github.com/pallets/click/commit/7d3a871eb71694e99438254686c139122bc4be64"><code>7d3a871</code></a> group without command passes return value to result callback</li>
  <li>Additional commits viewable in <a href="https://github.com/pallets/click/compare/7.1.1...8.1.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1694/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5GrupC
title: Option to un-filter facet not shown for `?col__exact=value`
state: open
created_at: 2022-03-30T04:44:02Z
updated_at: 2022-03-30T04:46:18Z
author_association: OWNER
type: issue
body:
  Spotted this on a page with `COUNTY__exact=Lee` in the URL:

  ![CleanShot 2022-03-29 at 21 41 46@2x](https://user-images.githubusercontent.com/9599/160752849-a9039343-3770-4655-920b-f19e25687a57.png)

  With `COUNTY=Lee` you get this instead:

  <img width="796" alt="image" src="https://user-images.githubusercontent.com/9599/160752982-48666208-ce09-4791-a499-279171d2d870.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1695/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Gu4wK
title: Show foreign key label when filtering
state: open
created_at: 2022-03-30T16:18:54Z
updated_at: 2023-01-29T20:56:20Z
author_association: OWNER
type: issue
body:
  For example here:

  <img width="624" alt="image" src="https://user-images.githubusercontent.com/9599/160882762-a5000ad7-7646-4f60-8d58-23ad9e79b184.png">

  3 corresponds to "Human Related: Other" - it would be neat to display this in this area of the page somehow.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1696/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5G4G8J
title: `Request.fake(..., url_vars={})`
state: closed
created_at: 2022-04-01T01:48:40Z
updated_at: 2022-04-01T02:02:18Z
closed_at: 2022-04-01T02:02:10Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I just created an alternative `.fake()` method because I wanted to fake the `url_vars` captured in the route as well:
  ```python
  from datasette.utils.asgi import Request
  class Request(Request):

      @classmethod
      def fake(cls, path_with_query_string, method="GET", scheme="http", url_vars=None):
          """Useful for constructing Request objects for tests"""
          path, _, query_string = path_with_query_string.partition("?")
          scope = {
              "http_version": "1.1",
              "method": method,
              "path": path,
              "raw_path": path_with_query_string.encode("latin-1"),
              "query_string": query_string.encode("latin-1"),
              "scheme": scheme,
              "type": "http",
          }
          if url_vars:
              scope["url_route"] = {
                  "kwargs": url_vars
              }
          return cls(scope, None)
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1697/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5G-piD
title: Add a warning about bots and Cloud Run
state: closed
created_at: 2022-04-03T05:57:17Z
updated_at: 2022-04-03T06:10:24Z
closed_at: 2022-04-03T06:10:24Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Recommend the https://github.com/simonw/datasette-block-robots plugin if you are going to run a large database in Cloud Run (one with a lot of rows).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1698/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HHR-X
title: Proposal: datasette query
state: open
created_at: 2022-04-05T12:36:43Z
updated_at: 2022-04-11T01:32:12Z
author_association: CONTRIBUTOR
type: issue
body:
  I started sketching out a plugin to add a `datasette query` subcommand to export data from the command line. This is based on discussions in #1356 and #1605. Before I get too far down this rabbit hole, I figure it's worth getting some feedback here (unless this should happen in `Discussions`). Here's what I'm thinking:

  At its most basic, it will write the results of a query to STDOUT.

  ```sh
  datasette query -d data.db 'select * from data' > results.json
  ```

  This isn't much improvement over using [sqlite-utils](https://github.com/simonw/sqlite-utils). To make better use of datasette and its ecosystem, run `datasette query` using a canned query defined in a `metadata.yml` file.

  For example, using the metadata file from [alltheplaces-datasette](https://github.com/eyeseast/alltheplaces-datasette/blob/main/metadata.yml):

  ```sh
  cd alltheplaces-datasette
  datasette query -d alltheplaces.db -m metadata.yml count_by_spider
  ```

  That query would be good to get as CSV, and we can auto-discover metadata and databases in the current directory:

  ```sh
  cd alltheplaces-datasette
  datasette query count_by_spider -f csv
  ```

  In this case, `count_by_spider` is a canned query defined on the `alltheplaces` database. If the same query is defined on multiple databases or its otherwise unclear which database `query` should use, pass the `-d` or `--database` option.

  If a query takes parameters, I can pass them in at runtime, using the `--param` or `-p` option:

  ```sh
  datasette query -d data.db -p value something 'select * from neighborhoods where some_column = :value'
  ```

  I'm very interested in feedback on this, including whether it should be a plugin or in Datasette core. (I don't have a strong opinion about this, but I'm prototyping it as a plugin to start.)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1699/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HNw5o
title: Use + for spaces instead of ~20
state: closed
created_at: 2022-04-06T15:40:48Z
updated_at: 2022-04-06T15:55:10Z
closed_at: 2022-04-06T15:55:05Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Tilde encoding introduced in #1657 means that database files with spaces in the name - e.g. the Apple Mail `Envelope Index` database - end up with URLs like this:

      http://127.0.0.1:8001/Envelope~20Index

  I think this would be prettier:

      http://127.0.0.1:9933/Envelope+Index

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1701/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HToDz
title: Be more consistent with column quoting
state: open
created_at: 2022-04-07T16:59:20Z
updated_at: 2022-04-07T16:59:20Z
author_association: OWNER
type: issue
body:
  This tutorial made me notice that Datasette is pretty inconsistent with how column quoting works: https://datasette.io/tutorials/learn-sql

  It has examples of each of `"table_name"` and `[table_name]` and `table_name`, and it uses single quoted values too.

  Datasette should generate SQL as consistently as possible to support learners.

  That tutorial should also provide a tiny bit of extra information about what's going on here.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1702/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4132NJ
title: Update beautifulsoup4 requirement from <4.11.0,>=4.8.1 to >=4.8.1,<4.12.0
state: closed
created_at: 2022-04-08T13:08:53Z
updated_at: 2022-04-08T22:51:05Z
closed_at: 2022-04-08T22:51:05Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1703
type: pull
body:
  Updates the requirements on [beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/bs4/) to permit the latest version.


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1703/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HZuXp
title: File PRs against incompatible plugins pinning to datasette<1.0
state: open
created_at: 2022-04-08T23:15:30Z
updated_at: 2022-04-08T23:15:30Z
author_association: OWNER
type: issue
body:
  As part of the preparation for the 1.0 release, test all existing known plugins against the alpha.

  For any that break, submit a PR suggesting they pin to a version <1.0 - and include a link to the documentation on how to upgrade the plugin for 1.0.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1704/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HZujG
title: How to upgrade your plugin for 1.0 documentation
state: open
created_at: 2022-04-08T23:16:47Z
updated_at: 2022-12-13T05:29:05Z
author_association: OWNER
body:
  Among other things, needed by:
  - #1704

type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1705/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HdJSj
title: [feature] immutable mode for a directory, not just individual sqlite file
state: open
created_at: 2022-04-10T00:50:57Z
updated_at: 2022-12-09T19:11:40Z
author_association: CONTRIBUTOR
type: issue
body:
  ## Motivation

  - I have a directory of sqlite databases
  - I'd like to use immutable mode when opening them for better performance [docs](https://docs.datasette.io/en/0.54/performance.html#immutable-mode)
  - Currently using this flag throws the following error

      IsADirectoryError: [Errno 21] Is a directory: '/name-of-directory'

  ## Proposal

  Immutable flag works for both single files and directories

      datasette -i /folder-of-sqlite-files

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1706/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5HebPC
title: .extract() doesn't set foreign key when extracted columns contain NULL value
state: closed
created_at: 2022-04-10T20:05:30Z
updated_at: 2022-08-27T14:45:04Z
closed_at: 2022-08-27T14:45:04Z
author_association: NONE
type: issue
state_reason: completed
body:
  I've run into an issue with `extract` and I don't believe this is the intended behaviour.

  I'm working with a database with music listening information. Currently it has one large table `listens` that contains all information. I'm trying to normalize the database by extracting relevant columns to separate tables (`artists`, `tracks`, `albums`). Not every track has an album.

  A simplified demonstration with just `track_title` and `album_title` columns:
  ```ipython
  In [1]: import sqlite_utils

  In [2]: db = sqlite_utils.Database(memory=True)

  In [3]: db["listens"].insert_all([
     ...:     {"id": 1, "track_title": "foo", "album_title": "bar"},
     ...:     {"id": 2, "track_title": "baz", "album_title": None}
     ...: ], pk="id")
  Out[3]: <Table listens (id, track_title, album_title)>
  ```

  The track in the first row has an album, the second track doesn't. Now I extract album information into a separate column:
  ```ipython
  In [4]: db["listens"].extract(columns=["album_title"], table="albums", fk_column="album_id")
  Out[4]: <Table listens (id, track_title, album_id)>

  In [5]: list(db["albums"].rows)
  Out[5]: [{'id': 1, 'album_title': 'bar'}, {'id': 2, 'album_title': None}]

  In [6]: list(db["listens"].rows)
  Out[6]: 
  [{'id': 1, 'track_title': 'foo', 'album_id': 1},
   {'id': 2, 'track_title': 'baz', 'album_id': None}]
  ```

  This behaves as expected -- the `album` table contains entries for both the existing album and the NULL album. The `listens` table has a foreign key only for the first row (since the album in the second row was empty).

  Now I want to extract the track information as well. Album information belongs to the track so I want to extract both columns to a new table.
  ```ipython
  In [7]: db["listens"].extract(columns=["track_title", "album_id"], table="tracks", fk_column="track_id")
  Out[7]: <Table listens (id, track_id)>

  In [8]: list(db["tracks"].rows)
  Out[8]: 
  [{'id': 1, 'track_title': 'foo', 'album_id': 1},
   {'id': 2, 'track_title': 'baz', 'album_id': None}]

  In [9]: list(db["listens"].rows)
  Out[9]: [{'id': 1, 'track_id': 1}, {'id': 2, 'track_id': None}]
  ```

  Extracting to the `tracks` table worked fine (both tracks are present with correct columns). However, the `listens` table only has a foreign key to the newly created tracks for the first row, the foreign key in the second row is NULL.

  Changing the order of extracts doesn't help.

  I poked around in the source a bit and I believe [this line](https://github.com/simonw/sqlite-utils/blob/433813612ff9b4b501739fd7543bef0040dd51fe/sqlite_utils/db.py#L1737) (essentially comparing `NULL = NULL`) is the problem, but I don't know enough about SQL to create a reliable fix myself.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/423/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Hifqr
title: [feature] expanded detail page
state: open
created_at: 2022-04-11T16:29:17Z
updated_at: 2022-04-11T16:33:00Z
author_association: CONTRIBUTOR
type: issue
body:
  Right now, if click on the detail page for a row you get the info for the row and links to related tables:
  ![Screenshot 2022-04-11 at 12-27-26 lm20 filing](https://user-images.githubusercontent.com/536941/162786802-90ac1a71-4624-47c4-ae55-b783f4f6c92d.png)

  It would be very cool if there was an option to expand the rows of the related tables from within this detail view.

  If you had that then datasette could fulfill a pretty common use case where you want to search for an entity and get a consolidate detail view about what you know about that entity.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1707/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HkHOk
title: Datasette 1.0 alpha upcoming release notes
state: open
created_at: 2022-04-11T22:57:12Z
updated_at: 2022-12-13T05:29:06Z
author_association: OWNER
type: issue
body:
  I'm going to try writing the release notes first, to see if that helps unblock me.

  # ⚠️ Any release notes in this issue are a draft, and should not be treated as the real thing ⚠️ 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1708/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HkHUe
title: Redesigned JSON API with ?_extra= parameters
state: open
created_at: 2022-04-11T22:57:49Z
updated_at: 2022-12-13T05:29:06Z
author_association: OWNER
body: This will be the single biggest breaking change for the 1.0 release.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1709/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HkHah
title: Guide for plugin authors to upgrade their plugins for 1.0
state: closed
created_at: 2022-04-11T22:58:25Z
updated_at: 2022-04-11T23:04:01Z
closed_at: 2022-04-11T23:03:25Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I'll also encourage testing against both Datasette 0.x and Datasette 1.0 using a GitHub Actions matrix.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1710/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HkHj7
title: Template context powered entirely by the JSON API format
state: open
created_at: 2022-04-11T22:59:27Z
updated_at: 2022-12-13T05:29:06Z
author_association: OWNER
type: issue
body:
  Datasette 1.0 will have a stable template context. I'm going to achieve this by refactoring the templates to work only with keys returned by the API (or some of its extras) - then the API documentation will double up as template documentation.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1711/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Hk8NW
title: Better error message if you try to create a table with no columns
state: closed
created_at: 2022-04-12T02:43:20Z
updated_at: 2022-04-13T22:40:15Z
closed_at: 2022-04-13T22:40:10Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Seen here:

  - https://github.com/simonw/geojson-to-sqlite/issues/30

  Attempting to create a table with no columns produced this confusing error:

  ```
  File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/geojson_to_sqlite/utils.py", line 69, in import_features
      db[table].create(column_types, pk=pk)
    File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 863, in create
      self.db.create_table(
    File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 517, in create_table
      self.execute(sql)
    File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 236, in execute
      return self.conn.execute(sql)
  sqlite3.OperationalError: near ")": syntax error
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/424/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5HqIeg
title: Make "<Binary: 2427344 bytes>" easier to read
state: closed
created_at: 2022-04-12T18:17:07Z
updated_at: 2022-04-12T19:12:22Z
closed_at: 2022-04-12T18:44:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  `Binary: 2,427,344 bytes` would be nicer - even better, include a tooltip showing that size translated using this function: https://github.com/simonw/datasette/blob/138e4d9a53e3982137294ba383303c3a848cfca4/datasette/utils/__init__.py#L837-L846

  ![CleanShot 2022-04-12 at 11 15 04@2x](https://user-images.githubusercontent.com/9599/163027324-b0b6092e-6e11-438b-8077-789025d0bb37.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1712/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5HwS5g
title: `sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher
state: closed
created_at: 2022-04-13T22:16:53Z
updated_at: 2023-04-15T20:14:58Z
closed_at: 2022-04-13T22:48:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Got this error while investigating:
  - #421

  Even though I was using the `LD_PRELOAD` trick from https://til.simonwillison.net/sqlite/ld-preload to use a newer version of SQLite.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1098531354_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/425/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Hwrdo
title: Datasette feature for publishing snapshots of query results
state: open
created_at: 2022-04-14T01:42:00Z
updated_at: 2022-07-04T05:16:35Z
author_association: OWNER
type: issue
body:
  https://twitter.com/simonw/status/1514392335718645760

  > Maybe [@datasetteproj](https://twitter.com/datasetteproj)  should grow a feature that lets you cache the results of a query and give that snapshot a stable permalink
  >
  > A plugin that publishes the JSON output of a query to an S3 bucket would be pretty neat... especially if it could also be configured to re-publish the results on a schedule

  A lot of people said they would find this useful.

  Probably going to build this as a plugin.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1713/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5H3VR_
title: CLI docs should link to Python docs and vice versa
state: closed
created_at: 2022-04-15T16:05:15Z
updated_at: 2023-07-22T22:13:22Z
closed_at: 2023-07-22T22:13:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For every command/API method there should be a link to the equivalent in the other form factor.

  Maybe also link to the API and CLI reference pages too.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/426/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODtX3eM5H4BVC
title: Retrieve the top-level story for a comment
state: open
created_at: 2022-04-15T20:25:39Z
updated_at: 2022-04-15T20:25:39Z
author_association: NONE
type: issue
body:
  I think that each comment inserted into the database should include a column `onstory` that contains the ID of the story on which the comment was made. This is exactly equivalent to the link after "on:" at the top of an HN comment page ([example](https://news.ycombinator.com/item?id=18358028)). We could do this either by directly retrieving the HTML page and using Beautiful Soup to find that link, or alternatively recurse up the tree in the Firebase API using the `parent` field (probably using `functools.lru_cache` in case a person has commented a bunch of times on the same story).

reactions:
  {"url": "https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODFdgUs5IMrfj
title: feature: display progress bar when downloading multi-page responses
state: open
created_at: 2022-04-21T16:37:12Z
updated_at: 2022-04-21T17:29:31Z
author_association: NONE
type: issue
body:
  ## Motivation

  For a long running command (longer than 1 minute) for a big table (like pull requests or commits), it can be tricky to know if the script is still running, or if a rate limit/error was encountered

  We know how many pages there are, so it may be possible to indicate how many remain.

  https://github.com/dogsheep/github-to-sqlite/blob/a6e237f75a4b86963d91dcb5c9582e3a1b3349d6/github_to_sqlite/utils.py#L367

  ## Resources

  - Using the existing Click API: 
     - https://click.palletsprojects.com/en/5.x/utils/#showing-progress-bars
  - Loading spinner: https://github.com/pavdmyt/yaspin
  - Progress bar: https://github.com/tqdm/tqdm

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/72/reactions", "total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5ISFuB
title: sqlite-utils convert date parsing recipe complains about trying to parse "*"
state: closed
created_at: 2022-04-22T19:27:10Z
updated_at: 2022-07-02T13:59:59Z
closed_at: 2022-07-02T13:59:32Z
author_association: NONE
type: issue
state_reason: completed
body:
  Missing values in my dataset are denoted by a single asterisk.  I am trying to parse string dates into dates.  This works fine for columns without missing values, but, when the column contains "*", I get the following:

  ```
  $ sqlite-utils convert ${dbfile} details dob 'r.parsedate(value)' 
    [------------------------------------]    0%Traceback (most recent call last):
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py", line 2508, in convert_value
      return fn(v)
    File "<string>", line 2, in fn
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/recipes.py", line 8, in parsedate
      parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).date().isoformat()
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/dateutil/parser/_parser.py", line 1368, in parse
      return DEFAULTPARSER.parse(timestr, **kwargs)
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/dateutil/parser/_parser.py", line 643, in parse
      raise ParserError("Unknown string format: %s", timestr)
  dateutil.parser._parser.ParserError: Unknown string format: *

  Traceback (most recent call last):
    File "/usr/local/bin/sqlite-utils", line 33, in <module>
      sys.exit(load_entry_point('sqlite-utils==3.25.1', 'console_scripts', 'sqlite-utils')())
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/cli.py", line 2698, in convert
      db[table].convert(
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py", line 2524, in convert
      self.db.execute(sql, where_args or [])
    File "/usr/local/Cellar/sqlite-utils/3.25.1/libexec/lib/python3.9/site-packages/sqlite_utils/db.py", line 458, in execute
      return self.conn.execute(sql, parameters)
  sqlite3.OperationalError: user-defined function raised exception
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/427/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5ISjhx
title: Refactor TableView to use asyncinject
state: closed
created_at: 2022-04-22T21:43:39Z
updated_at: 2022-12-01T21:15:18Z
closed_at: 2022-04-28T22:26:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I've been working on a dependency injection mechanism in a separate library:

  - https://github.com/simonw/asyncinject

  I think it's ready to try out with Datasette to see if it's a pattern that will work here.

  I'm going to attempt to refactor `TableView` to use it. There are two overall goals here:

  - Use `asyncinject` to add parallel execution of some aspects of the table page - most notably I want to be able to execute the `count(*)` query, the `select ...` query, the various faceting queries and the facet suggestion queries in parallel - and measure if doing so is good for performance.
  - Use it to execute different output formats (possibly with some changes to the existing `register_output_renderer()` plugin hook). I want CSV and JSON to use the same mechanism that plugins use.

  Stretch goal is to get this working with streaming data too, see:

  - #1101

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1715/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5ISnQl
title: Configure git blame to ignore Black commit
state: closed
created_at: 2022-04-22T21:56:37Z
updated_at: 2022-04-22T22:02:19Z
closed_at: 2022-04-22T22:02:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  GitHub can support this in blame views now too:

  https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1716/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c42qyUI
title: Add timeout option to Cloudrun build
state: closed
created_at: 2022-04-23T11:51:21Z
updated_at: 2022-04-24T14:03:08Z
closed_at: 2022-04-24T14:03:08Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1717
type: pull
body:
  I've found that the Cloudrun build phase often hits a timeout limit with large databases. I believe the default timeout is 10 minutes. This pull request just adds a `--timeout` option to the cloudrun `publish` command and passes the value on to the build step.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1717/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IV1kU
title: Code examples in the documentation should be formatted with Black
state: closed
created_at: 2022-04-24T15:22:50Z
updated_at: 2022-04-24T16:24:14Z
closed_at: 2022-04-24T16:18:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For example on this page: https://docs.datasette.io/en/stable/writing_plugins.html#packaging-a-plugin

  I wonder if there's an easy way for me to enforce this for Sphinx documentation?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1718/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IaUm3
title: Refactor `RowView` and remove `RowTableShared`
state: closed
created_at: 2022-04-25T18:06:24Z
updated_at: 2022-12-01T21:15:19Z
closed_at: 2022-04-25T18:33:44Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > The `RowTableShared` class is making this a whole lot more complicated.
  > 
  > I'm going to split the `RowView` view out into an entirely separate `views/row.py` module.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1715#issuecomment-1108875068_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1719/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IbhXO
title: Design plugin hook for extras
state: closed
created_at: 2022-04-26T00:08:10Z
updated_at: 2022-12-01T21:15:19Z
closed_at: 2022-04-26T20:20:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Refs:
  - #262
  - #1709

  I realized that this is a really natural plugin hook - and if I design it as a hook I can implement Datasette's core extras as default plugins.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1720/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Ibrp5
title: Research adding support for savepoints
state: open
created_at: 2022-04-26T01:04:01Z
updated_at: 2022-04-26T01:05:29Z
author_association: OWNER
type: issue
body:
  https://www.sqlite.org/lang_savepoint.html

  Savepoints are like regular transactions except they have names and can be nested.

  Would there be any value in adding support to them to `sqlite-utils`, potentially as some kind of context manager? Something like this:
  ```python
  with db.savepoint("name"):
      # do stuff
      with db.savepoint("name2"):
          # do more stuff
          raise Release # Rolls back to before "name2" savepoint
  ```
  I've never used this feature so I'm not comfortable adding anything like this without a bunch of extra research.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/428/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IgVej
title: Implement plugin hooks: `register_table_extras`, `register_row_extras`, `register_query_extras`
state: open
created_at: 2022-04-26T20:21:49Z
updated_at: 2022-12-13T05:29:07Z
author_association: OWNER
body:
  Designed in:
  - #1720

  Part of:
  - #262
  - #1709

type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1721/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Igf-_
title: `db.primary_keys()` and `db.table_columns()` don't show up in traces
state: open
created_at: 2022-04-26T21:08:36Z
updated_at: 2022-04-26T21:08:36Z
author_association: OWNER
type: issue
body:
  Noticed this while working on:
  - #1715

  This code here isn't showing up in traces: https://github.com/simonw/datasette/blob/579f59dcec43a91dd7d404e00b87a00afd8515f2/datasette/views/table.py#L218-L220

  Because those functions don't use the regular trace-instrumented `db.execute()` code path - they work directly against a connection instead: https://github.com/simonw/datasette/blob/579f59dcec43a91dd7d404e00b87a00afd8515f2/datasette/utils/__init__.py#L610-L626

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1722/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IgnCw
title: Research running SQL in table view in parallel using `asyncio.gather()`
state: closed
created_at: 2022-04-26T21:42:48Z
updated_at: 2022-04-27T18:53:44Z
closed_at: 2022-04-26T22:19:09Z
author_association: OWNER
body:
  Spun off from:
  - #1715

type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1723/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IhCMM
title: ?_trace=1 doesn't work on Global Power Plants demo
state: closed
created_at: 2022-04-27T00:15:02Z
updated_at: 2022-04-27T06:15:14Z
closed_at: 2022-04-27T00:18:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_trace=1 is not showing the trace JSON at the bottom of the page.

  Confirmed that `trace_debug` is `true` on https://global-power-plants.datasettes.com/-/settings

  Possibly related:

  - https://github.com/simonw/datasette-total-page-time/issues/1

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1724/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IhDE5
title: Performance question - what is happening in this gap?
state: open
created_at: 2022-04-27T00:21:11Z
updated_at: 2022-04-27T00:21:11Z
author_association: OWNER
type: issue
body:
  Trace from https://latest-with-plugins.datasette.io/github/commits?_facet=repo&_trace=1&_facet=committer

  ![CleanShot 2022-04-26 at 17 20 06@2x](https://user-images.githubusercontent.com/9599/165413811-db2cd599-2acc-46ce-b9c2-f9bc45b879e9.png)

  What's going on in that gap? Can I improve the tracing output to show some non-SQL queries to figure that out?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1725/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Iiik8
title: Security page in the documentation
state: open
created_at: 2022-04-27T08:43:30Z
updated_at: 2022-04-27T08:43:30Z
author_association: OWNER
body: A page talking about how to run Datasette securely, and security concerns to take into account.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1726/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IlYeN
title: Research: demonstrate if parallel SQL queries are worthwhile
state: open
created_at: 2022-04-27T18:54:21Z
updated_at: 2022-09-26T14:48:31Z
author_association: OWNER
type: issue
body:
  I added parallel SQL query execution here:
  - https://github.com/simonw/datasette/issues/1723

  My hunch is that this will take advantage of multiple cores, since Python's `sqlite3` module releases the GIL once a query is passed to SQLite.

  I'd really like to prove this is the case though. Just not sure how to do it!

  Larger question: is this performance optimization actually improving performance at all? Under what circumstances is it worthwhile?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1727/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Imz12
title: Writable canned queries fail with useless non-error against immutable databases
state: closed
created_at: 2022-04-28T03:10:34Z
updated_at: 2022-08-14T16:34:40Z
closed_at: 2022-08-14T16:34:40Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  I've been banging my head against a wall for a while and would appreciate any pointers...

  - I have a writeable canned query to update rows in the db. 
  - I'm using the github-oauth plugin for authentication. 
  - I have `allow` set on the query to accept my GitHub id and a GH organisation. 
  - Authentication seems to work as expected both locally and on Cloudrun -- viewing `/-/actor` gives the same result in both environments
  - I can access the 'padlocked' canned query in both environments.

  Everything seems to be the same, but the canned query works perfectly when run locally, and fails when I try it on Cloudrun. I'm redirected back to the canned query page and the db is not changed. There's nothing in the Cloudstor logs to indicate an error.

  Any clues as to where I should be looking for the problem?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1728/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5IrllF
title: Implement ?_extra and new API design for TableView
state: open
created_at: 2022-04-28T22:28:14Z
updated_at: 2022-12-13T05:29:07Z
author_association: OWNER
body:
  Part of:
  - #262
  - #1518

type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1729/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Iro1H
title: SQL tracing should much more closely track the SQL query execution
state: open
created_at: 2022-04-28T22:41:04Z
updated_at: 2022-04-28T22:41:10Z
author_association: OWNER
type: issue
body:
  In #1727 I realized that the SQL tracing was measuring a whole bunch of stuff outside of the SQL query itself.

  I started experimenting with this fix for that but it didn't work - I got back an empty JSON array of traces for some reason:

  ```diff
  diff --git a/datasette/database.py b/datasette/database.py
  index ba594a8..d7f9172 100644
  --- a/datasette/database.py
  +++ b/datasette/database.py
  @@ -7,7 +7,7 @@ import sys
   import threading
   import uuid
   
  -from .tracer import trace
  +from .tracer import trace, trace_child_tasks
   from .utils import (
       detect_fts,
       detect_primary_keys,
  @@ -207,30 +207,31 @@ class Database:
                   time_limit_ms = custom_time_limit
   
               with sqlite_timelimit(conn, time_limit_ms):
  -                try:
  -                    cursor = conn.cursor()
  -                    cursor.execute(sql, params if params is not None else {})
  -                    max_returned_rows = self.ds.max_returned_rows
  -                    if max_returned_rows == page_size:
  -                        max_returned_rows += 1
  -                    if max_returned_rows and truncate:
  -                        rows = cursor.fetchmany(max_returned_rows + 1)
  -                        truncated = len(rows) > max_returned_rows
  -                        rows = rows[:max_returned_rows]
  -                    else:
  -                        rows = cursor.fetchall()
  -                        truncated = False
  -                except (sqlite3.OperationalError, sqlite3.DatabaseError) as e:
  -                    if e.args == ("interrupted",):
  -                        raise QueryInterrupted(e, sql, params)
  -                    if log_sql_errors:
  -                        sys.stderr.write(
  -                            "ERROR: conn={}, sql = {}, params = {}: {}\n".format(
  -                                conn, repr(sql), params, e
  +                with trace("sql", database=self.name, sql=sql.strip(), params=params):
  +                    try:
  +                        cursor = conn.cursor()
  +                        cursor.execute(sql, params if params is not None else {})
  +                        max_returned_rows = self.ds.max_returned_rows
  +                        if max_returned_rows == page_size:
  +                            max_returned_rows += 1
  +                        if max_returned_rows and truncate:
  +                            rows = cursor.fetchmany(max_returned_rows + 1)
  +                            truncated = len(rows) > max_returned_rows
  +                            rows = rows[:max_returned_rows]
  +                        else:
  +                            rows = cursor.fetchall()
  +                            truncated = False
  +                    except (sqlite3.OperationalError, sqlite3.DatabaseError) as e:
  +                        if e.args == ("interrupted",):
  +                            raise QueryInterrupted(e, sql, params)
  +                        if log_sql_errors:
  +                            sys.stderr.write(
  +                                "ERROR: conn={}, sql = {}, params = {}: {}\n".format(
  +                                    conn, repr(sql), params, e
  +                                )
                               )
  -                        )
  -                        sys.stderr.flush()
  -                    raise
  +                            sys.stderr.flush()
  +                        raise
   
               if truncate:
                   return Results(rows, truncated, cursor.description)
  @@ -238,9 +239,8 @@ class Database:
               else:
                   return Results(rows, False, cursor.description)
   
  -        with trace("sql", database=self.name, sql=sql.strip(), params=params):
  -            results = await self.execute_fn(sql_operation_in_thread)
  -        return results
  +        with trace_child_tasks():
  +            return await self.execute_fn(sql_operation_in_thread)
   
       @property
       def size(self):
  ```

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1727#issuecomment-1111602802_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1730/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5I0_KS
title: Custom page variables aren't decoded
state: open
created_at: 2022-04-30T14:55:46Z
updated_at: 2022-05-03T01:50:45Z
author_association: NONE
type: issue
body:
  I have a page `templates/filer/{filer_id}.html`. It uses `filer_id` in a `sql()` call to fetch data. With 0.61.1 this no longer works because the spaces in IDs isn't preserved. Instead, the escaped version is passed into the template and the id isn't present in my db.

  Datasette should unescape the url component before passing them into the template.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1732/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM43LrKB
title: Depend on click-default-group-wheel
state: closed
created_at: 2022-05-02T18:03:10Z
updated_at: 2022-05-02T18:52:42Z
closed_at: 2022-05-02T18:05:00Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/429
type: pull
body:
  Trying to get this to work with Pyodide.

  Refs: https://github.com/simonw/click-default-group-wheel/issues/3

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/429/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5I6RV0
title: Get Datasette compatible with Pyodide
state: closed
created_at: 2022-05-02T19:01:58Z
updated_at: 2022-05-04T15:14:01Z
closed_at: 2022-05-02T20:15:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I've already got this working as a prototype. Here are the changes I had to make:

  - Replace the two dependencies that don't publish pure Python wheels to PyPI: `click-default-group` and `python-baseconv`
  - Get Datasette to work without threading - which it turns out is exclusively used for database connections
  - Make the `uvicorn` dependency optional (only needed when Datasette runs in the CLI)

  TODO:

  - [x] Switch to `click-default-group-wheel`
  - [x] https://github.com/simonw/datasette/issues/1734
  - [x] Work around `uvicorn` import error
  - [x] https://github.com/simonw/datasette/issues/1735
  - [x] #1737

  Goal is to be able to do the following directly in https://pyodide.org/en/stable/console.html

  ```python
  import micropip
  await micropip.install("datasette")
  from datasette.app import Datasette
  ds = Datasette()
  await ds.client.get("/.json")
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1733/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5I6S-v
title: Remove python-baseconv dependency
state: closed
created_at: 2022-05-02T19:08:37Z
updated_at: 2022-05-02T23:25:49Z
closed_at: 2022-05-02T19:39:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I was going to vendor `baseconv.py`, but then I reconsidered - what if there are plugins out there that expect `import baseconv` to work because they have depended on Datasette?
  >
  > I used https://cs.github.com/ and as far as I can tell there aren't any!
  >
  > So I'm going to remove that dependency and work out a smarter way to do this - probably by providing a utility function within Datasette itself.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115258737_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1734/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5I6YU0
title: Datasette setting to disable threading (for Pyodide)
state: closed
created_at: 2022-05-02T19:31:08Z
updated_at: 2022-05-02T23:25:49Z
closed_at: 2022-05-02T20:13:52Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I'm going to add a Datasette setting to disable threading entirely, designed for usage in this particular case.
  >
  > I thought about adding a new setting, then I noticed this:
  >
  >     datasette mydatabase.db --setting num_sql_threads 10
  >
  > I'm going to let users set that to `0` to disable threaded execution of SQL queries.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115278325_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1735/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c43MIU0
title: Clean up compatibility with Pyodide
state: closed
created_at: 2022-05-02T20:14:38Z
updated_at: 2022-05-02T20:15:28Z
closed_at: 2022-05-02T20:15:27Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1736
body: Closes #1735, closes #1733
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1736/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5I7IOW
title: Automated test for Pyodide compatibility
state: closed
created_at: 2022-05-02T23:24:25Z
updated_at: 2022-05-02T23:40:50Z
closed_at: 2022-05-02T23:40:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Refs:
  - #1733

  Need something in the test suite such that if Datasette breaks against Pyodide in the future we hear about it.

  I'm thinking this is an opportunity to use [shot-scraper javascript](https://github.com/simonw/shot-scraper#scraping-pages-using-javascript).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1737/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5I7Ys6
title: "Cannot use _sort and _sort_desc at the same time"
state: closed
created_at: 2022-05-03T01:06:24Z
updated_at: 2022-08-14T16:13:55Z
closed_at: 2022-08-14T16:13:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Triggered this error while playing with the sort desc checkbox and the apply button that are only visible on this page at mobile screen width:

  https://latest.datasette.io/fixtures/compound_three_primary_keys?_sort_desc=pk1

  Navigate to that page (with the browser narrow enough to show the box), un-check the box and click Apply:

  ![sort-bug](https://user-images.githubusercontent.com/9599/166390804-cb289b29-63dc-4986-b7f9-81cf2ae04914.gif)

  Also notable: I managed to get to a page with `?_sort_desk=pk1` in the URL three times by clicking around with that button.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1738/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5I8CtQ
title: .db downloads should be served with an ETag
state: closed
created_at: 2022-05-03T05:11:21Z
updated_at: 2022-05-04T18:21:18Z
closed_at: 2022-05-03T14:59:51Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I noticed that my Pyodide Datasette prototype is downloading the same database file every single time rather than browser caching it:

  ![image](https://user-images.githubusercontent.com/9599/166407074-dee19587-0667-4424-9e88-d3b5b90fd819.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1739/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5I9nqx
title: Document how to use `PRAGMA temp_store` to avoid errors when running VACUUM against huge databases
state: open
created_at: 2022-05-03T13:33:58Z
updated_at: 2022-06-14T23:26:37Z
author_association: NONE
type: issue
body:
  I'm trying to figure out a way to get the `table.extract()` method to complete successfully -- I'm not sure if maybe the cause (and a possible solution) of this on Ubuntu Server 22.04 is to adjust some of the PRAGMA values within SQLite itself ... on another Linux system (PopOS), using this method on this same database appears to work just fine.

  Here's the bit that's causing the error, and the resulting error output:
  ```python
  # combine these columns into 1 table "bib_properties" :
  # best_title
  # bib_level_code
  # mat_type
  # material_code
  # best_author
  db["circ_trans"].extract(
      ["best_title", "bib_level_code", "mat_type", "material_code", "best_author"], 
      table="bib_properties", 
      fk_column="bib_properties_id"
  )

  db["circ_trans"].extract(
      ["call_number"], 
      table="call_number", 
      fk_column="call_number_id",
      rename={"call_number": "value"}
  )
  ```

  ```python
  ---------------------------------------------------------------------------
  OperationalError                          Traceback (most recent call last)
  Input In [17], in <cell line: 7>()
        1 # combine these columns into 1 table "bib_properties" :
        2 # best_title
        3 # bib_level_code
        4 # mat_type
        5 # material_code
        6 # best_author
  ----> 7 db["circ_trans"].extract(
        8     ["best_title", "bib_level_code", "mat_type", "material_code", "best_author"], 
        9     table="bib_properties", 
       10     fk_column="bib_properties_id"
       11 )
       13 db["circ_trans"].extract(
       14     ["call_number"], 
       15     table="call_number", 
       16     fk_column="call_number_id",
       17     rename={"call_number": "value"}
       18 )

  File ~/jupyter/venv/lib/python3.10/site-packages/sqlite_utils/db.py:1764, in Table.extract(self, columns, table, fk_column, rename)
     1761         column_order.append(c.name)
     1763 # Drop the unnecessary columns and rename lookup column
  -> 1764 self.transform(
     1765     drop=set(columns),
     1766     rename={magic_lookup_column: fk_column},
     1767     column_order=column_order,
     1768 )
     1770 # And add the foreign key constraint
     1771 self.add_foreign_key(fk_column, table, "id")

  File ~/jupyter/venv/lib/python3.10/site-packages/sqlite_utils/db.py:1526, in Table.transform(self, types, rename, drop, pk, not_null, defaults, drop_foreign_keys, column_order)
     1524 with self.db.conn:
     1525     for sql in sqls:
  -> 1526         self.db.execute(sql)
     1527     # Run the foreign_key_check before we commit
     1528     if pragma_foreign_keys_was_on:

  File ~/jupyter/venv/lib/python3.10/site-packages/sqlite_utils/db.py:465, in Database.execute(self, sql, parameters)
      463     return self.conn.execute(sql, parameters)
      464 else:
  --> 465     return self.conn.execute(sql)

  OperationalError: database or disk is full
  ```

  This database is about 17G in total size, so I'm assuming the error is coming from the vacuum ... where i'm assuming it's maybe trying to do the temp storage in a location that doesn't have sufficient room. The disk space is more than ample on the host in question (1.8T is free in the directory where the sqlite db resides) The `/tmp` directory however is limited on a smaller disk associated with the OS

  I'm trying to think if there's a way to set the `PRAGMA temp_store` or maybe if it's `temp_store_directory` that I'm after ... to use the same local directory of where the file is located (maybe this is a property of the version of sqlite on the system?) 

  ```python
  # SET the temp file store to be a file ...
  print(db.execute('PRAGMA temp_store').fetchall())
  print(db.execute('PRAGMA temp_store=FILE').fetchall())

  print(db.execute('PRAGMA temp_store').fetchall())

  # the users home directory ...
  print(db.execute("PRAGMA temp_store_directory='/home/plchuser/'").fetchall())
  print(db.execute("PRAGMA sqlite3_temp_directory='/home/plchuser/'").fetchall())

  print(db.execute("PRAGMA temp_store_directory").fetchall())
  print(db.execute("PRAGMA sqlite3_temp_directory").fetchall())
  ```
  ```text
  [(1,)]
  []
  [(1,)]
  []
  []
  [('/home/plchuser/',)]
  []
  ```

  Here's the docs on the Temporary File Storage Locations 
  https://www.sqlite.org/tempfiles.html

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/430/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c43U1z7
title: chore: Set permissions for GitHub actions
state: closed
created_at: 2022-05-05T01:03:08Z
updated_at: 2022-05-31T19:28:41Z
closed_at: 2022-05-31T19:28:40Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1740
type: pull
body:
   Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

  - Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

  https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

  https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

  [Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

  Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1740/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5JK0Cv
title: Allow making m2m relation of a table to itself
state: open
created_at: 2022-05-06T08:30:43Z
updated_at: 2022-06-23T14:12:51Z
author_association: NONE
type: issue
body:
  I am building a database, in which one of the tables has a many-to-many relationship to itself. As far as I can see, this is not (yet) possible using `.m2m()` in sqlite-utils. This may be a bit of a niche use case, so feel free to close this issue if you feel it would introduce too much complexity compared to the benefits.

  Example: suppose I have a table of people, and I want to store the information that John and Mary have two children, Michael and Suzy. It would be neat if I could do something like this:

  ```python
  from sqlite_utils import Database

  db = Database(memory=True)
  db["people"].insert({"name": "John"}, pk="name").m2m(
      "people", [{"name": "Michael"}, {"name": "Suzy"}], m2m_table="parent_child", pk="name"
  )
  db["people"].insert({"name": "Mary"}, pk="name").m2m(
      "people", [{"name": "Michael"}, {"name": "Suzy"}], m2m_table="parent_child", pk="name"
  )
  ```

  But if I do that, the many-to-many table `parent_child` has only one column:
  ```
  CREATE TABLE [parent_child] (
     [people_id] TEXT REFERENCES [people]([name]),
     PRIMARY KEY ([people_id], [people_id])
  )
  ```

  This could be solved by adding one or two keyword_arguments to `.m2m()`, e.g. `.m2m(..., left_name=None, right_name=None)` or `.m2m(..., names=(None, None))`.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/431/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5JtnBX
title: Support `rows_where()`, `delete_where()` etc for attached alias databases
state: open
created_at: 2022-05-16T06:38:58Z
updated_at: 2022-06-14T22:16:48Z
author_association: NONE
type: issue
body:
  Hi,

  I noticed `rows_where()` doesn't return any rows from tables which are from attached databases. The `exists()` function returns false. As far as I can see this is because the `table_names()` function only looks for table names in the current database and not in attached (or temp) databases.

  Besides, `rows_where()`, also `insert_all()` and `delete_where()` didn't do what I was expecting because of this. For the moment I've patched `table_names()` for myself, see below but I'm not sure what the total impact is on the other functions like lookup truncate etc which all use `exists()`. Also `view_names()` doesn't look for views in attached or temp databases. 
  ```python
   def table_names(self, fts4: bool = False, fts5: bool = False) -> List[str]:
          "A list of string table names in this database."
          where = ["type = 'table'"]
          if fts4:
              where.append("sql like '%USING FTS4%'")
          if fts5:
              where.append("sql like '%USING FTS5%'")
          dbs = [x[1] for x in self.execute('pragma database_list').fetchall()]    
          lst=[]
          for db in dbs:    
              sql = "select name from {} where {}".format(db+".sqlite_master"," AND ".join(where))
              lst.extend(r[0] for r in self.execute(sql).fetchall())
          return lst
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/432/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5JxBHL
title: ?_trace=1 fails with datasette-geojson for some reason
state: open
created_at: 2022-05-16T19:06:05Z
updated_at: 2022-05-16T19:42:13Z
author_association: OWNER
type: issue
body:
  view-source:https://calands.datasettes.com/calands/CPAD_2020a_SuperUnits.geojson?_sort=id&id__exact=4&_labels=on&_trace=1 is showing me a blank page.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1742/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5JyG1M
title: `datasette.utils.to_css_class()` should be a documented internal
state: open
created_at: 2022-05-16T23:57:26Z
updated_at: 2022-05-16T23:57:26Z
author_association: OWNER
body:
  Because I'm using it in this plugin:
  - https://github.com/simonw/datasette-upload-dbs/issues/1

type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1743/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5J2cZS
title: `--nolock` feature for opening locked databases
state: closed
created_at: 2022-05-17T18:25:16Z
updated_at: 2022-05-17T19:46:38Z
closed_at: 2022-05-17T19:40:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The getting started docs currently suggest you try this to browse your Chrome history:

      datasette ~/Library/Application\ Support/Google/Chrome/Default/History

  But if Chrome is running you will likely get this error:

      sqlite3.OperationalError: database is locked

  Turns out there's a workaround for this which I just spotted [on the SQLite forum](https://sqlite.org/forum/forumpost/86a67f6995):

  > You can do this using a [URI filename](https://sqlite.org/uri.html):
  > ```
  > sqlite3 'file:places.sqlite?mode=ro&nolock=1'
  > ```
  > That opens the file `places.sqlite` in read-only mode with locking disabled. This isn't safe, in that changes to the database made by other corrections are likely to cause this connection to return incorrect results or crash. Read-only mode should at least mean that you don't corrupt the database in the process.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1744/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5J2iwX
title: CLI eats my cursor
state: closed
created_at: 2022-05-17T18:52:52Z
updated_at: 2023-11-04T00:46:30Z
closed_at: 2023-11-04T00:46:30Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  I'm not sure why this happens but `sqlite-utils` makes my terminal cursor disappear after running commands like `sqlite-utils insert`. I've only noticed this behavior in `sqlite-utils`, not in any other CLI tools

  I can still type commands after it runs but the text cursor is invisible

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/433/reactions", "total_count": 5, "+1": 5, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5J2tym
title: Documentation on running cog
state: closed
created_at: 2022-05-17T19:41:06Z
updated_at: 2022-05-17T19:45:51Z
closed_at: 2022-05-17T19:43:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Noticed that `cog -r docs/*.rst` isn't documented in https://docs.datasette.io/en/latest/contributing.html#editing-and-building-the-documentation

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1745/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5KGPtQ
title: `detect_fts()` identifies the wrong table if tables have names that are subsets of each other
state: closed
created_at: 2022-05-20T13:28:31Z
updated_at: 2022-06-14T23:24:09Z
closed_at: 2022-06-14T23:24:09Z
author_association: NONE
type: issue
state_reason: completed
body:
  Windows 10
  Python 3.9.6

  When I was running a full text search through the Python library, I noticed that the query was being run on a different full text search table than the one I was trying to search.

  I took a look at the following function

  https://github.com/simonw/sqlite-utils/blob/841ad44bacaff05ec79ef78166d12e80c82ba6d7/sqlite_utils/db.py#L2213

  and noticed:

  ```python
  sql LIKE '%VIRTUAL TABLE%USING FTS%content=%{table}%'
  ```

  My database contains tables with similar names and %{table}% was matching another table that ended differently in its name.
  I have included a sample test that shows this occurring:

  I search for Marsupials in db["books"] and The Clue of the Broken Blade is returned. 

  This occurs since the search for Marsupials was "successfully" done against db["booksb"] and rowid 1 is returned.  "The Clue of the Broken Blade" has a rowid of 1 in db["books"] and this is what is returned from the search.

  ```python
  def test_fts_search_with_similar_table_names(fresh_db):
      db = Database(memory=True)
      db["books"].insert_all(
          [
              {
                  "title": "The Clue of the Broken Blade",
                  "author": "Franklin W. Dixon",
              },
              {
                  "title": "Habits of Australian Marsupials",
                  "author": "Marlee Hawkins",
              },
          ]
      )
      db["booksb"].insert(
          {
              "title": "Habits of Australian Marsupials",
              "author": "Marlee Hawkins",
          }
      )

      db["booksb"].enable_fts(["title", "author"])
      db["books"].enable_fts(["title", "author"])


      query = "Marsupials"

      assert [
              {   "rowid": 1,
                  "title": "Habits of Australian Marsupials",
                  "author": "Marlee Hawkins",
              },
          ] == list(db["books"].search(query))
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/434/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5KHkc6
title: Switch documentation theme to Furo
state: closed
created_at: 2022-05-20T18:42:17Z
updated_at: 2022-05-20T21:28:29Z
closed_at: 2022-05-20T21:28:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/pradyunsg/furo

  I just did this for `shot-scraper` and I really like it: https://shot-scraper.datasette.io/en/latest/

  - https://github.com/simonw/shot-scraper/issues/77

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1746/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5KHn4Y
title: Add tutorials to the getting started guide
state: closed
created_at: 2022-05-20T19:01:52Z
updated_at: 2022-05-20T19:12:30Z
closed_at: 2022-05-20T19:05:34Z
author_association: OWNER
body: On https://docs.datasette.io/en/stable/getting_started.html
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1747/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5KHpKY
title: Add copy buttons next to code examples in the documentation
state: closed
created_at: 2022-05-20T19:09:00Z
updated_at: 2022-05-20T19:15:00Z
closed_at: 2022-05-20T19:11:32Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Similar to the ones in `datasette-copyable` which are implemented here: https://github.com/executablebooks/sphinx-copybutton/tree/f84c001a0507f8ec46779d0701b079a265564583

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1748/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5KIW4P
title: Switch to Furo documentation theme
state: closed
created_at: 2022-05-20T21:46:39Z
updated_at: 2022-05-20T21:56:10Z
closed_at: 2022-05-20T21:54:43Z
author_association: OWNER
type: issue
state_reason: completed
body:
  As seen in:
  - https://github.com/simonw/datasette/issues/1746
  - https://github.com/simonw/shot-scraper/issues/77

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/435/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5KIZc1
title: Add "copy to clipboard" button to code examples in documentation
state: closed
created_at: 2022-05-20T21:53:23Z
updated_at: 2022-05-20T21:57:53Z
closed_at: 2022-05-20T21:57:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Follows:
  - #435

  Imitates:
  - https://github.com/simonw/datasette/issues/1748

  I'll use https://github.com/executablebooks/sphinx-copybutton - here's the Datasette commit: https://github.com/simonw/datasette/commit/1465fea4798599eccfe7e8f012bd8d9adfac3039

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/436/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODEm0Qs44PPLy
title: Ageinfo workaround
state: open
created_at: 2022-05-21T21:08:29Z
updated_at: 2022-05-21T21:09:16Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/twitter-to-sqlite/pulls/66
type: pull
body:
  I'm not sure if this is due to a new format or just because my ageinfo file is blank, but trying to import an archive would crash when it got to that file.  This PR adds a guard clause in the `ageinfo` transformer and sets a default value that doesn't throw an exception.  Seems likely to be the same issue mentioned by danp in https://github.com/dogsheep/twitter-to-sqlite/issues/54, my ageinfo file looks the same.  Added that same ageinfo file to the test archive as well to help confirm my workaround didn't break anything.

  Let me know if you want any changes!

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/66/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM44P4GG
title: docs to dogs
state: closed
created_at: 2022-05-22T15:50:33Z
updated_at: 2022-05-30T21:32:41Z
closed_at: 2022-05-30T21:32:41Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/437
body: Fixes a typo.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/437/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODLZ_YM5KUREo
title: When running `auth` command, don't overwrite an existing auth.json file
state: closed
created_at: 2022-05-24T16:42:20Z
updated_at: 2022-09-07T15:07:38Z
closed_at: 2022-08-22T16:17:19Z
author_association: NONE
type: issue
state_reason: completed
body:
  Ran the `auth` command in the same directory I'd previously set up an auth.json file for `twitter-to-sqlite` and it was completely overwritten.  Not the biggest issue, but still unexpected.  Ideally, for me, the keys would just be added to the existing file, but getting a warning and a chance to back out would be a good solution as well.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5KWITI
title: LDAP auth plugin
state: open
created_at: 2022-05-25T01:35:12Z
updated_at: 2022-05-25T01:35:12Z
author_association: NONE
type: issue
body:
  A [search of the plugins directory](https://datasette.io/plugins?q=ldap) doesn't turn up anything, but is is possible to set up a Datasette app which uses my organisation's LDAP for auth?

  If not, how much work would it be to write one (I _may_ have some spare cycles on my team to do this, but we haven't written a datasette plugin before).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1749/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Kg_Tf
title: illegal UTF-16 surrogate
state: closed
created_at: 2022-05-26T22:49:52Z
updated_at: 2022-05-27T08:21:53Z
closed_at: 2022-05-27T08:21:53Z
author_association: NONE
type: issue
state_reason: completed
body:
  I am trying to insert `https://artsdatabanken.no/Fab2018/api/export/csv` into a SQLite database, but I have an error when using `sqlite-utils`:

  ```
  sqlite-utils insert --csv --delimiter ";" --encoding="utf-16-le" --pk "Id" csv fremmedart test.db
    [------------------------------------]    0%
  Error: 'utf-16-le' codec can't decode bytes in position 98-99: illegal UTF-16 surrogate

  The input you provided uses a character encoding other than utf-8.

  You can fix this by passing the --encoding= option with the encoding of the file.

  If you do not know the encoding, running 'file filename.csv' may tell you.

  It's often worth trying: --encoding=latin-1
  ```

  I tried to convert the file using `iconv -f "utf-16le" -t "utf-8"`, but I still get a similar error (slightly different position):

  ```
  sqlite-utils insert --csv --delimiter ";" --encoding=utf-8 --pk "Id" csv_utf8 fremmedart test.db
    [------------------------------------]    0%
  Error: 'utf-8' codec can't decode byte 0xd9 in position 99: invalid continuation byte

  The input you provided uses a character encoding other than utf-8.

  You can fix this by passing the --encoding= option with the encoding of the file.

  If you do not know the encoding, running 'file filename.csv' may tell you.

  It's often worth trying: --encoding=latin-1
  ```

  I have no issues reading such file using this Python code:
  ```python
  content = open('csv', encoding='utf-16-le').read())
  ```

  `in2csv` works too.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/438/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODFE5qs44jvRV
title: Update README.md
state: open
created_at: 2022-05-27T03:13:59Z
updated_at: 2022-05-27T03:13:59Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/google-takeout-to-sqlite/pulls/11
body: Fix typo
type: pull
reactions:
  {"url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/11/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5KiQzI
title: Misleading progress bar against utf-16-le CSV input
state: open
created_at: 2022-05-27T08:34:49Z
updated_at: 2022-06-15T03:53:43Z
author_association: NONE
type: issue
body:
  The program crashes without any error.
  ```
  wget "https://artsdatabanken.no/Fab2018/api/export/csv"
  sqlite-utils create-database test.db
  sqlite-utils insert --csv --delimiter ";" --encoding "utf-16-le" test test.db csv 
    [------------------------------------]    0%
    [#################-------------------]   49%  00:00:01
  ```
  I would like to highlight various issues:
  1. sqlite-utils catches exceptions without printing the stacktrace and/or reraising the exception, so there is no easy way to use `pdb` or similar to debug the program, solution: add a debug option
  2. Silent crash: this is related to (1.), and it happens when there is a catch-all mechanism; solution: let the program fail.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/439/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5KixcM
title: CSV files with too many values in a row cause errors
state: closed
created_at: 2022-05-27T10:54:44Z
updated_at: 2022-06-14T22:23:01Z
closed_at: 2022-06-14T20:12:46Z
author_association: NONE
type: issue
state_reason: completed
body:
  *Original title: csv.DictReader can have None as key*

  In some cases, `csv.DictReader` can have `None` as key for unnamed columns, and a list of values as value.
  `sqlite_utils.utils.rows_from_file` cannot handle that:

  ```python
  url="https://artsdatabanken.no/Fab2018/api/export/csv"
  db = sqlite_utils.Database(":memory")

  with urlopen(url) as fab:
      reader, _ = sqlite_utils.utils.rows_from_file(fab, encoding="utf-16le")   
      db["fab2018"].insert_all(reader, pk="Id")
  ```

  Result:
  ```
  Traceback (most recent call last):
    File "<stdin>", line 3, in <module>
    File "/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py", line 2924, in insert_all
      chunk = list(chunk)
    File "/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py", line 3454, in fix_square_braces
      if any("[" in key or "]" in key for key in record.keys()):
    File "/home/user/.local/pipx/venvs/sqlite-utils/lib/python3.8/site-packages/sqlite_utils/db.py", line 3454, in <genexpr>
      if any("[" in key or "]" in key for key in record.keys()):
  TypeError: argument of type 'NoneType' is not iterable
  ```

  Code:
  https://github.com/simonw/sqlite-utils/blob/59be60c471fd7a2c4be7f75e8911163e618ff5ca/sqlite_utils/db.py#L3454

  `sqlite-utils insert` from command line is not affected by this issue.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/440/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Km26e
title: Allow `label_column` to specify array of columns
state: open
created_at: 2022-05-28T18:45:48Z
updated_at: 2022-05-28T18:45:48Z
author_association: NONE
type: issue
body:
  I think it would be great if the Datasette metadata would allow the `label_column` table key to list multiple columns. Something like:
  ```json
      "tables": {
          "person": {
              "label_column": ["first_name", "last_name"]
          },
  ```
  It would even be interesting with a "label expression" similar to a Python f-string. E.g. `{row.last_name}, {row.first_name}`.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1750/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Km5fQ
title: Add scrollbars to table presentation in default layout
state: closed
created_at: 2022-05-28T19:44:57Z
updated_at: 2022-05-28T19:52:17Z
closed_at: 2022-05-28T19:52:17Z
author_association: NONE
type: issue
state_reason: completed
body:
  (As you will be able to tell from the terminology I use, I am not a frontend guy, but I hope you will understand.)

  When a table is wide and needs horizontal scrolling to see the columns towards the end, the user needs to scroll horizontally. However, since the container for the HTML table (`div` with class `table-wrapper`) isn't limited by the window size, I first need to vertically scroll near to the bottom of the page in order to scroll horizontally. Then I can scroll back up again. This isn't very user friendly. Instead, I think it would make sense to constrain the table's size (when necessary), so that the vertical and horizontal scrollbars either always are visible or at least not far out of reach.

  I understand that I could provide my own template and / or CSS, but I think it would probably make sense to adjust the default in this regard.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1751/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5KnAW2
title: Research if I can drop Janus
state: open
created_at: 2022-05-28T22:46:52Z
updated_at: 2022-05-28T22:46:52Z
author_association: OWNER
type: issue
body:
  > It seems to me Janus dependency is not necessary, `async with app.database_write_mutex(): out = await app.transaction(func)` may be enough.

  Comment here: https://lobste.rs/s/fki4tj/architecture_notes_datasette#c_a2ihon

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1752/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5K91qp
title: Combining `rows_where()` and `search()` to limit which rows are searched
state: closed
created_at: 2022-06-02T06:01:55Z
updated_at: 2022-06-14T21:57:57Z
closed_at: 2022-06-14T21:54:38Z
author_association: NONE
type: issue
state_reason: completed
body:
  What is the right way to limit a full text search query to some rows of a table?

  For example, I have a table that contains the following columns: `title`, `content`, `owner` (each row represents a document). The `owner` column is a username. It feels right to store all documents in one table, instead of having one table per owner. In particular because I'd like to full text search all documents, only documents owned by one user and documents owned by a set of users.

  I tried to combine `.rows_where("owner = ?", "1234")` and `.search()` from the `Table` class but I don't think that is meant to work. I discovered `.search_sql()` as a way to generate the FTS SQL statement. By hand I can edit it to add a `AND [original].[owner] = :owner` to the `where` clause. This seems to do what I want.

  My two questions:
  1. is adding a `AND ...` to the `where` clause actually the right thing to do or should I be doing something else (my SQL skills are low)?
  2. is there a built-in to sqlite-utils way to achieve this?

  Right now I am thinking I will make my own version of `search_sql()` that generates a query that contains an additional `owner = :owner` for my particular use-case.

  Bonus question: is this generally useful/something to add to sqlite-utils or too niche?

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/441/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c45Kojn
title: Bump furo from 2022.4.7 to 2022.6.4.1
state: closed
created_at: 2022-06-06T13:10:22Z
updated_at: 2022-06-22T13:22:37Z
closed_at: 2022-06-22T13:22:35Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1753
type: pull
body:
  Bumps [furo](https://github.com/pradyunsg/furo) from 2022.4.7 to 2022.6.4.1.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pradyunsg/furo/blob/main/docs/changelog.md">furo's changelog</a>.</em></p>
  <blockquote>
  <h1>Changelog</h1>
  <!-- raw HTML omitted -->
  <h2>2022.06.04.1 -- Naughty Nickel bugfix</h2>
  <ul>
  <li>Fix the URL used in the &quot;Edit this page&quot; for Read the Docs builds.</li>
  </ul>
  <h2>2022.06.04 -- Naughty Nickel</h2>
  <ul>
  <li>✨ Advertise Sphinx 5 compatibility.</li>
  <li>✨ Change to <code>basic-ng</code> as the base theme (from {pypi}<code>sphinx-basic-ng</code>).</li>
  <li>Document site-wide announcement banners.</li>
  <li>Drop the pin on pygments.</li>
  <li>Improve edit button, using <code>basic-ng</code>'s <code>edit-this-page</code> component.</li>
  <li>Tweak headings to better match what users expect.</li>
  <li>Tweak how Sphinx's default HTML is rendered, using docutils post-transforms (this replaces parsing+modifying it with BeautifulSoup).</li>
  <li>When built with docutils 0.18, footnotes are rendered differently and stylised differently in Furo.</li>
  </ul>
  <h2>2022.04.07 -- Magical Mauve</h2>
  <ul>
  <li>✨ Make sphinx-copybutton look better.</li>
  <li>Add margin to indentations in line blocks.</li>
  <li>Add styling for non-arabic list styles</li>
  <li>Add support for <code>html_baseurl</code>.</li>
  <li>Improve &quot;Edit this page&quot; icon to be more accessible.</li>
  <li>Improve <code>html_sidebars</code> example.</li>
  <li>Tweak positioning of back to top on desktop.</li>
  </ul>
  <h2>2022.03.04 -- Lucent Lilac</h2>
  <ul>
  <li>Improve support for print media.</li>
  <li>Reduce heading sizes for h3 and below.</li>
  <li>Don't allow selecting headerlink content.</li>
  <li>Improve how overflow wrapping is handled.</li>
  <li>Add a reference from the configuration variables to the color customisation page.</li>
  </ul>
  <h2>2022.02.23 -- Keen Kobi</h2>
  <ul>
  <li>✨ Add a &quot;Back to Top&quot; button that shows up when scrolling up.</li>
  <li>Add a URL to GitHub in Project-URLs.</li>
  <li>Break long words in the prev/next buttons.</li>
  <li>Fix includes in Kitchen sink.</li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pradyunsg/furo/commit/1142fad58c60bc30e84d1bdc44325a1f7eb6ff4d"><code>1142fad</code></a> Prepare release: 2022.06.04.1</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/211abb46989fa930d0ed6a7802f10657019a7999"><code>211abb4</code></a> Update changelog</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/06cdba6f1ffcaa0407bd7788448b9e6ce15586aa"><code>06cdba6</code></a> Fix the edit this page URL</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/43ce4915c03f743d5c92f3f40ff0601f1416e2eb"><code>43ce491</code></a> Back to development</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/fb6e486aa0fbe06e1b706fcb5e0c54412f39f7bc"><code>fb6e486</code></a> Prepare release: 2022.06.04</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/090b02e56614abf43c95b32305d58b2c02c244da"><code>090b02e</code></a> Update changelog</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/098d51d209a2e5675c33b9a3b2213de8425db739"><code>098d51d</code></a> Fix the Just the Docs link</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/7fa8d083a1f6630e06b629f2cfc319a2a5b806d1"><code>7fa8d08</code></a> Change to a post-transform for wrapping math blocks and table</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/51f1e5232a5a7054201dc281c8dbabea7aef8868"><code>51f1e52</code></a> Speed up determining if there's multiple toc entries</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/99a6ff893ea1a57ff61aa45221fcf6653aa19a6b"><code>99a6ff8</code></a> Update caniuse-lite NPM package</li>
  <li>Additional commits viewable in <a href="https://github.com/pradyunsg/furo/compare/2022.04.07...2022.06.04.1">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.4.7&new-version=2022.6.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1753/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODFdgUs45K1L3
title: Fixing 'NoneType' object has no attribute 'items'
state: closed
created_at: 2022-06-06T13:58:11Z
updated_at: 2022-07-18T19:40:12Z
closed_at: 2022-07-18T19:40:12Z
author_association: CONTRIBUTOR
pull_request: dogsheep/github-to-sqlite/pulls/73
type: pull
body:
  Under some conditions, GitHub caches removed starred repositories and ends up leaving dangling `None` user references.

      Traceback (most recent call last):
        File "/home/dogsheep/dogsheep/github-to-sqlite/bin/github-to-sqlite", line 8, in <module>
          sys.exit(cli())
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1130, in __call__
          return self.main(*args, **kwargs)
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1055, in main
          rv = self.invoke(ctx)
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1657, in invoke
          return _process_result(sub_ctx.command.invoke(sub_ctx))
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1404, in invoke
          return ctx.invoke(self.callback, **ctx.params)
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 760, in invoke
          return __callback(*args, **kwargs)
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/cli.py", line 181, in starred
          utils.save_stars(db, user, stars)
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py", line 494, in save_stars
          repo_id = save_repo(db, repo)
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py", line 308, in save_repo
          to_save["owner"] = save_user(db, to_save["owner"])
        File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py", line 229, in save_user
          for key, value in user.items()
      AttributeError: 'NoneType' object has no attribute 'items'

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/73/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5LeMmn
title: Gunicorn
state: open
created_at: 2022-06-09T14:18:46Z
updated_at: 2022-06-09T14:18:46Z
author_association: NONE
type: issue
body:
  I've read issue #514 which resulted in running Datasette via systemd as recommended approach. We've also adopted this (for now), but I notice that Uvicorn [says the following](https://www.uvicorn.org/#running-with-gunicorn):

  > Uvicorn includes a Gunicorn worker class allowing you to run ASGI applications, with all of Uvicorn's performance benefits, while also giving you Gunicorn's fully-featured process management.
  > 
  > This allows you to increase or decrease the number of worker processes on the fly, restart worker processes gracefully, or perform server upgrades without downtime.
  > 
  > For production deployments we recommend using gunicorn with the uvicorn worker class.

  We usually deploy Python applications via Gunicorn for these process management features (e.g. `--daemon` and `--pid`). Is this something that would/could work with Datasette as well?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1755/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5LeqYH
title: Mechanism for creating databases in WAL mode
state: open
created_at: 2022-06-09T15:39:28Z
updated_at: 2022-06-09T15:39:28Z
author_association: OWNER
type: issue
body:
  The `--create` option currently creates databases if they are missing, but does not enable WAL mode for them.

  It turns out WAL mode is useful for databases that are accepting writes!

  I think a `--create-wal` option that both creates them AND sets WAL mode on any that are created would be a good idea.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1756/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c45fz-O
title: feat: add a wildcard for _json columns
state: open
created_at: 2022-06-11T01:01:17Z
updated_at: 2022-09-06T00:51:21Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: simonw/datasette/pulls/1757
type: pull
body:
  This allows _json to accept a wildcard for when there are many JSON columns that the user wants to convert. I hope this is useful. I've tested it on our datasette and haven't ran into any issues. I imagine on a large set of results, there could be some performance issues, but it will probably be negligible for most use cases.

  On a side note, I ran into an issue where I had to upgrade black on my system beyond the pinned version in setup.py. Here is the upstream issue <<https://github.com/psf/black/issues/2964> . I didn't include this in the PR yet since I didn't look into the issue too far, but I can if you would like.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1757/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5LsOyE
title: `maximize_csv_field_size_limit()` utility function
state: closed
created_at: 2022-06-13T19:54:54Z
updated_at: 2022-06-14T21:55:15Z
closed_at: 2022-06-14T21:31:49Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This code here runs only if `cli.py` is imported: https://github.com/simonw/sqlite-utils/blob/7ddf5300886a32d6daf60cf1d71efe492b65c87e/sqlite_utils/cli.py#L50-L59

  I found myself needing the same fix in another library:

  - https://github.com/simonw/datasette-socrata/issues/13

  It should be a documented utility function.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/442/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5LsqMG
title: Make `utils.rows_from_file()` a documented API
state: closed
created_at: 2022-06-13T21:53:24Z
updated_at: 2022-06-20T19:49:37Z
closed_at: 2022-06-14T20:12:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > `rows_from_file()` isn't part of the documented API but maybe it should be!

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1154385916_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/443/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5LyG1T
title: CSV `extras_key=` and `ignore_extras=` equivalents for CLI tool
state: open
created_at: 2022-06-14T22:22:47Z
updated_at: 2022-07-07T16:39:18Z
author_association: OWNER
type: issue
body:
  > I forgot to add equivalents of `extras_key=` and `ignore_extras=` to the CLI tool - will do that in a separate issue.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1155767915_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/444/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c454SlE
title: Extract facet portions of table.html out into included templates
state: closed
created_at: 2022-06-17T22:04:04Z
updated_at: 2022-06-20T18:05:45Z
closed_at: 2022-06-20T18:05:45Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1759
type: pull
body:
  To allow users customise the facet content as they would prefer such as sorting of facet results. ordering of suggested facets etc.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1759/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5MIfoP
title: `sqlite_utils.utils.TypeTracker` should be a documented API
state: closed
created_at: 2022-06-20T19:08:28Z
updated_at: 2022-06-20T19:49:02Z
closed_at: 2022-06-20T19:46:58Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I've used it in a couple of external places now:

  - https://github.com/simonw/datasette-socrata/blob/32fb256a461bf0e790eca10bdc7dd9d96c20f7c4/datasette_socrata/__init__.py#L264-L280
  - https://github.com/simonw/datasette-lite/blob/caa8eade10f0321c64f9f65c4561186f02d57c5b/webworker.js#L55-L64

  Refs:
  - https://github.com/simonw/datasette-lite/issues/32

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/445/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5MInsT
title: Use Just to automate running tests and linters locally
state: closed
created_at: 2022-06-20T19:51:09Z
updated_at: 2022-06-21T19:28:35Z
closed_at: 2022-06-20T19:54:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I keep committing code that fails additional tests like `mypy` and `flake8` and `black`. Automate those using Just.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/446/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5MNXS0
title: Incorrect syntax highlighting in docs CLI reference
state: closed
created_at: 2022-06-21T14:53:10Z
updated_at: 2022-06-21T18:48:47Z
closed_at: 2022-06-21T18:48:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://sqlite-utils.datasette.io/en/stable/cli-reference.html#insert

  ![CE020DDA-27FB-49C3-9EA6-37457DC4C321](https://user-images.githubusercontent.com/9599/174830380-06530537-b870-41c0-a8af-03c7fa720c6f.jpeg)

  It looks like Python keywords are being incorrectly highlighted here.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/447/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5MPjNB
title: Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto'
state: closed
created_at: 2022-06-21T21:48:27Z
updated_at: 2023-05-08T22:01:00Z
closed_at: 2023-05-08T22:01:00Z
author_association: NONE
type: issue
state_reason: completed
body:
  Attempting to run the example given here (without extra bracket ;-):
  https://sqlite-utils.datasette.io/en/stable/python-api.html#reading-rows-from-a-file
  ```
  from sqlite_utils.utils import rows_from_file
  import io

  rows, format = rows_from_file(io.StringIO("id,name\n1,Cleo"))
  print(list(rows), format)
  # Outputs [{'id': '1', 'name': 'Cleo'}] Format.CSV
  ```

  Gives error
  ```
  >"c:\Program Files\Python37\python.exe" test2.py
  Traceback (most recent call last):
    File "test2.py", line 4, in <module>
      rows, format = rows_from_file(io.StringIO("id,name\n1,Cleo"))
    File "C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main\sqlite_utils\utils.py", line 300, in rows_from_file
      first_bytes = buffered.peek(2048).strip()
  AttributeError: '_io.StringIO' object has no attribute 'readinto'
  ```

  I am running Python on Windows.
  ```
  >"c:\Program Files\Python37\python.exe"
  Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
  Type "help", "copyright", "credits" or "license" for more information.
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/448/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5MSSwk
title: Utilities for duplicating tables and creating a table with the results of a query
state: closed
created_at: 2022-06-22T09:41:43Z
updated_at: 2022-07-15T21:46:13Z
closed_at: 2022-07-15T21:21:36Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  is there a duplicate table functionality? Otherwise, I'd be happy to submit a PR.

  In sqlite3 it would look like:

  ```python
  import sqlite3 as sl

  con = sl.connect('prompt-tune.db')

  def db_duplicate_table(table_name, table_name_new, con=con):
      # Duplicates table `table_name` to a new table `table_name_new`.
      try:
          cur = con.cursor()
          cur.execute(f"""CREATE TABLE {table_name_new} AS SELECT * FROM {table_name}""")
      except Exception as e:
          print(e)
      finally:
          cur.close()

  db_duplicate_table('orig_table', 'new_table')
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/449/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c46Hsvj
title: Bump furo from 2022.4.7 to 2022.6.21
state: closed
created_at: 2022-06-22T13:22:31Z
updated_at: 2022-07-18T01:06:27Z
closed_at: 2022-07-18T01:06:27Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1760
type: pull
body:
  Bumps [furo](https://github.com/pradyunsg/furo) from 2022.4.7 to 2022.6.21.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pradyunsg/furo/blob/main/docs/changelog.md">furo's changelog</a>.</em></p>
  <blockquote>
  <h1>Changelog</h1>
  <!-- raw HTML omitted -->
  <h2>2022.06.21 -- Opulent Opal</h2>
  <ul>
  <li>Fix <code>docutils &lt;= 0.17.x</code> compatibility</li>
  <li>Bump to the latest Node.js LTS</li>
  </ul>
  <h2>2022.06.04.1 -- Naughty Nickel bugfix</h2>
  <ul>
  <li>Fix the URL used in the &quot;Edit this page&quot; for Read the Docs builds.</li>
  </ul>
  <h2>2022.06.04 -- Naughty Nickel</h2>
  <ul>
  <li>✨ Advertise Sphinx 5 compatibility.</li>
  <li>✨ Change to <code>basic-ng</code> as the base theme (from {pypi}<code>sphinx-basic-ng</code>).</li>
  <li>Document site-wide announcement banners.</li>
  <li>Drop the pin on pygments.</li>
  <li>Improve edit button, using <code>basic-ng</code>'s <code>edit-this-page</code> component.</li>
  <li>Tweak headings to better match what users expect.</li>
  <li>Tweak how Sphinx's default HTML is rendered, using docutils post-transforms (this replaces parsing+modifying it with BeautifulSoup).</li>
  <li>When built with docutils 0.18, footnotes are rendered differently and stylised differently in Furo.</li>
  </ul>
  <h2>2022.04.07 -- Magical Mauve</h2>
  <ul>
  <li>✨ Make sphinx-copybutton look better.</li>
  <li>Add margin to indentations in line blocks.</li>
  <li>Add styling for non-arabic list styles</li>
  <li>Add support for <code>html_baseurl</code>.</li>
  <li>Improve &quot;Edit this page&quot; icon to be more accessible.</li>
  <li>Improve <code>html_sidebars</code> example.</li>
  <li>Tweak positioning of back to top on desktop.</li>
  </ul>
  <h2>2022.03.04 -- Lucent Lilac</h2>
  <ul>
  <li>Improve support for print media.</li>
  <li>Reduce heading sizes for h3 and below.</li>
  <li>Don't allow selecting headerlink content.</li>
  <li>Improve how overflow wrapping is handled.</li>
  <li>Add a reference from the configuration variables to the color customisation page.</li>
  </ul>
  <h2>2022.02.23 -- Keen Kobi</h2>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pradyunsg/furo/commit/a2e8f42b9602e5606080b8a42e64adc04e510f9c"><code>a2e8f42</code></a> Prepare release: 2022.06.21</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/d4967e56ed653def30b4fa12871512747574d18e"><code>d4967e5</code></a> Update changelog</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/2969c2300aef17a72424007b67ea243a897e40bc"><code>2969c23</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://github-redirect.dependabot.com/pradyunsg/furo/issues/455">#455</a>)</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/23654933a0cf1012ac261029e36e23c89c5a4d69"><code>2365493</code></a> Use findall from docutils only if it's available</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/bb3c355862204665afe580ff09f06927b706872e"><code>bb3c355</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pradyunsg/furo/issues/450">#450</a> from polyzen/patch-1</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/4fb1a7446d54e657579d3e52907c093bb9b0ed32"><code>4fb1a74</code></a> Bump to the latest Node.js LTS</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/9970aacda99801197eb9b00875e5073f50e7ad09"><code>9970aac</code></a> Back to development</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/1142fad58c60bc30e84d1bdc44325a1f7eb6ff4d"><code>1142fad</code></a> Prepare release: 2022.06.04.1</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/211abb46989fa930d0ed6a7802f10657019a7999"><code>211abb4</code></a> Update changelog</li>
  <li><a href="https://github.com/pradyunsg/furo/commit/06cdba6f1ffcaa0407bd7788448b9e6ce15586aa"><code>06cdba6</code></a> Fix the edit this page URL</li>
  <li>Additional commits viewable in <a href="https://github.com/pradyunsg/furo/compare/2022.04.07...2022.06.21">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.4.7&new-version=2022.6.21)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1760/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5MV3Ib
title: ensure_ascii=False
state: open
created_at: 2022-06-22T19:58:13Z
updated_at: 2022-06-22T19:58:30Z
author_association: NONE
type: issue
body:
  Hi, thanks for the project!

  For the JSON output, I would consider defaulting to `ensure_ascii=False` (UTF-8 seems pretty universal) or making it an option. When dealing with non-Latin text, `ensure_ascii=True` (the default) can triple the size of the output.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1761/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c46ftHo
title: Bump black from 22.1.0 to 22.6.0
state: closed
created_at: 2022-06-28T13:11:32Z
updated_at: 2022-06-28T17:40:25Z
closed_at: 2022-06-28T17:40:25Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1763
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 22.1.0 to 22.6.0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>22.6.0</h2>
  <h3>Style</h3>
  <ul>
  <li>Fix unstable formatting involving <code>#fmt: skip</code> and <code># fmt:skip</code> comments (notice the lack of spaces) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2970">#2970</a>)</li>
  </ul>
  <h3>Preview style</h3>
  <ul>
  <li>Docstring quotes are no longer moved if it would violate the line length limit (<a href="https://github-redirect.dependabot.com/psf/black/issues/3044">#3044</a>)</li>
  <li>Parentheses around return annotations are now managed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2990">#2990</a>)</li>
  <li>Remove unnecessary parentheses around awaited objects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2991">#2991</a>)</li>
  <li>Remove unnecessary parentheses in <code>with</code> statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2926">#2926</a>)</li>
  <li>Remove trailing newlines after code block open (<a href="https://github-redirect.dependabot.com/psf/black/issues/3035">#3035</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Add <code>scripts/migrate-black.py</code> script to ease introduction of Black to a Git project (<a href="https://github-redirect.dependabot.com/psf/black/issues/3038">#3038</a>)</li>
  </ul>
  <h3>Output</h3>
  <ul>
  <li>Output Python version and implementation as part of <code>--version</code> flag (<a href="https://github-redirect.dependabot.com/psf/black/issues/2997">#2997</a>)</li>
  </ul>
  <h3>Packaging</h3>
  <ul>
  <li>Use <code>tomli</code> instead of <code>tomllib</code> on Python 3.11 builds where <code>tomllib</code> is not available (<a href="https://github-redirect.dependabot.com/psf/black/issues/2987">#2987</a>)</li>
  </ul>
  <h3>Parser</h3>
  <ul>
  <li><a href="https://peps.python.org/pep-0654/#except">PEP 654</a> syntax (for example, <code>except *ExceptionGroup:</code>) is now supported (<a href="https://github-redirect.dependabot.com/psf/black/issues/3016">#3016</a>)</li>
  <li><a href="https://peps.python.org/pep-0646">PEP 646</a> syntax (for example, <code>Array[Batch, *Shape]</code> or <code>def fn(*args: *T) -&gt; None</code>) is now supported (<a href="https://github-redirect.dependabot.com/psf/black/issues/3071">#3071</a>)</li>
  </ul>
  <h3>Vim Plugin</h3>
  <ul>
  <li>Fix <code>strtobool</code> function. It didn't parse true/on/false/off. (<a href="https://github-redirect.dependabot.com/psf/black/issues/3025">#3025</a>)</li>
  </ul>
  <p><strong>Full Changelog</strong>: <a href="https://github.com/psf/black/compare/22.3.0...22.6.0">https://github.com/psf/black/compare/22.3.0...22.6.0</a></p>
  <hr />
  <p><strong>Thank you!</strong></p>
  <ul>
  <li><a href="https://github.com/jpy-git"><code>@​jpy-git</code></a> for improving our parentheses formatting significantly</li>
  <li><a href="https://github.com/siuryan"><code>@​siuryan</code></a> for fixing a fmt: skip bug, making it a little less annoying to use :)</li>
  <li><a href="https://github.com/isidentical"><code>@​isidentical</code></a> for implementing support for PEP 654 and 646 syntax</li>
  <li><a href="https://github.com/defntvdm"><code>@​defntvdm</code></a> for fixing our vim plugin, especially as we (the maintainers) don't really know vim script sadly</li>
  <li><a href="https://github.com/idorrington92"><code>@​idorrington92</code></a> for fixing the docstring bug where Black would move the closing quotes causing it to violate the line length limit (whoops!)</li>
  <li><a href="https://github.com/hbrunn"><code>@​hbrunn</code></a> for contributing the migrate-black script</li>
  <li><a href="https://github.com/saroad2"><code>@​saroad2</code></a> for improving newline handling after code blocks and test infrastructure improvements</li>
  </ul>
  <p>... and everyone else who contributed documentation, tests, or other improvements to the Black project!</p>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>22.6.0</h2>
  <h3>Style</h3>
  <ul>
  <li>Fix unstable formatting involving <code>#fmt: skip</code> and <code># fmt:skip</code> comments (notice the
  lack of spaces) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2970">#2970</a>)</li>
  </ul>
  <h3>Preview style</h3>
  <ul>
  <li>Docstring quotes are no longer moved if it would violate the line length limit (<a href="https://github-redirect.dependabot.com/psf/black/issues/3044">#3044</a>)</li>
  <li>Parentheses around return annotations are now managed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2990">#2990</a>)</li>
  <li>Remove unnecessary parentheses around awaited objects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2991">#2991</a>)</li>
  <li>Remove unnecessary parentheses in <code>with</code> statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2926">#2926</a>)</li>
  <li>Remove trailing newlines after code block open (<a href="https://github-redirect.dependabot.com/psf/black/issues/3035">#3035</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Add <code>scripts/migrate-black.py</code> script to ease introduction of Black to a Git project
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/3038">#3038</a>)</li>
  </ul>
  <h3>Output</h3>
  <ul>
  <li>Output Python version and implementation as part of <code>--version</code> flag (<a href="https://github-redirect.dependabot.com/psf/black/issues/2997">#2997</a>)</li>
  </ul>
  <h3>Packaging</h3>
  <ul>
  <li>Use <code>tomli</code> instead of <code>tomllib</code> on Python 3.11 builds where <code>tomllib</code> is not
  available (<a href="https://github-redirect.dependabot.com/psf/black/issues/2987">#2987</a>)</li>
  </ul>
  <h3>Parser</h3>
  <ul>
  <li><a href="https://peps.python.org/pep-0654/#except">PEP 654</a> syntax (for example,
  <code>except *ExceptionGroup:</code>) is now supported (<a href="https://github-redirect.dependabot.com/psf/black/issues/3016">#3016</a>)</li>
  <li><a href="https://peps.python.org/pep-0646">PEP 646</a> syntax (for example,
  <code>Array[Batch, *Shape]</code> or <code>def fn(*args: *T) -&gt; None</code>) is now supported (<a href="https://github-redirect.dependabot.com/psf/black/issues/3071">#3071</a>)</li>
  </ul>
  <h3>Vim Plugin</h3>
  <ul>
  <li>Fix <code>strtobool</code> function. It didn't parse true/on/false/off. (<a href="https://github-redirect.dependabot.com/psf/black/issues/3025">#3025</a>)</li>
  </ul>
  <h2>22.3.0</h2>
  <h3>Preview style</h3>
  <ul>
  <li>Code cell separators <code>#%%</code> are now standardised to <code># %%</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2919">#2919</a>)</li>
  <li>Remove unnecessary parentheses from <code>except</code> statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2939">#2939</a>)</li>
  <li>Remove unnecessary parentheses from tuple unpacking in <code>for</code> loops (<a href="https://github-redirect.dependabot.com/psf/black/issues/2945">#2945</a>)</li>
  <li>Avoid magic-trailing-comma in single-element subscripts (<a href="https://github-redirect.dependabot.com/psf/black/issues/2942">#2942</a>)</li>
  </ul>
  <h3>Configuration</h3>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/psf/black/commit/f6c139c5215ce04fd3e73a900f1372942d58eca0"><code>f6c139c</code></a> Prepare docs for release 22.6.0 (<a href="https://github-redirect.dependabot.com/psf/black/issues/3139">#3139</a>)</li>
  <li><a href="https://github.com/psf/black/commit/eb5d175c9cd3c14a0731f8afd0cc5a18264353e7"><code>eb5d175</code></a> Update preview style docs to include recent changes (<a href="https://github-redirect.dependabot.com/psf/black/issues/3136">#3136</a>)</li>
  <li><a href="https://github.com/psf/black/commit/d848209d38cadaa060a023c223495e7874984ddc"><code>d848209</code></a> Bump furo from 2022.6.4.1 to 2022.6.21 in /docs (<a href="https://github-redirect.dependabot.com/psf/black/issues/3138">#3138</a>)</li>
  <li><a href="https://github.com/psf/black/commit/fa6caa6ca8489103d22d23f8f4ae4d3569bb115e"><code>fa6caa6</code></a> Only call get_future_imports when needed (<a href="https://github-redirect.dependabot.com/psf/black/issues/3135">#3135</a>)</li>
  <li><a href="https://github.com/psf/black/commit/6463fb874f6fd93d9a3b857e24987d5fa6ae0d57"><code>6463fb8</code></a> Bump sphinx from 5.0.1 to 5.0.2 in /docs (<a href="https://github-redirect.dependabot.com/psf/black/issues/3128">#3128</a>)</li>
  <li><a href="https://github.com/psf/black/commit/e3c9b0430eae5de35fdbeed047f9b2f07f9b78de"><code>e3c9b04</code></a> Replace link to Requests documentation (<a href="https://github-redirect.dependabot.com/psf/black/issues/3125">#3125</a>)</li>
  <li><a href="https://github.com/psf/black/commit/6c1bd08f16b636de38b92aeb2e0a1e8ebef0a0b1"><code>6c1bd08</code></a> Test run black on self (<a href="https://github-redirect.dependabot.com/psf/black/issues/3114">#3114</a>)</li>
  <li><a href="https://github.com/psf/black/commit/799adb53239e4a1e87253d40bc1cbd34f9103c52"><code>799adb5</code></a> Bump actions/setup-python from 3 to 4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/3121">#3121</a>)</li>
  <li><a href="https://github.com/psf/black/commit/162ecd1d2cf9471efefb5b61c17d28b73acb79a1"><code>162ecd1</code></a> Use is_number_token instead of assertion (<a href="https://github-redirect.dependabot.com/psf/black/issues/3069">#3069</a>)</li>
  <li><a href="https://github.com/psf/black/commit/8c8675c62aef4fb662c686d19ebd35dc047258f0"><code>8c8675c</code></a> Update documentation dependencies (<a href="https://github-redirect.dependabot.com/psf/black/issues/3118">#3118</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/psf/black/compare/22.1.0...22.6.0">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.1.0&new-version=22.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1763/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5NA0gK
title: Add --ignore option to more commands
state: closed
created_at: 2022-07-02T13:52:02Z
updated_at: 2022-07-15T22:39:09Z
closed_at: 2022-07-15T22:37:45Z
author_association: OWNER
type: issue
state_reason: completed
body:
  As seen in https://sqlite-utils.datasette.io/en/stable/cli-reference.html#add-foreign-key

  Could make this TIL trick unnecessary: https://til.simonwillison.net/bash/ignore-errors

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/450/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5NB_vB
title: Keep track of config_dir in directory mode (for plugins)
state: closed
created_at: 2022-07-03T16:57:49Z
updated_at: 2022-07-18T01:12:45Z
closed_at: 2022-07-18T01:12:45Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  I started working on using `config_dir` with my [datasette-query-files plugin](https://github.com/eyeseast/datasette-query-files) and realized Datasette doesn't actually hold onto the `config_dir` argument. It gets used in `__init__` but then forgotten. It would be nice to be able to use it in plugins, though.

  Here's the reference issue: https://github.com/eyeseast/datasette-query-files/issues/4

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1764/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5NCAIl
title: Document plugins providing new plugin hook-
state: closed
created_at: 2022-07-03T17:05:14Z
updated_at: 2023-08-31T23:08:24Z
closed_at: 2023-08-31T23:06:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I've used this pattern twice now: https://til.simonwillison.net/datasette/register-new-plugin-hooks - in `datasette-graphql` and `datasette-low-disk-space-hook`. I should describe the pattern on https://docs.datasette.io/en/stable/writing_plugins.html

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1765/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c46wdOW
title: Keep track of config_dir
state: closed
created_at: 2022-07-03T17:37:02Z
updated_at: 2022-07-18T01:12:45Z
closed_at: 2022-07-18T01:12:45Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1766
type: pull
body:
  Closes #1764 

  Small change that adds `self.config_dir = config_dir` to `Datasette.__init__`. This will let plugins also use `config_dir`, if available.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1766/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOD079W84600uh
title: Fix former command name in readme
state: open
created_at: 2022-07-05T02:09:13Z
updated_at: 2022-07-05T02:09:13Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/dogsheep-photos/pulls/37
body: Looks like a previous commit missed a `photo-to-sqlite`→ `dogsheep-photos` replacement.
type: pull
reactions:
  {"url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/37/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5NKqog
title: Ability to set a custom favicon
state: open
created_at: 2022-07-05T18:41:12Z
updated_at: 2022-07-05T18:56:43Z
author_association: OWNER
type: issue
body:
  If you're running a website on Datasette, like https://www.niche-museums.com/ or https://til.simonwillison.net/ - you should have the ability to easily specify a custom favicon.

  Currently the `/favicon.ico` view is hard-coded to do this: https://github.com/simonw/datasette/blob/9f1eb0d4eac483b953392157bd9fd6cc4df37de7/datasette/app.py#L179-L188

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1767/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5NQsls
title: Upgrade to 3.10.6-slim-bullseye Docker base image
state: closed
created_at: 2022-07-06T18:37:49Z
updated_at: 2022-08-14T15:54:36Z
closed_at: 2022-08-14T15:54:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For the package published to Docker Hub and also the containers used by `datasette package` and `datasette publish cloudrun`.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1768/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5NZgVF
title: Make sqlite_utils.utils.chunks a documented function
state: closed
created_at: 2022-07-08T06:01:04Z
updated_at: 2022-07-15T22:09:34Z
closed_at: 2022-07-15T21:59:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I want to use it in another project: https://github.com/simonw/sqlite-utils/blob/8a9fe6498faf783a1fdeb1793e661ad194a05267/sqlite_utils/utils.py#L471-L474

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/451/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM47JUun
title: Add duplicate table feature
state: closed
created_at: 2022-07-09T20:24:31Z
updated_at: 2022-07-15T21:21:37Z
closed_at: 2022-07-15T21:21:36Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/452
type: pull
body:
  This PR addresses a feature request raised in issue #449. Specifically this PR adds a functionality that lets users duplicate a table via:

  ```python
  table_new = db["my_table"].duplicate("new_table")
  ```

  Test added in file `tests/test_duplicate.py`.

  Happy to make changes to meet maintainers' feedback, if any. 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/452/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5NrMp_
title: 'unclosed file' warning when using insert_upsert_implementation from Python
state: closed
created_at: 2022-07-13T09:34:35Z
updated_at: 2022-07-15T21:52:25Z
closed_at: 2022-07-15T21:52:21Z
author_association: NONE
type: issue
state_reason: completed
body:
  I'm using the `[insert_upsert_implementation](https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/cli.py)` function directly in my Python code to import a csv file with all the bells and whistles `sqlite-utils` provides, but I'm getting a resource warning that a io.TextWrapper object is not closed.

  The warning goes away when wrapping the code from [this line](https://github.com/simonw/sqlite-utils/blob/42440d6345c242ee39778045e29143fb550bd2c2/sqlite_utils/cli.py#L924) in a try/finally block like:

  ```
  try:
     ...
     ...
  finally:
     decoded.close()
  ```
  (might be that `sniff_buffer` must also be closed if non null, but I might be wrong)

  I suspect Python closes the reference automatically when the sqlite-utils cli run is done, but since my code doesn't exit, I'm getting the warning.

  Alternatively, it'd be cool if the 'import csv/tsv' functionality could be added directly to the Database class.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/453/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c47eFtx
title: Update pytest-asyncio requirement from <0.19,>=0.17 to >=0.17,<0.20
state: closed
created_at: 2022-07-15T13:10:15Z
updated_at: 2022-07-18T01:06:38Z
closed_at: 2022-07-18T01:06:38Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1769
type: pull
body:
  Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-asyncio/releases">pytest-asyncio's releases</a>.</em></p>
  <blockquote>
  <h2>pytest-asyncio 0.19.0</h2>
  <hr />
  <h2>title: 'pytest-asyncio: pytest support for asyncio'</h2>
  <p><a href="https://pypi.python.org/pypi/pytest-asyncio"><img src="https://img.shields.io/pypi/v/pytest-asyncio.svg" alt="image" /></a></p>
  <p><a href="https://github.com/pytest-dev/pytest-asyncio/actions?workflow=CI"><img src="https://github.com/pytest-dev/pytest-asyncio/workflows/CI/badge.svg" alt="image" /></a></p>
  <p><a href="https://codecov.io/gh/pytest-dev/pytest-asyncio"><img src="https://codecov.io/gh/pytest-dev/pytest-asyncio/branch/master/graph/badge.svg" alt="image" /></a></p>
  <p><a href="https://github.com/pytest-dev/pytest-asyncio"><img src="https://img.shields.io/pypi/pyversions/pytest-asyncio.svg" alt="Supported Python versions" /></a></p>
  <p><a href="https://github.com/ambv/black"><img src="https://img.shields.io/badge/code%20style-black-000000.svg" alt="image" /></a></p>
  <p>pytest-asyncio is an Apache2 licensed library, written in Python, for
  testing asyncio code with pytest.</p>
  <p>asyncio code is usually written in the form of coroutines, which makes
  it slightly more difficult to test using normal testing tools.
  pytest-asyncio provides useful fixtures and markers to make testing
  easier.</p>
  <pre lang="{.sourceCode" data-meta=".python}"><code>@pytest.mark.asyncio
  async def test_some_asyncio_code():
      res = await library.do_something()
      assert b&quot;expected result&quot; == res
  </code></pre>
  <p>pytest-asyncio has been strongly influenced by
  <a href="https://github.com/eugeniy/pytest-tornado">pytest-tornado</a>.</p>
  <h1>Features</h1>
  <ul>
  <li>fixtures for creating and injecting versions of the asyncio event
  loop</li>
  <li>fixtures for injecting unused tcp/udp ports</li>
  <li>pytest markers for treating tests as asyncio coroutines</li>
  <li>easy testing with non-default event loops</li>
  <li>support for [async def]{.title-ref} fixtures and async generator
  fixtures</li>
  <li>support <em>auto</em> mode to handle all async fixtures and tests
  automatically by asyncio; provide <em>strict</em> mode if a test suite
  should work with different async frameworks simultaneously, e.g.
  <code>asyncio</code> and <code>trio</code>.</li>
  </ul>
  <h1>Installation</h1>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-asyncio/blob/master/CHANGELOG.rst">pytest-asyncio's changelog</a>.</em></p>
  <blockquote>
  <h1>0.19.0 (22-07-13)</h1>
  <ul>
  <li>BREAKING: The default <code>asyncio_mode</code> is now <em>strict</em>. <code>[#293](https://github.com/pytest-dev/pytest-asyncio/issues/293) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/293&gt;</code>_</li>
  <li>Removes <code>setup.py</code> since all relevant configuration is present <code>setup.cfg</code>. Users requiring an editable installation of pytest-asyncio need to use pip v21.1 or newer. <code>[#283](https://github.com/pytest-dev/pytest-asyncio/issues/283) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/283&gt;</code>_</li>
  <li>Declare support for Python 3.11.</li>
  </ul>
  <h1>0.18.3 (22-03-25)</h1>
  <ul>
  <li>Adds <code>pytest-trio &lt;https://pypi.org/project/pytest-trio/&gt;</code>_ to the test dependencies</li>
  <li>Fixes a bug that caused pytest-asyncio to try to set up async pytest_trio fixtures in strict mode. <code>[#298](https://github.com/pytest-dev/pytest-asyncio/issues/298) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/298&gt;</code>_</li>
  </ul>
  <h1>0.18.2 (22-03-03)</h1>
  <ul>
  <li>Fix asyncio auto mode not marking static methods. <code>[#295](https://github.com/pytest-dev/pytest-asyncio/issues/295) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/295&gt;</code>_</li>
  <li>Fix a compatibility issue with Hypothesis 6.39.0. <code>[#302](https://github.com/pytest-dev/pytest-asyncio/issues/302) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/302&gt;</code>_</li>
  </ul>
  <h1>0.18.1 (22-02-10)</h1>
  <ul>
  <li>Fixes a regression that prevented async fixtures from working in synchronous tests. <code>[#286](https://github.com/pytest-dev/pytest-asyncio/issues/286) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/286&gt;</code>_</li>
  </ul>
  <h1>0.18.0 (22-02-07)</h1>
  <ul>
  <li>Raise a warning if <a href="https://github.com/pytest"><code>@​pytest</code></a>.mark.asyncio is applied to non-async function. <code>[#275](https://github.com/pytest-dev/pytest-asyncio/issues/275) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/275&gt;</code>_</li>
  <li>Support parametrized <code>event_loop</code> fixture. <code>[#278](https://github.com/pytest-dev/pytest-asyncio/issues/278) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/278&gt;</code>_</li>
  </ul>
  <h1>0.17.2 (22-01-17)</h1>
  <ul>
  <li>Require <code>typing-extensions</code> on Python<!-- raw HTML omitted -->`_</li>
  <li>Fix a regression in tests collection introduced by 0.17.1, the plugin works fine with non-python tests again. <code>[#267](https://github.com/pytest-dev/pytest-asyncio/issues/267) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/267&gt;</code>_</li>
  </ul>
  <h1>0.17.1 (22-01-16)</h1>
  <ul>
  <li>Fixes a bug that prevents async Hypothesis tests from working without explicit <code>asyncio</code> marker when <code>--asyncio-mode=auto</code> is set. <code>[#258](https://github.com/pytest-dev/pytest-asyncio/issues/258) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/258&gt;</code>_</li>
  <li>Fixed a bug that closes the default event loop if the loop doesn't exist <code>[#257](https://github.com/pytest-dev/pytest-asyncio/issues/257) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/257&gt;</code>_</li>
  <li>Added type annotations. <code>[#198](https://github.com/pytest-dev/pytest-asyncio/issues/198) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/198&gt;</code>_</li>
  <li>Show asyncio mode in pytest report headers. <code>[#266](https://github.com/pytest-dev/pytest-asyncio/issues/266) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/266&gt;</code>_</li>
  <li>Relax <code>asyncio_mode</code> type definition; it allows to support pytest 6.1+. <code>[#262](https://github.com/pytest-dev/pytest-asyncio/issues/262) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/262&gt;</code>_</li>
  </ul>
  <h1>0.17.0 (22-01-13)</h1>
  <ul>
  <li><code>pytest-asyncio</code> no longer alters existing event loop policies. <code>[#168](https://github.com/pytest-dev/pytest-asyncio/issues/168) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/168&gt;</code><em>, <code>[#188](https://github.com/pytest-dev/pytest-asyncio/issues/188) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/168&gt;</code></em></li>
  <li>Drop support for Python 3.6</li>
  <li>Fixed an issue when pytest-asyncio was used in combination with <code>flaky</code> or inherited asynchronous Hypothesis tests. <code>[#178](https://github.com/pytest-dev/pytest-asyncio/issues/178) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/178&gt;</code>_ <code>[#231](https://github.com/pytest-dev/pytest-asyncio/issues/231) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/231&gt;</code>_</li>
  <li>Added <code>flaky &lt;https://pypi.org/project/flaky/&gt;</code>_ to test dependencies</li>
  <li>Added <code>unused_udp_port</code> and <code>unused_udp_port_factory</code> fixtures (similar to <code>unused_tcp_port</code> and <code>unused_tcp_port_factory</code> counterparts. <code>[#99](https://github.com/pytest-dev/pytest-asyncio/issues/99) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/99&gt;</code>_</li>
  <li>Added the plugin modes: <em>strict</em>, <em>auto</em>, and <em>legacy</em>. See <code>documentation &lt;https://github.com/pytest-dev/pytest-asyncio#modes&gt;</code>_ for details. <code>[#125](https://github.com/pytest-dev/pytest-asyncio/issues/125) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/125&gt;</code>_</li>
  <li>Correctly process <code>KeyboardInterrupt</code> during async fixture setup phase <code>[#219](https://github.com/pytest-dev/pytest-asyncio/issues/219) &lt;https://github.com/pytest-dev/pytest-asyncio/issues/219&gt;</code>_</li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/2da33c4feb4a7523f19dd791a6b0ee6200bf14d8"><code>2da33c4</code></a> docs: Prepare v0.19.0 release. (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/385">#385</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/07beb804a9c934862e0770bf51beb30b7e5a4230"><code>07beb80</code></a> opt into strict mode by default (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/380">#380</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/25c54a541707d7e4f3f4098c5d28a9534380180e"><code>25c54a5</code></a> Clarify documentation of event_loop fixture (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/375">#375</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/49f07a46fcc876decbb6c0fd431ef52172c5654c"><code>49f07a4</code></a> Bump typing-extensions from 4.2.0 to 4.3.0 in /dependencies/default (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/382">#382</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/739198bd62dd89c4d3bab4821d275883272ad2d6"><code>739198b</code></a> Bump hypothesis from 6.48.0 to 6.48.3 in /dependencies/default (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/381">#381</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/db72f254b47b2b123cd0ba8892d6c89a481fee96"><code>db72f25</code></a> Bump importlib-metadata from 4.11.4 to 4.12.0 in /dependencies/default (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/378">#378</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/4cf16cf984ee8cb4a290a8d4852db12f615368f2"><code>4cf16cf</code></a> Bump hypothesis from 6.47.3 to 6.48.0 in /dependencies/default (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/377">#377</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/f13c85f0f70e2e26b583bc5278876ff3229ad10c"><code>f13c85f</code></a> docs: Fix typo in README.</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/b463f72675cb680f5982013459b910ce3797a015"><code>b463f72</code></a> Python 3.11 support (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/370">#370</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/860ff5113c3e73ade396c632c822a171a81e5b78"><code>860ff51</code></a> Bump hypothesis from 6.47.2 to 6.47.3 in /dependencies/default (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/373">#373</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-asyncio/compare/v0.17.0...v0.19.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1769/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5N334V
title: `handle_exception` plugin hook for custom error handling
state: closed
created_at: 2022-07-15T20:52:49Z
updated_at: 2022-08-14T15:25:51Z
closed_at: 2022-08-14T15:25:51Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I need this for a couple of plugins, both of which are broken at the moment:
  - https://github.com/simonw/datasette-sentry/issues/1
  - https://github.com/simonw/datasette-show-errors/issues/2

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1770/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5N4Fit
title: CLI command for duplicating tables
state: closed
created_at: 2022-07-15T21:31:27Z
updated_at: 2022-07-15T21:48:23Z
closed_at: 2022-07-15T21:45:51Z
author_association: OWNER
body:
  CLI equivalent of:
  - #449

type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/454/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5N5v-r
title: minor a11y: <select> has no visual indicator when tabbed to
state: closed
created_at: 2022-07-17T04:30:14Z
updated_at: 2022-12-18T06:34:20Z
closed_at: 2022-12-18T06:28:12Z
author_association: NONE
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1771/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c47iWbd
title: Convert to setup.cfg
state: open
created_at: 2022-07-18T03:39:53Z
updated_at: 2022-07-18T03:39:53Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: simonw/datasette/pulls/1772
type: pull
body:
  Recent versions of setuptools can run most things from setup.cfg so one can have a simpler version that does not require executing code on install.

  The bulk of the changes were automated by running https://pypi.org/project/setup-py-upgrade/ with a few minor edits for the bits that it can not auto convert (the initial `get_long_description()` and `get_version()` can not be automatically converted)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1772/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODFdgUs5N_YgH
title: 500 error in github-to-sqlite demo
state: closed
created_at: 2022-07-18T19:39:32Z
updated_at: 2022-07-18T21:16:18Z
closed_at: 2022-07-18T21:14:22Z
author_association: MEMBER
type: issue
state_reason: completed
body:
  https://github-to-sqlite.dogsheep.net/github/issue_comments throws a 500:

  > `cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)`

  https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/metrics?project=datasette-222320 suggests this started happening 3 days ago.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM47pwAb
title: in extract code, check equality with IS instead of = for nulls
state: closed
created_at: 2022-07-19T13:40:25Z
updated_at: 2022-08-27T14:45:03Z
closed_at: 2022-08-27T14:45:03Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/455
body:
  sqlite "IS" is equivalent to SQL "IS NOT DISTINCT FROM"

  closes #423

type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/455/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5OGLo5
title: feature request: pivot command
state: open
created_at: 2022-07-20T00:58:08Z
updated_at: 2022-07-20T17:50:50Z
author_association: CONTRIBUTOR
type: issue
body:
  pivoting long-format table to wide-format tables is pretty common and kind of pain. would love to see this feature in sqlite-utils!

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/456/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5OnO8l
title: 500 error if sorted by a column not in the ?_col= list
state: closed
created_at: 2022-07-27T01:20:27Z
updated_at: 2022-08-14T16:06:25Z
closed_at: 2022-08-14T15:44:05Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For example: https://latest.datasette.io/fixtures/sortable?_sort_desc=sortable&_col=sortable_with_nulls

  That's `?_sort_desc=sortable&_col=sortable_with_nulls`

  <img width="494" alt="image" src="https://user-images.githubusercontent.com/9599/181139719-06e3bf66-3fde-4f9a-bfd1-62b326121601.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1773/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM48Mmde
title: Link to installation instructions
state: closed
created_at: 2022-07-27T17:38:36Z
updated_at: 2022-08-27T03:55:52Z
closed_at: 2022-07-27T17:57:50Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/457
body: Also testing https://docs.readthedocs.io/en/stable/pull-requests.html
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/457/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5OsU--
title: Support custom names for registered functions
state: closed
created_at: 2022-07-28T00:13:00Z
updated_at: 2022-08-27T03:56:01Z
closed_at: 2022-07-28T00:13:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  In this example:

  ```python
      @db.register_function
      def reverse_string(s):
          return "".join(reversed(list(s)))

      print(db.execute('select reverse_string("hello")').fetchone()[0])
  ```
  There's currently no way to over-ride the automatically selected name for the SQL function.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/458/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5O4HGm
title: Request of feature for mongo
state: open
created_at: 2022-07-31T01:00:05Z
updated_at: 2022-07-31T01:00:05Z
author_association: NONE
body: Will love if can we use datasette for mongo and all pipelines and workflows
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1774/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5O4Kno
title: i18n support
state: open
created_at: 2022-07-31T02:51:04Z
updated_at: 2023-02-10T18:04:40Z
author_association: NONE
body: I want contribute for translate UI to es, de, de and it if you share strings
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1775/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5O9LIp
title: Single quoted transform recipes on Windows do not work as expected
state: open
created_at: 2022-08-01T16:14:54Z
updated_at: 2022-08-01T16:14:54Z
author_association: CONTRIBUTOR
type: issue
body:
  Trying to follow the tutorial for sqlite-utils and datasette https://datasette.io/tutorials/clean-data on Windows 11 OS `Microsoft Windows [Version 10.0.22622.440]`, with sqlite-utils and datasette installed using pipx.

  ```
  pipx list
  package datasette 0.61.1, installed using Python 3.10.4
      - datasette.exe
  package sqlite-utils 3.28, installed using Python 3.10.4
      - sqlite-utils.exe
  ```  

  In the step to transform dates into ISO dates the quoted value `'r.parsedatetime(value)'` is copied verbatim into the columns instead of applying the output of the Python recipe.

  ```
  sqlite-utils convert manatees.db locations \
    REPDATE created_date last_edited_date \
    'r.parsedatetime(value)' --dry-run

  1975/01/31 00:00:00+00
   --- becomes:
  r.parsedatetime(value)

  Would affect 13568 rows
  ```

  However, if I change the code from single quotes to double quotes, it works as expected.

  ```
  sqlite-utils convert manatees.db locations \
    REPDATE created_date last_edited_date \
    "r.parsedatetime(value)" --dry-run

  1975/01/31 00:00:00+00
   --- becomes:
  1975-01-31T00:00:00+00:00

  Would affect 13568 rows
  ```

  Specifying the transform code recipe should work with single quotes on Windows.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/459/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM48hI-_
title: Cross-link CLI to Python docs
state: closed
created_at: 2022-08-02T16:18:28Z
updated_at: 2022-08-18T21:58:10Z
closed_at: 2022-08-18T21:58:07Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/460
type: pull
body:
  Work in progress, partly to test the ReadTheDocs preview link action.

  Refs:
  - #426

  <!-- readthedocs-preview readthedocs-preview start -->
  ----
  :books: Documentation preview :books:: https://readthedocs-preview--460.org.readthedocs.build/en/460/

  <!-- readthedocs-preview readthedocs-preview end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/460/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5PDntJ
title: Consider including animated SVG console demos
state: open
created_at: 2022-08-02T20:10:04Z
updated_at: 2022-08-02T20:12:14Z
author_association: OWNER
type: issue
body:
  I recorded this one using https://github.com/nbedos/termtosvg - with `pipx install termtosvg` and then `termtosvg` - execute demo - `exit` to save.

  ![sqlite-utils-insert-json](https://user-images.githubusercontent.com/9599/182464206-f4976af4-eda8-4020-8257-4ada1867fb44.svg)

  ```json
  [
    {
      "id": 1,
      "name": "Catimus"
    },
    {
      "id": 2,
      "name": "Feliopia"
    }
  ]
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/461/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM48iLGF
title: Discord badge
state: closed
created_at: 2022-08-02T20:56:04Z
updated_at: 2022-08-02T21:15:57Z
closed_at: 2022-08-02T21:15:52Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/462
type: pull
body:
  Also testing fix for:
  - https://github.com/readthedocs/readthedocs-preview/issues/10

  <!-- readthedocs-preview sqlite-utils start -->
  ----
  :books: Documentation preview :books:: https://sqlite-utils--462.org.readthedocs.build/en/462/

  <!-- readthedocs-preview sqlite-utils end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/462/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c488nq6
title: Use Read the Docs action v1
state: closed
created_at: 2022-08-10T10:30:50Z
updated_at: 2022-08-20T00:04:17Z
closed_at: 2022-08-20T00:04:17Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1778
type: pull
body:
  Read the Docs repository was renamed from `readthedocs/readthedocs-preview` to `readthedocs/actions/`. Now, the `preview` action is under `readthedocs/actions/preview` and is tagged as `v1`

  <!-- readthedocs-preview datasette start -->
  ----
  :books: Documentation preview :books:: https://datasette--1778.org.readthedocs.build/en/1778/

  <!-- readthedocs-preview datasette end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1778/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM488n6D
title: Use Read the Docs action v1
state: closed
created_at: 2022-08-10T10:31:47Z
updated_at: 2022-08-18T08:30:14Z
closed_at: 2022-08-17T23:11:16Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/463
type: pull
body:
  Read the Docs repository was renamed from `readthedocs/readthedocs-preview` to `readthedocs/actions/`. Now, the `preview` action is under `readthedocs/actions/preview` and is tagged as `v1`

  <!-- readthedocs-preview sqlite-utils start -->
  ----
  :books: Documentation preview :books:: https://sqlite-utils--463.org.readthedocs.build/en/463/

  <!-- readthedocs-preview sqlite-utils end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/463/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5PjNAw
title: google cloudrun updated their limits on maxscale based on memory and cpu count
state: closed
created_at: 2022-08-10T13:27:21Z
updated_at: 2022-08-14T19:42:59Z
closed_at: 2022-08-14T17:07:34Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  if you don't set an explicit limit on container scaling, then [google defaults to 100](https://cloud.google.com/run/docs/configuring/max-instances#limits)

  google recently updated the [limits on container scaling](https://cloud.google.com/run/docs/configuring/max-instances#limits), such that if you set up datasette to use more memory or cpu, then you need to set the maxScale argument much smaller than 100.

  would be nice if `datasette publish` could do this math for you and set the right maxScale. 

  [Log of an failing publish run](https://github.com/labordata/warehouse/runs/7764725972?check_suite_focus=true#step:8:332).

  ```
  ERROR: (gcloud.run.deploy) spec.template.spec.containers[0].resources.limits.cpu: Invalid value specified for cpu. For the specified value, maxScale may not exceed 15.
  Consider running your workload in a region with greater capacity, decreasing your requested cpu-per-instance, or requesting an increase in quota for this region if you are seeing sustained usage near this limit, see https://cloud.google.com/run/quotas. Your project may gain access to further scaling by adding billing information to your account.
  Traceback (most recent call last):
    File "/home/runner/.local/bin/datasette", line 8, in <module>
      sys.exit(cli())
    File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/runner/.local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/home/runner/.local/lib/python3.8/site-packages/datasette/publish/cloudrun.py", line 160, in cloudrun
      check_call(
    File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command 'gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/labordata/datasette warehouse --memory 8Gi --cpu 2' returned non-zero exit status 1.
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1779/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5PuUOW
title: `facet_time_limit_ms` and `sql_time_limit_ms` overlap?
state: open
created_at: 2022-08-12T17:55:37Z
updated_at: 2022-08-15T23:50:08Z
author_association: NONE
type: issue
body:
  I needed more than the default 200ms to facet a specific column in a database I was working with, so I ran `datasette` with `--setting facet_time_limit_ms 30000` &mdash; definitely overkill! 

  But it still didn't work; it took a moment to realize I also needed to up my `sql_time_limit_ms` to something larger too.

  I'm happy to submit a PR that documents this behavior if it's helpful. Or, if there's a code change we'd like to make (like making sure `sql_time_limit_ms` is always set to the larger of itself and `facet_time_limit_ms`), happy to do that too.

  Apologies if I missed this somewhere in the docs. And: thanks. I'm really enjoying the simple, effective tooling datasette gives me out of the box for exploring my databases!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1780/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5PwEaP
title: Link from documentation to source code
state: closed
created_at: 2022-08-13T16:19:57Z
updated_at: 2022-08-17T23:38:03Z
closed_at: 2022-08-17T23:38:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Twitter conversation asking for ways to automate this here: https://twitter.com/simonw/status/1558260492015046656

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/464/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5PwlsG
title: Ensure Datasette Lite is promoted in docs and README
state: closed
created_at: 2022-08-14T05:12:35Z
updated_at: 2022-08-14T15:24:40Z
closed_at: 2022-08-14T15:24:40Z
author_association: OWNER
body: As of 0.62 https://lite.datasette.io is a supported piece of the overall Datasette ecosystem.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1781/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5PxICo
title: Release notes for Datasette 0.62
state: closed
created_at: 2022-08-14T15:26:45Z
updated_at: 2022-08-14T17:40:45Z
closed_at: 2022-08-14T17:32:54Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I've written a lot of these already for the alphas:

  - https://github.com/simonw/datasette/releases/tag/0.62a0
  - https://github.com/simonw/datasette/releases/tag/0.62a1

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1782/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5P1k1V
title: Better guidance as to what to do after you've installed Datasette
state: open
created_at: 2022-08-15T20:11:06Z
updated_at: 2022-08-15T20:14:01Z
author_association: OWNER
type: issue
body:
  Feedback [from Discord](https://discord.com/channels/823971286308356157/823971286941302908/1008822978793984060):

  > hello, love the project and came for help and to point out a possible gap in the docs. starting with "getting started" and "installation" every thing looks great, but then there's a giant leap after you have it installed and running. from the user perspective of "i have a csv of set of csvs that i want to turn into a table(s), what do i do next?" --- so something like maybe a page for creating your first project should go after "installation".

  - https://docs.datasette.io/en/0.62/getting_started.html
  - https://docs.datasette.io/en/0.62/installation.html

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1783/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5P2aSe
title: Include "entrypoint" option on `--load-extension`?
state: closed
created_at: 2022-08-16T00:22:57Z
updated_at: 2022-08-23T18:34:31Z
closed_at: 2022-08-23T18:34:31Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  ## Problem 

  SQLite extensions have the option to define multiple "entrypoints" in each loadable extension. For example, the upcoming version of `sqlite-lines` will have 2 entrypoints: the default `sqlite3_lines_init` (which SQLite will automatically guess for) and `sqlite3_lines_noread_init`. The `sqlite3_lines_noread_init` version omits functions that read from the filesystem, which is necessary for security purposes when running untrusted SQL (which Datasette does).

  (Similar multiple entrypoints will also be added for sqlite-http).



  The `--load-extension` flag, however, doesn't give the option to specify a different entrypoint, so the default one is always used. 

  ## Proposal

  I want there to be a new command line option of the `--load-extension` flag to specify a custom entrypoint like so:
  ```
  datasette my.db \
    --load-extension ./lines0 sqlite3_lines0_noread_init
  ```

  Then, under the hood, this line of code:

  https://github.com/simonw/datasette/blob/7af67b54b7d9bca43e948510fc62f6db2b748fa8/datasette/app.py#L562

  Would look something like this:

  ```python
   conn.execute("SELECT load_extension(?, ?)", [extension, entrypoint]) 
  ```

  One potential problem: For backward compatibility, I'm not sure if Click allows cli flags to have variable number of options ("arity"). So I guess it could also use a `:` delimiter like `--static`:

  ```
  datasette my.db \
    --load-extension ./lines0:sqlite3_lines0_noread_init
  ```

  Or maybe even a new flag name?

  ```
  datasette my.db \
    --load-extension-entrypoint ./lines0 sqlite3_lines0_noread_init
  ```


  Personally I prefer the `:` option... and maybe even `--load-extension` -> `--load`? Definitely out of scope for this issue tho

  ```
  datasette my.db \
    --load./lines0:sqlite3_lines0_noread_init
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1784/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5P7IKz
title: Can't use cog menu to facet by first column in a view
state: open
created_at: 2022-08-16T21:27:23Z
updated_at: 2022-08-16T21:27:23Z
author_association: OWNER
type: issue
body:
  https://latest.datasette.io/fixtures/paginated_view

  <img width="701" alt="image" src="https://user-images.githubusercontent.com/9599/184988225-42f6218c-ea91-483d-af6d-52d1d1eff97b.png">

  Compare with:

  <img width="701" alt="image" src="https://user-images.githubusercontent.com/9599/184988263-3a9351d2-3db0-44d7-aa40-05ccd8dee870.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1785/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM49Wsnq
title: beanbag-docutils>=2.0
state: closed
created_at: 2022-08-17T22:41:39Z
updated_at: 2022-08-17T23:38:07Z
closed_at: 2022-08-17T23:38:02Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/465
body: Refs #464
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/465/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM49Wv9T
title: Use Read the Docs action v1 (#463)
state: closed
created_at: 2022-08-17T23:11:50Z
updated_at: 2022-08-17T23:11:54Z
closed_at: 2022-08-17T23:11:54Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/466
type: pull
body:
  Read the Docs repository was renamed from `readthedocs/readthedocs-preview` to `readthedocs/actions/`. Now, the `preview` action is under `readthedocs/actions/preview` and is tagged as `v1`

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/466/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5QA98H
title: Adjust height of textarea for no JS case
state: closed
created_at: 2022-08-18T01:15:15Z
updated_at: 2022-10-27T21:50:12Z
closed_at: 2022-08-18T16:06:09Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Datasette Lite: https://lite.datasette.io/?sql=https://gist.githubusercontent.com/simonw/1f8a91123ccefd8844187225b1832d7a/raw/5069075b86aa79358fbab3d4482d1d269077d632/recipes.sql#/data?sql=select+id%2C+name%2C+ingredients%2C+%28%0A++select+json_group_array%28value%29+from+json_each%28ingredients%29%0A++where+value+in+%28select+value+from+json_each%28%3Ap0%29%29%0A%29+as+matching_ingredients%0Afrom+recipes%0Awhere+json_array_length%28matching_ingredients%29+%3E+0%0Aorder+by+json_array_length%28matching_ingredients%29+desc&p0=%5B%22sugar%22%2C+%22cheese%22%5D

  ![46F8101E-8CE3-4F61-B200-F865E6B5DBCC](https://user-images.githubusercontent.com/9599/185270723-f55513b0-b561-434d-9d7c-4fe5be9756e0.jpeg)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1786/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5QEwEd
title: Move "datasette --get" from Getting Started to CLI Reference
state: closed
created_at: 2022-08-18T17:53:39Z
updated_at: 2022-08-18T21:57:09Z
closed_at: 2022-08-18T21:56:21Z
author_association: OWNER
body: It really shouldn't be here: https://docs.datasette.io/en/0.62/getting_started.html#datasette-get
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1787/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5QF7w0
title: Make it more obvious that Datasette publish can publish multiple databases
state: closed
created_at: 2022-08-18T22:57:51Z
updated_at: 2022-08-18T23:06:16Z
closed_at: 2022-08-18T23:06:16Z
author_association: OWNER
body: Feedback initially for `datasette-publish-fly` but it applies to the others too.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1788/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c49e3_k
title: Add new entrypoint option to `--load-extension`
state: closed
created_at: 2022-08-19T19:27:47Z
updated_at: 2022-08-23T18:42:52Z
closed_at: 2022-08-23T18:34:30Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1789
type: pull
body:
  Closes #1784 

  The `--load-extension` flag can now accept an optional "entrypoint" value, to specify which entrypoint SQLite should load from the given extension. 

  ```bash
  # would load default entrypoint like before
  datasette data.db --load-extension ext

  # loads the extensions with the "sqlite3_foo_init" entrpoint
  datasette data.db --load-extension ext:sqlite3_foo_init

  # loads the extensions with the "sqlite3_bar_init" entrpoint
  datasette data.db --load-extension ext:sqlite3_bar_init
  ```

  For testing, I added a small SQLite extension in C at `tests/ext.c`. If compiled, then pytest will run the unit tests in `test_load_extensions.py`to verify that Datasette loads in extensions correctly (and loads the correct entrypoints). Compiling the extension requires a C compiler, I compiled it on my Mac with:

  ```
  gcc ext.c -I path/to/sqlite -fPIC -shared -o ext.dylib
  ```

  Where `path/to/sqlite` is a directory that contains the SQLite amalgamation header files.

  Re documentation: I added a bit to the help text for `--load-extension` (which I believe should auto-add to documentation?), and the existing extension documentation is spatialite specific. Let me know if a new extensions documentation page would be helpful!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1789/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODLZ_YM5QMfmL
title: -a option is used for "--auth" and for "--all"
state: closed
created_at: 2022-08-21T10:50:48Z
updated_at: 2022-08-21T21:11:57Z
closed_at: 2022-08-21T21:11:57Z
author_association: NONE
body: I'm not sure which option is best, instead of -a -all.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5QM6J5
title: A better HTML title for canned query pages
state: open
created_at: 2022-08-21T18:27:46Z
updated_at: 2022-08-21T18:27:46Z
author_association: OWNER
type: issue
body:
  https://scotrail.datasette.io/scotrail/assemble_sentence?terms=This+train+is+formed+of%2Cbomb+which

  Current title is:

  `scotrail: with phrases as (  select    key, value  from    json_each(&#39;[&#34;&#39; || replace(:terms, &#39;,&#39;, &#39;&#34;,&#34;&#39;) || &#39;&#34;]&#39;)),matches as (select  phrases.key,  phrases.value,  (    select File from announcements    where announcements.Transcription like &#39;%&#39; || trim(phrases.value) || &#39;%&#39;    order by length(announcements.Transcription)    limit 1  ) as Filefrom  phrases),results as (  select key, announcements.Transcription, announcements.mp3  from announcements join matches on announcements.File = matches.File  order by key)select  &#39;Combined sentence:&#39; as mp3,  group_concat(Transcription, &#39; &#39;) as Transcription,  -1 as keyfrom results  unionselect  mp3, Transcription, keyfrom resultsorder by key`

  I think a better title would be:

  `scotrail: assemble_sentence, terms = This train is formed of,bomb which`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1790/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5QVIp1
title: Updating metadata.json on Datasette for MacOS
state: open
created_at: 2022-08-23T10:41:16Z
updated_at: 2022-08-23T13:29:51Z
author_association: NONE
type: issue
body:
  I've installed Datasette for Mac as per [the documentation](https://docs.datasette.io/en/stable/installation.html#datasette-desktop-for-mac) and it's working great!

  However, I'm not sure how to go about adding something like "[Canned Queries](https://docs.datasette.io/en/stable/sql_queries.html#canned-queries)" or utilising other advanced features or settings by manipulating the `metadata.json` or `settings.json` files.

  I can view these files from the Datasette App from the top right "burger" menu but it only shows the contents of the file with no way to edit or change it.

  Am I missing something? Where can I update the `metadata.json` file using the MacOS App?

  PS: This is a fantastic tool! Thanks so much for all the effort and especially adding a bunch of different ways to get started quickly!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1791/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QW3EN
title: Mechanism for ensuring a table has all the columns
state: closed
created_at: 2022-08-23T15:50:23Z
updated_at: 2022-08-27T23:19:41Z
closed_at: 2022-08-27T23:17:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Suggested by @jefftriplett on Discord: https://discord.com/channels/823971286308356157/997738192360964156/1011655389063958600

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/467/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM49qP2V
title: db[table].create(..., transform=True) and create-table --transform
state: closed
created_at: 2022-08-23T17:27:58Z
updated_at: 2022-08-27T23:17:55Z
closed_at: 2022-08-27T23:17:55Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/468
type: pull
body:
  Work in progress. Still needs documentation and tests (and to cover more cases of things that might have changed).

  Refs:
  - #467

  <!-- readthedocs-preview sqlite-utils start -->
  ----
  :books: Documentation preview :books:: https://sqlite-utils--468.org.readthedocs.build/en/468/

  <!-- readthedocs-preview sqlite-utils end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/468/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c49qmC2
title: Test `--load-extension` in GitHub Actions
state: closed
created_at: 2022-08-23T18:43:29Z
updated_at: 2022-08-24T00:11:46Z
closed_at: 2022-08-24T00:11:45Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1792
type: pull
body:
  Refs:
  - #1789

  <!-- readthedocs-preview datasette start -->
  ----
  :books: Documentation preview :books:: https://datasette--1792.org.readthedocs.build/en/1792/

  <!-- readthedocs-preview datasette end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1792/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c492dPw
title: Added a useful resource
state: closed
created_at: 2022-08-26T08:41:26Z
updated_at: 2022-09-06T00:41:25Z
closed_at: 2022-09-06T00:41:24Z
author_association: NONE
pull_request: simonw/datasette/pulls/1793
type: pull
body:
  Have added a useful resource about the types of databases in SQL i.e SQLite, PostgreSQL, MySQL &, etc from the scaler topics.

  <!-- readthedocs-preview datasette start -->
  ----
  :books: Documentation preview :books:: https://datasette--1793.org.readthedocs.build/en/1793/

  <!-- readthedocs-preview datasette end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1793/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c495vvy
title: fix word break in facets by adding ul.tight-bullets li word-break: break-all
state: closed
created_at: 2022-08-27T03:47:25Z
updated_at: 2022-09-06T00:45:41Z
closed_at: 2022-09-06T00:45:41Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1794
type: pull
body:
  I noticed that long words break the layout of facets:

  ![image](https://user-images.githubusercontent.com/128286/187013146-fb2bbb60-a225-441b-ba8e-b9e74fb04f93.png)

  So I added CSS to add a line break. This is how the result looks now:

  ![image](https://user-images.githubusercontent.com/128286/187013175-a706fc72-9e69-4a75-9bdf-bdaa34a0cf51.png)

  I don't know enough about facet edge cases to decide if this change might break other things but it looks better for me so maybe this is helpful.

  <!-- readthedocs-preview datasette start -->
  ----
  :books: Documentation preview :books:: https://datasette--1794.org.readthedocs.build/en/1794/

  <!-- readthedocs-preview datasette end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1794/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QpBiI
title: sqlite-utils rows --order option
state: closed
created_at: 2022-08-27T03:49:51Z
updated_at: 2022-08-27T04:30:49Z
closed_at: 2022-08-27T04:10:32Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For consistency with `search`: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#search

  ```
    -o, --order TEXT       Order by ('column' or 'column desc')
  ```

  I wanted to run `sqlite-utils rows db.db mytable --order 'rowid desc'` to see the most recently imported rows.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/469/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QpBrG
title: Upgrade `--load-extension` to accept entrypoints like Datasette
state: closed
created_at: 2022-08-27T03:53:20Z
updated_at: 2022-08-27T05:55:49Z
closed_at: 2022-08-27T05:55:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Imitate:
  - https://github.com/simonw/datasette/pull/1789
  ```
  # would load default entrypoint like before
  datasette data.db --load-extension ext

  # loads the extensions with the "sqlite3_foo_init" entrpoint
  datasette data.db --load-extension ext:sqlite3_foo_init

  # loads the extensions with the "sqlite3_bar_init" entrpoint
  datasette data.db --load-extension ext:sqlite3_bar_init
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/470/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QpB1s
title: sqlite-utils query --functions mechanism for registering extra functions
state: closed
created_at: 2022-08-27T03:57:53Z
updated_at: 2022-09-07T03:46:26Z
closed_at: 2022-08-27T05:10:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It would be really cool if you could register additional custom SQL functions for use with the `sqlite-utils query` command - something like this:

  ```
  sqlite-utils data.db 'update images set domain = extract_domain(url)' --functions '
  from urllib.parse import urlparse

  def extract_domain(url):
      return urlparse(url).netloc
  '
  ```
  Every function defined in that code block would be registered with the connection, unless the name began with an underscore.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/471/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QpFHX
title: Reuse the locals/globals fix from --functions for other code accepting options
state: closed
created_at: 2022-08-27T05:12:05Z
updated_at: 2022-08-27T05:20:12Z
closed_at: 2022-08-27T05:20:12Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I figured out a workaround for the ugly `global x` hack here:
  - https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229120653

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/472/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4950Az
title: Support entrypoints for `--load-extension`
state: closed
created_at: 2022-08-27T05:53:59Z
updated_at: 2022-08-27T05:55:52Z
closed_at: 2022-08-27T05:55:47Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/473
type: pull
body:
  Refs #470

  <!-- readthedocs-preview sqlite-utils start -->
  ----
  :books: Documentation preview :books:: https://sqlite-utils--473.org.readthedocs.build/en/473/

  <!-- readthedocs-preview sqlite-utils end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/473/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QpkVl
title: Add an option for specifying column names when inserting CSV data
state: open
created_at: 2022-08-27T15:29:59Z
updated_at: 2022-08-31T03:42:36Z
author_association: NONE
type: issue
body:
  https://sqlite-utils.datasette.io/en/stable/cli.html#csv-files-without-a-header-row

  > The first row of any CSV or TSV file is expected to contain the names of the columns in that file.

  > If your file does not include this row, you can use the `--no-headers` option to specify that the tool should not use that fist row as headers.

  > If you do this, the table will be created with column names called `untitled_1` and `untitled_2` and so on. You can then rename them using the `sqlite-utils transform ... --rename` command.

  It would be nice to be able to specify the column names when importing CSV/TSV without a header row, via an extra command line option.

  (renaming a column of a large table can take a long time, which makes it an inconvenient workaround)

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/474/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Qpn9R
title: Consider automatically cleaning up curly quotes in searches
state: open
created_at: 2022-08-27T16:35:25Z
updated_at: 2022-08-27T16:35:25Z
author_association: OWNER
type: issue
body:
  If your phone helpfully adds curly quotes for you then phrase searches against FTS won't work: “Rebecca Sugar”

  In regular (not `?_searchmode=raw` search mode Datasette could clean these up for you to help avoid that mistake.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1795/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QqAo1
title: table.default_values introspection property
state: closed
created_at: 2022-08-27T22:33:31Z
updated_at: 2022-08-27T22:44:46Z
closed_at: 2022-08-27T22:43:02Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Interesting challenge with `default_value`: I need to be able to tell if the default values passed to `.create()` differ from those in the database already.
  >
  > Introspecting that is a bit tricky:
  >
  > ```pycon
  > >>> import sqlite_utils
  > >>> db = sqlite_utils.Database(memory=True)
  > >>> db["blah"].create({"id": int, "name": str}, not_null=("name",), defaults={"name": "bob"})
  > <Table blah (id, name)>
  > >>> db["blah"].columns
  > [Column(cid=0, name='id', type='INTEGER', notnull=0, default_value=None, is_pk=0), Column(cid=1, name='name', type='TEXT', notnull=1, default_value="'bob'", is_pk=0)]
  > ```
  > Note how a default value of the Python string `bob` is represented in the results of `PRAGMA table_info()` as `default_value="'bob'"` - it's got single quotes added to it!
  > 
  > So comparing default values from introspecting the database needs me to first parse that syntax. This may require a new table introspection method.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/468#issuecomment-1229279539_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/475/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QqCWq
title: Release notes for 3.29
state: closed
created_at: 2022-08-27T23:21:21Z
updated_at: 2022-08-28T04:07:15Z
closed_at: 2022-08-28T04:07:03Z
author_association: OWNER
body: https://github.com/simonw/sqlite-utils/compare/3.28...104f37fa4d2e7e5999c1d829267b62c737f74d3e
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/476/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODEpn8M5Qq20Z
title: Problem with my user
state: open
created_at: 2022-08-28T16:59:37Z
updated_at: 2022-08-28T16:59:37Z
author_association: NONE
type: issue
body:
  If I call the program with:
      inaturalist-to-sqlite inaturalist.db ftricas
  the program exits with an error:
   `Importing 36 observations
  Traceback (most recent call last):
    File "/home/ftricas/.pyenv/versions/3.10.6/bin/inaturalist-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
      return self.main(*args, **kwargs)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1055, in main
      rv = self.invoke(ctx)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 760, in invoke
      return __callback(*args, **kwargs)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/inaturalist_to_sqlite/cli.py", line 51, in cli
      save_observation(observation, db)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/inaturalist_to_sqlite/utils.py", line 34, in save_observation
      db["observations"]
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/sqlite_utils/db.py", line 2965, in insert
      return self.insert_all(
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/sqlite_utils/db.py", line 3068, in insert_all
      self.create(
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/sqlite_utils/db.py", line 1564, in create
      self.db.create_table(
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/sqlite_utils/db.py", line 951, in create_table
      sql = self.create_table_sql(
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/sqlite_utils/db.py", line 765, in create_table_sql
      foreign_keys = self.resolve_foreign_keys(name, foreign_keys or [])
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/sqlite_utils/db.py", line 702, in resolve_foreign_keys
      other_table = table.guess_foreign_table(column)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/sqlite_utils/db.py", line 2061, in guess_foreign_table
      raise NoObviousTable(
  sqlite_utils.db.NoObviousTable: No obvious foreign key table for column 'taxon' - tried ['taxon', 'taxons']
  `
  If I call the program with your user everything seems to go well and then, I can call the program with my own user without problems. Moreover, I can call the program again with my own user and everything goes well now.

  Additional info, the command:
      
      sqlite-utils tables inaturalist.db

  shows that the correct name can be 'taxons'.

  There is another small problem with a warning:
       
     warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "

reactions:
  {"url": "https://api.github.com/repos/dogsheep/inaturalist-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODtX3eM497MOV
title: The program fails when the user has no submissions
state: open
created_at: 2022-08-28T17:25:45Z
updated_at: 2022-08-28T17:25:45Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/hacker-news-to-sqlite/pulls/5
type: pull
body:
  Tested with:
      
       hacker-news-to-sqlite user hacker-news.db fernand0

  Result:
  `
  Traceback (most recent call last):
    File "/home/ftricas/.pyenv/versions/3.10.6/bin/hacker-news-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
      return self.main(*args, **kwargs)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1055, in main
      rv = self.invoke(ctx)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/click/core.py", line 760, in invoke
      return __callback(*args, **kwargs)
    File "/home/ftricas/.pyenv/versions/3.10.6/lib/python3.10/site-packages/hacker_news_to_sqlite/cli.py", line 27, in user
      submitted = user.pop("submitted", None) or []
  AttributeError: 'NoneType' object has no attribute 'pop'
  `

  There is a problem of style with the patch (but not sure what to do) because with the new inicialization ( submitted = []) the part 

       or []

  is not needed. Maybe there is a more adequate way of doing this.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Qq-Bt
title: Conda Forge
state: closed
created_at: 2022-08-28T19:03:08Z
updated_at: 2022-09-07T03:46:55Z
closed_at: 2022-09-07T03:46:55Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hello! I have successfully put this package on to Conda Forge, and I have extending the invitation for the owner/maintainers of this package to be maintainers on Conda Forge as well. Let me know if you are interested! Thanks.
  https://github.com/conda-forge/sqlite-utils-feedstock

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/477/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5QrH0p
title: `sqlite-utils tables data.db table1 table2`
state: open
created_at: 2022-08-28T22:05:53Z
updated_at: 2022-08-28T22:22:35Z
author_association: OWNER
type: issue
body:
  The `sqlite-utils tables` command currently lists all tables.

  If you have a huge table in there then running it with `--counts` can get expensive, because of the huge table.

  Would be useful if it could accept an optional list of tables that it should execute against, as an alternative to the default of all of them.

  This should be a backwards compatible change. Current design is:  https://sqlite-utils.datasette.io/en/stable/cli-reference.html#tables

  ```
  Usage: sqlite-utils tables [OPTIONS] PATH

    List the tables in the database

    Example:

        sqlite-utils tables trees.db
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/478/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Qxexh
title: Research an upgrade to CodeMirror 6
state: closed
created_at: 2022-08-30T04:27:46Z
updated_at: 2023-07-03T04:58:21Z
closed_at: 2023-07-03T04:58:21Z
author_association: OWNER
type: issue
state_reason: completed
body:
  There are still a bunch of bugs in CodeMirror 5 that affect various mobile browsers - see Datasette Discord report here: https://discord.com/channels/823971286308356157/823971286941302908/1013878624992108645

  https://user-images.githubusercontent.com/9599/187349269-7b7c0c8c-3894-4810-82f0-de7c1eb940b3.mp4

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1796/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Qxpy5
title: OperationalError: cannot VACUUM from within a transaction
state: open
created_at: 2022-08-30T05:34:24Z
updated_at: 2022-08-30T05:34:24Z
author_association: CONTRIBUTOR
type: issue
body:
  Maybe when calling `.vacuum()` and other DB-level write-lock operations `sqlite_utils` could guard against this error message by automatically committing first?

  ```
       46 db["media"].optimize()  # type: ignore
  ---> 47 db.vacuum()

  File ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:1047, in Database.vacuum(self)
     1045 def vacuum(self):
     1046     "Run a SQLite ``VACUUM`` against the database."
  -> 1047     self.execute("VACUUM;")

  File ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:470, in Database.execute(self, sql, parameters)
      468     return self.conn.execute(sql, parameters)
      469 else:
  --> 470     return self.conn.execute(sql)

  OperationalError: cannot VACUUM from within a transaction
  ```

  It might also be nice to add a sentence or two about how transactions are committed on the [docs page](https://sqlite-utils.datasette.io/en/latest/python-api.html#detect-fts). When I was swapping out my sqlite3 code for this library it was nice that everything was pretty much drop-in but I was/am unsure what to do about the places I explicitly call `.commit()` in my code

  Related to https://github.com/simonw/sqlite-utils/issues/121

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/479/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4-B7Mc
title: search_sql add include_rank option
state: closed
created_at: 2022-08-30T09:10:29Z
updated_at: 2022-08-31T03:40:35Z
closed_at: 2022-08-31T03:40:35Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/480
type: pull
body:
  I haven't tested this yet but wanted to get a heads-up whether this kind of change would be useful or if I should just duplicate the function and tweak it within my code

  <!-- readthedocs-preview sqlite-utils start -->
  ----
  :books: Documentation preview :books:: https://sqlite-utils--480.org.readthedocs.build/en/480/

  <!-- readthedocs-preview sqlite-utils end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/480/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4-NhzQ
title: Bump black from 22.6.0 to 22.8.0
state: closed
created_at: 2022-09-01T13:25:14Z
updated_at: 2022-09-05T18:51:52Z
closed_at: 2022-09-05T18:51:52Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1797
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 22.6.0 to 22.8.0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>22.8.0</h2>
  <h3>Highlights</h3>
  <ul>
  <li>Python 3.11 is now supported, except for <em>blackd</em> as aiohttp does not support 3.11 as of publishing (<a href="https://github-redirect.dependabot.com/psf/black/issues/3234">#3234</a>)</li>
  <li>This is the last release that supports running <em>Black</em> on Python 3.6 (formatting 3.6 code will continue to be supported until further notice)</li>
  <li>Reword the stability policy to say that we may, in rare cases, make changes that affect code that was not previously formatted by <em>Black</em> (<a href="https://github-redirect.dependabot.com/psf/black/issues/3155">#3155</a>)</li>
  </ul>
  <h3>Stable style</h3>
  <ul>
  <li>Fix an infinite loop when using <code># fmt: on/off</code> in the middle of an expression or code block (<a href="https://github-redirect.dependabot.com/psf/black/issues/3158">#3158</a>)</li>
  <li>Fix incorrect handling of <code># fmt: skip</code> on colon (<code>:</code>) lines (<a href="https://github-redirect.dependabot.com/psf/black/issues/3148">#3148</a>)</li>
  <li>Comments are no longer deleted when a line had spaces removed around power operators (<a href="https://github-redirect.dependabot.com/psf/black/issues/2874">#2874</a>)</li>
  </ul>
  <h3>Preview style</h3>
  <ul>
  <li>Single-character closing docstring quotes are no longer moved to their own line as this is invalid. This was a bug introduced in version 22.6.0. (<a href="https://github-redirect.dependabot.com/psf/black/issues/3166">#3166</a>)</li>
  <li><code>--skip-string-normalization</code> / <code>-S</code> now prevents docstring prefixes from being normalized as expected (<a href="https://github-redirect.dependabot.com/psf/black/issues/3168">#3168</a>)</li>
  <li>When using <code>--skip-magic-trailing-comma</code> or <code>-C</code>, trailing commas are stripped from subscript expressions with more than 1 element (<a href="https://github-redirect.dependabot.com/psf/black/issues/3209">#3209</a>)</li>
  <li>Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside parentheses (<a href="https://github-redirect.dependabot.com/psf/black/issues/3162">#3162</a>)</li>
  <li>Fix a string merging/split issue when a comment is present in the middle of implicitly concatenated strings on its own line (<a href="https://github-redirect.dependabot.com/psf/black/issues/3227">#3227</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li><code>blackd</code> now supports enabling the preview style via the <code>X-Preview</code> header (<a href="https://github-redirect.dependabot.com/psf/black/issues/3217">#3217</a>)</li>
  </ul>
  <h3>Configuration</h3>
  <ul>
  <li>Black now uses the presence of debug f-strings to detect target version (<a href="https://github-redirect.dependabot.com/psf/black/issues/3215">#3215</a>)</li>
  <li>Fix misdetection of project root and verbose logging of sources in cases involving <code>--stdin-filename</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/3216">#3216</a>)</li>
  <li>Immediate <code>.gitignore</code> files in source directories given on the command line are now also respected, previously only <code>.gitignore</code> files in the project root and automatically discovered directories were respected (<a href="https://github-redirect.dependabot.com/psf/black/issues/3237">#3237</a>)</li>
  </ul>
  <h3>Documentation</h3>
  <ul>
  <li>Recommend using BlackConnect in IntelliJ IDEs (<a href="https://github-redirect.dependabot.com/psf/black/issues/3150">#3150</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Vim plugin: prefix messages with <code>Black: </code> so it's clear they come from Black (<a href="https://github-redirect.dependabot.com/psf/black/issues/3194">#3194</a>)</li>
  <li>Docker: changed to a /opt/venv installation + added to PATH to be available to non-root users (<a href="https://github-redirect.dependabot.com/psf/black/issues/3202">#3202</a>)</li>
  </ul>
  <h3>Output</h3>
  <ul>
  <li>Change from deprecated <code>asyncio.get_event_loop()</code> to create our event loop which removes DeprecationWarning (<a href="https://github-redirect.dependabot.com/psf/black/issues/3164">#3164</a>)</li>
  <li>Remove logging from internal <code>blib2to3</code> library since it regularly emits error logs about failed caching that can and should be ignored (<a href="https://github-redirect.dependabot.com/psf/black/issues/3193">#3193</a>)</li>
  </ul>
  <h3>Parser</h3>
  <ul>
  <li>Type comments are now included in the AST equivalence check consistently so accidental deletion raises an error. Though type comments can't be tracked when running on PyPy 3.7 due to standard library limitations. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2874">#2874</a>)</li>
  </ul>
  <h3>Performance</h3>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>22.8.0</h2>
  <h3>Highlights</h3>
  <ul>
  <li>Python 3.11 is now supported, except for <em>blackd</em> as aiohttp does not support 3.11 as
  of publishing (<a href="https://github-redirect.dependabot.com/psf/black/issues/3234">#3234</a>)</li>
  <li>This is the last release that supports running <em>Black</em> on Python 3.6 (formatting 3.6
  code will continue to be supported until further notice)</li>
  <li>Reword the stability policy to say that we may, in rare cases, make changes that
  affect code that was not previously formatted by <em>Black</em> (<a href="https://github-redirect.dependabot.com/psf/black/issues/3155">#3155</a>)</li>
  </ul>
  <h3>Stable style</h3>
  <ul>
  <li>Fix an infinite loop when using <code># fmt: on/off</code> in the middle of an expression or code
  block (<a href="https://github-redirect.dependabot.com/psf/black/issues/3158">#3158</a>)</li>
  <li>Fix incorrect handling of <code># fmt: skip</code> on colon (<code>:</code>) lines (<a href="https://github-redirect.dependabot.com/psf/black/issues/3148">#3148</a>)</li>
  <li>Comments are no longer deleted when a line had spaces removed around power operators
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/2874">#2874</a>)</li>
  </ul>
  <h3>Preview style</h3>
  <ul>
  <li>Single-character closing docstring quotes are no longer moved to their own line as
  this is invalid. This was a bug introduced in version 22.6.0. (<a href="https://github-redirect.dependabot.com/psf/black/issues/3166">#3166</a>)</li>
  <li><code>--skip-string-normalization</code> / <code>-S</code> now prevents docstring prefixes from being
  normalized as expected (<a href="https://github-redirect.dependabot.com/psf/black/issues/3168">#3168</a>)</li>
  <li>When using <code>--skip-magic-trailing-comma</code> or <code>-C</code>, trailing commas are stripped from
  subscript expressions with more than 1 element (<a href="https://github-redirect.dependabot.com/psf/black/issues/3209">#3209</a>)</li>
  <li>Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside
  parentheses (<a href="https://github-redirect.dependabot.com/psf/black/issues/3162">#3162</a>)</li>
  <li>Fix a string merging/split issue when a comment is present in the middle of implicitly
  concatenated strings on its own line (<a href="https://github-redirect.dependabot.com/psf/black/issues/3227">#3227</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li><code>blackd</code> now supports enabling the preview style via the <code>X-Preview</code> header (<a href="https://github-redirect.dependabot.com/psf/black/issues/3217">#3217</a>)</li>
  </ul>
  <h3>Configuration</h3>
  <ul>
  <li>Black now uses the presence of debug f-strings to detect target version (<a href="https://github-redirect.dependabot.com/psf/black/issues/3215">#3215</a>)</li>
  <li>Fix misdetection of project root and verbose logging of sources in cases involving
  <code>--stdin-filename</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/3216">#3216</a>)</li>
  <li>Immediate <code>.gitignore</code> files in source directories given on the command line are now
  also respected, previously only <code>.gitignore</code> files in the project root and
  automatically discovered directories were respected (<a href="https://github-redirect.dependabot.com/psf/black/issues/3237">#3237</a>)</li>
  </ul>
  <h3>Documentation</h3>
  <ul>
  <li>Recommend using BlackConnect in IntelliJ IDEs (<a href="https://github-redirect.dependabot.com/psf/black/issues/3150">#3150</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/psf/black/commit/2018e667a6a36ee3fbfa8041cd36512f92f60d49"><code>2018e66</code></a> Prepare docs for release 22.8.0 (<a href="https://github-redirect.dependabot.com/psf/black/issues/3248">#3248</a>)</li>
  <li><a href="https://github.com/psf/black/commit/0019261abcf6d9e564ba32d3cc15534b9026f29e"><code>0019261</code></a> Update stable branch after publishing to PyPI (<a href="https://github-redirect.dependabot.com/psf/black/issues/3223">#3223</a>)</li>
  <li><a href="https://github.com/psf/black/commit/7757078ecd84d349bb24ab61e79062ba50162ef9"><code>7757078</code></a> Improve &amp; update release process to reflect recent changes (<a href="https://github-redirect.dependabot.com/psf/black/issues/3242">#3242</a>)</li>
  <li><a href="https://github.com/psf/black/commit/767604e03f5e454ae5b5c268cd5831c672f46de8"><code>767604e</code></a> Use .gitignore files in the initial source directories (<a href="https://github-redirect.dependabot.com/psf/black/issues/3237">#3237</a>)</li>
  <li><a href="https://github.com/psf/black/commit/2c90480e1a102ab0fac57737d2ba5143d82abed7"><code>2c90480</code></a> Use strict mypy checking (<a href="https://github-redirect.dependabot.com/psf/black/issues/3222">#3222</a>)</li>
  <li><a href="https://github.com/psf/black/commit/ba618a307a30a119b4fafe526ebf7d5f092ba981"><code>ba618a3</code></a> Add parens around implicit string concatenations where it increases readabili...</li>
  <li><a href="https://github.com/psf/black/commit/c0cc19b5b3371842d696875897bebefebd5e1596"><code>c0cc19b</code></a> Delay worker count determination</li>
  <li><a href="https://github.com/psf/black/commit/afed2c01903465f9a486ac481a66aa3413cc1b01"><code>afed2c0</code></a> Load .gitignore and exclude regex at time of use</li>
  <li><a href="https://github.com/psf/black/commit/e269f44b25737360e0dc65379f889dfa931dc68a"><code>e269f44</code></a> Lazily import parallelized format modules</li>
  <li><a href="https://github.com/psf/black/commit/c47b91f513052cd39b818ea7c19716423c85c04e"><code>c47b91f</code></a> Fix misdetection of project root with <code>--stdin-filename</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/3216">#3216</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/psf/black/compare/22.6.0...22.8.0">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.6.0&new-version=22.8.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

  <!-- readthedocs-preview datasette start -->
  ----
  :books: Documentation preview :books:: https://datasette--1797.org.readthedocs.build/en/1797/

  <!-- readthedocs-preview datasette end -->

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1797/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5RCTRp
title: Parts of YAML file do not work when db name is "off"
state: closed
created_at: 2022-09-01T22:10:57Z
updated_at: 2022-09-02T00:02:53Z
closed_at: 2022-09-01T23:56:33Z
author_association: NONE
type: issue
state_reason: completed
body:
  I guess this issue is not very important and probably rare.

  To reproduce:
  * create and populate a db named `off.db`
  * in the yaml file, add any kind of information below `databases:\n   off:`
  * the data are not taken into account (because "off" is interpreted as "false")

  YAML file:
  ```yaml
  title: Some title
  description_html: |-
    <p>This is an experiment.</p>
  databases:
    off:
      tables:
        products_from_owners:
          title: products_from_owners*
          description_html: |-
            <p>Description</p>
  ```

  The result for http://xxxx.xxx/-/metadata gives:
  ```json
  {
      "title": "Some title",
      "description_html": "<p>This is an experiment.</p>",
      "databases": {
          "false": {
              "tables": {
                  "products_from_owners": {
                      "title": "products_from_owners*",
                      "description_html": "<p>Description</p>"
                  }
              }
          }
      }
  }
  ```
  => see the `"false"` instead of `"off"`.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1798/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


