id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
1808215339,I_kwDOBm6k_c5rxy0r,2104,Tables starting with an underscore should be treated as hidden,9599,open,0,,,2,2023-07-17T17:13:53Z,2023-07-18T22:41:37Z,,OWNER,,"Plugins can then take advantage of this pattern, for example:
- https://github.com/simonw/datasette-auth-tokens/pull/8",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2104/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1808116827,I_kwDOBm6k_c5rxaxb,2103,data attribute on Datasette tables exposing the primary key of the row,9599,open,0,,,0,2023-07-17T16:18:25Z,2023-07-17T16:18:25Z,,OWNER,,Maybe put it on the `<tr>` but probably better to go on the `td.type-pk`.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2103/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1765870617,I_kwDOBm6k_c5pQQwZ,2087,`--settings settings.json` option,9599,open,0,,,2,2023-06-20T17:48:45Z,2023-07-14T17:02:03Z,,OWNER,,"https://discord.com/channels/823971286308356157/823971286941302908/1120705940728066080

> May I add a request to the whole metadata / settings ? Allow to pass `--settings path/to/settings.json` instead of having to rely exclusively on directory mode to centralize settings (this would reflect the behavior of providing metadata)",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2087/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1803264272,I_kwDOBm6k_c5re6EQ,2101,alter: true support for JSON write API,9599,open,0,,,1,2023-07-13T15:24:11Z,2023-07-13T15:24:18Z,,OWNER,,"Requested here: https://discord.com/channels/823971286308356157/823971286941302908/1129034187073134642

> The former datasette-insert plugin had an option `?alter=1` to auto-add new columns. Does the JSON write API also have this?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2101/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
771608692,MDU6SXNzdWU3NzE2MDg2OTI=,14,UNIQUE constraint failed: workouts.id,1234956,open,0,,,5,2020-12-20T15:11:20Z,2023-07-10T14:46:52Z,,NONE,,"I'm getting an error on my initial attempt to import data:

```console
$ healthkit-to-sqlite 20201119\ healthkit\ export.zip healthkit.db
Importing from HealthKit  [###################################-]   98%  00:00:01
Traceback (most recent call last):
  File ""venv/bin/healthkit-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/cli.py"", line 57, in cli
    convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py"", line 34, in convert_xml_to_sqlite
    workout_to_db(el, db, zipfile)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py"", line 57, in workout_to_db
    pk = db[""workouts""].insert(record, alter=True, hash_id=""id"").last_pk
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1660, in insert
    return self.insert_all(
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1778, in insert_all
    self.insert_chunk(
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1588, in insert_chunk
    result = self.db.execute(query, params)
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 213, in execute
    return self.conn.execute(sql, parameters)
sqlite3.IntegrityError: UNIQUE constraint failed: workouts.id
```",197882382,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1795219865,I_kwDOCGYnMM5rAOGZ,566,`--no-headers` doesn't work on most formats,33625,open,0,,,2,2023-07-09T03:43:36Z,2023-07-09T04:13:35Z,,NONE,,"Version 3.33

```
sqlite-utils query library.db 'select asin from audible' --fmt plain --no-headers | head -3
asin
0062804006
0062891421
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/566/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1795187493,I_kwDODLZ_YM5rAGMl,12,Switch to pyproject.toml,9599,closed,0,,,2,2023-07-09T01:06:56Z,2023-07-09T01:19:43Z,2023-07-09T01:19:42Z,MEMBER,,First of my CLI tools to use https://til.simonwillison.net/python/pyproject,213286752,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
771202454,MDU6SXNzdWU3NzEyMDI0NTQ=,1153,"Use YAML examples in documentation by default, not JSON",9599,closed,0,,,22,2020-12-18T22:20:15Z,2023-07-08T20:09:48Z,2023-07-08T20:08:13Z,OWNER,,"YAML configuration is much better for multi-line strings, and I'm increasingly adding configuration options to Datasette that benefit from that - fragments of HTML in `description_html` or SQL queries used to configure things like https://github.com/simonw/datasette-atom for example.

Rather than confusing things by showing both in the documentation, I should switch all of the default examples to use YAML instead.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1153/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1794604602,PR_kwDOBm6k_c5U-akg,2096,Clarify docs for descriptions in metadata,15906,open,0,,,0,2023-07-08T01:57:58Z,2023-07-08T01:58:13Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2096,"G'day! I got confused while debugging, earlier today. That's on me, but it does strike me a little repetition in the metadata documentation might help those flicking around it rather than reading it from top to bottom. No worries if you think otherwise.

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2096.org.readthedocs.build/en/2096/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2096/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1794097871,I_kwDOBm6k_c5q78LP,2095,"Introduce ""dark mode"" CSS",3315059,open,0,,,0,2023-07-07T19:15:58Z,2023-07-07T19:15:58Z,,NONE,,Using [the CSS media query `prefers-color-scheme`](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-color-scheme) we can provide a dark-mode version of Datasette,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2095/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1785360409,I_kwDOCGYnMM5qanAZ,563,`--empty-null` option when importing CSV,9599,closed,0,,,1,2023-07-03T05:23:36Z,2023-07-03T05:44:43Z,2023-07-03T05:42:30Z,OWNER,,"CSV files with empty cells in (which come through as the empty string) are common and a bit gross.

Having an option that means ""and if it's an empty string store `null` instead) would be cool.

I brainstormed name options here https://chat.openai.com/share/c947b738-ee7d-419c-af90-bc84e90987da",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/563/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1355148385,I_kwDOBm6k_c5Qxexh,1796,Research an upgrade to CodeMirror 6,9599,closed,0,,,4,2022-08-30T04:27:46Z,2023-07-03T04:58:21Z,2023-07-03T04:58:21Z,OWNER,,"There are still a bunch of bugs in CodeMirror 5 that affect various mobile browsers - see Datasette Discord report here: https://discord.com/channels/823971286308356157/823971286941302908/1013878624992108645

https://user-images.githubusercontent.com/9599/187349269-7b7c0c8c-3894-4810-82f0-de7c1eb940b3.mp4

",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1796/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1784794489,I_kwDOCGYnMM5qYc15,562,Explore the intersection between sqlite-utils and dataclasses,9599,open,0,,,1,2023-07-02T19:23:08Z,2023-07-02T19:26:39Z,,OWNER,,"> Aside: this makes me think it might be cool if `sqlite-utils` had a way of working with dataclasses rather than just dicts, and knew how to create a SQLite table to match a dataclass and maybe how to code-generate dataclasses for a specific table schema (dynamically or even using code-generation that can be written to disk, for better editor integrations).

_Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1616742529_
            ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/562/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1783304750,I_kwDOBm6k_c5qSxIu,2094,JS Plugin Hooks for the Code Editor,15178711,open,0,,,0,2023-07-01T00:51:57Z,2023-07-01T00:51:57Z,,CONTRIBUTOR,,"When #2052 merges, I'd like to add support to add extensions/functions to the Datasette code editor. 

I'd eventually like to build a JS plugin for [`sqlite-docs`](https://github.com/asg017/sqlite-docs), to add things like:

- Inline documentation for tables/columns on hover
- Inline docs for custom functions that are loaded in
- More detailed autocomplete for tables/columns/functions

I did some hacking to see what this would look like, see here:

<img width=""1223"" alt=""image"" src=""https://github.com/simonw/datasette/assets/15178711/64f95cbc-1492-4365-896f-b88c6d08a649"">
<img width=""1223"" alt=""image"" src=""https://github.com/simonw/datasette/assets/15178711/73e602ba-5f45-417a-997e-5aea1738527a"">

There can be a new hook that allows JS plugins to add new ""extension"" in the CodeMirror editorview here:

https://github.com/simonw/datasette/blob/8cd60fd1d899952f1153460469b3175465f33f80/datasette/static/cm-editor-6.0.1.js#L25

Will need some more planning. For example, the Codemirror bundle in Datasette has functions that we could re-export for plugins to use (so we don't load 2 version of `""@codemirror/autocomplete""`, for example. ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2094/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1781047747,I_kwDOBm6k_c5qKKHD,2092,test_homepage intermittent failure,9599,closed,0,,,2,2023-06-29T15:20:37Z,2023-06-29T15:26:28Z,2023-06-29T15:24:13Z,OWNER,,"e.g. in https://github.com/simonw/datasette/actions/runs/5413590227/jobs/9839373852

```
=================================== FAILURES ===================================
________________________________ test_homepage _________________________________
[gw0] linux -- Python 3.7.17 /opt/hostedtoolcache/Python/3.7.17/x64/bin/python

ds_client = <datasette.app.DatasetteClient object at 0x7f85d271ef50>

    @pytest.mark.asyncio
    async def test_homepage(ds_client):
        response = await ds_client.get(""/.json"")
        assert response.status_code == 200
        assert ""application/json; charset=utf-8"" == response.headers[""content-type""]
        data = response.json()
        assert data.keys() == {""fixtures"": 0}.keys()
        d = data[""fixtures""]
        assert d[""name""] == ""fixtures""
        assert d[""tables_count""] == 24
        assert len(d[""tables_and_views_truncated""]) == 5
        assert d[""tables_and_views_more""] is True
        # 4 hidden FTS tables + no_primary_key (hidden in metadata)
        assert d[""hidden_tables_count""] == 6
        # 201 in no_primary_key, plus 6 in other hidden tables:
>       assert d[""hidden_table_rows_sum""] == 207, data
E       AssertionError: {'fixtures': {'color': '9403e5', 'hash': None, 'hidden_table_rows_sum': 0, 'hidden_tables_count': 6, ...}}
E       assert 0 == 207
```
My guess is that this is a timing error, where very occasionally the ""count rows but stop counting if it exceeds a time limit"" thing fails.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2092/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1781005740,I_kwDOBm6k_c5qJ_2s,2090,Adopt ruff for linting,9599,open,0,,,2,2023-06-29T14:56:43Z,2023-06-29T15:05:04Z,,OWNER,,https://beta.ruff.rs/docs/,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2090/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1780973290,I_kwDOBm6k_c5qJ37q,2089,codespell test failure,9599,closed,0,,,5,2023-06-29T14:40:10Z,2023-06-29T14:48:11Z,2023-06-29T14:48:10Z,OWNER,,"https://github.com/simonw/datasette/actions/runs/5413443676/jobs/9838999356
```
  codespell docs/*.rst --ignore-words docs/codespell-ignore-words.txt
  codespell datasette -S datasette/static --ignore-words docs/codespell-ignore-words.txt
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.9.17/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.17/x64/lib
docs/metadata.rst:192: displaing ==> displaying
```
This failure is legit, it found a spelling mistake: https://github.com/simonw/datasette/blob/ede62036180993dbd9d4e5d280fc21c183cda1c3/docs/metadata.rst#L192",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2089/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1756975532,PR_kwDOBm6k_c5S_5Jl,2083,Bump blacken-docs from 1.13.0 to 1.14.0,49699333,closed,0,,,0,2023-06-14T13:57:52Z,2023-06-29T14:31:55Z,2023-06-29T14:31:54Z,CONTRIBUTOR,simonw/datasette/pulls/2083,"Bumps [blacken-docs](https://github.com/asottile/blacken-docs) from 1.13.0 to 1.14.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href=""https://github.com/adamchainz/blacken-docs/blob/main/CHANGELOG.rst"">blacken-docs's changelog</a>.</em></p>
<blockquote>
<h2>1.14.0 (2023-06-13)</h2>
<ul>
<li>Support Python 3.12.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/4b94add9bc83d772ae5e31a679a18609c1750513""><code>4b94add</code></a> Version 1.14.0</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/292cb18828c73dfa5273c88ce261b453fb9fdf43""><code>292cb18</code></a> Support Python 3.12 (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/251"">#251</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/72ecf2c719027fd464f5561d4ab119f736e8a4ad""><code>72ecf2c</code></a> [pre-commit.ci] pre-commit autoupdate (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/249"">#249</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/ea8f1ca88f2f0a58371bb07dab9c647d56d5b3ed""><code>ea8f1ca</code></a> Upgrade requirements (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/248"">#248</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/9979d7a78be5b7749ebb000f10f4515e5dc9a574""><code>9979d7a</code></a> Upgrade requirements (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/247"">#247</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/7b71075ceb458be255e24da587c0275818b51faa""><code>7b71075</code></a> [pre-commit.ci] pre-commit autoupdate (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/245"">#245</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/6280868a8b944193e72436bb467749a21b790d7c""><code>6280868</code></a> [pre-commit.ci] pre-commit autoupdate (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/244"">#244</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/83efc37a18b2aea999985439841d644833cb9b90""><code>83efc37</code></a> Upgrade requirements (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/243"">#243</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/7aedc23ae3fa040c30e122d8beb66928a05d5320""><code>7aedc23</code></a> [pre-commit.ci] pre-commit autoupdate (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/242"">#242</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/d78d593d9a523c8d82d234b94d8ced690ee55fe6""><code>d78d593</code></a> [pre-commit.ci] pre-commit autoupdate (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/241"">#241</a>)</li>
<li>Additional commits viewable in <a href=""https://github.com/asottile/blacken-docs/compare/1.13.0...1.14.0"">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=blacken-docs&package-manager=pip&previous-version=1.13.0&new-version=1.14.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2083.org.readthedocs.build/en/2083/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2083/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1054244712,I_kwDOBm6k_c4-1n9o,1510,Datasette 1.0 documented template context (maybe via API docs),9599,open,0,,3268330,3,2021-11-15T23:23:58Z,2023-06-28T02:05:21Z,,OWNER,,Documented context plus protective unit tests. Goal is that custom templates built for 1.x will not break without a 2.x release.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1510/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1777548699,I_kwDOCGYnMM5p8z2b,561,`--stop-after` option for `insert` and `upsert` commands,9599,closed,0,,,1,2023-06-27T18:44:15Z,2023-06-27T18:50:09Z,2023-06-27T18:50:08Z,OWNER,,I found myself wanting to insert rows from a 849MB CSV file without processing the whole thing: https://huggingface.co/datasets/jerpint-org/HackAPrompt-Playground-Submissions/tree/main,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/561/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1773458985,PR_kwDOCGYnMM5T2mMb,560,Use sqlean if available in environment,9599,closed,0,,,10,2023-06-25T19:48:48Z,2023-06-26T08:21:00Z,2023-06-25T23:25:51Z,OWNER,simonw/sqlite-utils/pulls/560,"Refs:
- #559 

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--560.org.readthedocs.build/en/560/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/560/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
810618495,MDU6SXNzdWU4MTA2MTg0OTU=,235,Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified,6913891,closed,0,,,18,2021-02-17T23:33:23Z,2023-06-26T01:47:01Z,2023-06-25T23:25:53Z,NONE,,"Thanks for what seems like a truly great suite of libraries. I wanted to try out Datasette, but never got more than half way through your YouTube video with the SF tree dataset. Whenever I try to extract a column, I get a `sqlite3.OperationalError: table sqlite_master may not be modified` error from Python. This snippet reproduces the error on my system, Python 3.9.1 and sqlite-utils 3.5 on an M1 Macbook Pro running in rosetta mode:
```
curl ""https://data.nasa.gov/resource/y77d-th95.json"" | \
    sqlite-utils insert meteorites.db meteorites - --pk=id
sqlite-utils extract meteorites.db meteorites  recclass
```

I have tried googling the problem, but all I've found is that this *might* be a problem with the sqlite3 database running in defensive mode, but I definitely can't know for sure. Does the problem seem familiar to you?

",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/235/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1773450152,I_kwDOCGYnMM5ptLOo,559,sqlean support,9599,closed,0,,,0,2023-06-25T19:27:26Z,2023-06-25T23:25:53Z,2023-06-25T23:25:53Z,OWNER,,"If sqlean is available, use that.

Refs:
- https://github.com/nalgeon/sqlean.py/issues/1#issuecomment-1605707788

This will provide a good workaround for:
- #235 ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/559/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323223872,MDU6SXNzdWUzMjMyMjM4NzI=,260,Validate metadata.json on startup,9599,open,0,,,7,2018-05-15T13:42:56Z,2023-06-21T12:51:22Z,,OWNER,,"It's easy to misspell the name of a database or table and then be puzzled when the metadata settings silently fail.

To avoid this, let's sanity check the provided metadata.json on startup and quit with a useful error message if we find any obvious mistakes.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/260/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1764792125,I_kwDOBm6k_c5pMJc9,2086,Show information on startup in directory configuration mode,9599,open,0,,,0,2023-06-20T07:13:33Z,2023-06-20T07:13:33Z,,OWNER,,"https://discord.com/channels/823971286308356157/823971286941302908/1120516587036889098

> One thing that would be helpful would be message at launch indicating a metadata.json is getting picked up. I'm using directory mode and was editing the wrong file for awhile before I realize nothing I was doing was having any effect.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2086/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1762180409,I_kwDOBm6k_c5pCL05,2085,Interactive row selection in Datasette ,24938923,open,0,,,0,2023-06-18T08:29:45Z,2023-06-18T08:31:23Z,,NONE,,"Simon did a excellent [prototype](https://til.simonwillison.net/datasette/row-selection-prototype) of an interactive row selection in Datasette.

I hope this [functionality](https://camo.githubusercontent.com/3d4a0f31fb6a27fd279f809af5b53dc3b76faa63c7721e228951c5252b645a77/68747470733a2f2f7374617469632e73696d6f6e77696c6c69736f6e2e6e65742f7374617469632f323032332f6461746173657474652d7069636b65722e676966) can be turned into a Datasette plugin.
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2085/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1761613778,I_kwDOBm6k_c5pABfS,2084,Support facets for columns that contain timestamps,19492893,open,0,,,0,2023-06-17T03:33:54Z,2023-06-17T03:33:54Z,,NONE,,"
Django has this very nice filter for datetime fields -

<img width=""176"" alt=""image"" src=""https://github.com/simonw/datasette/assets/19492893/3c66d7c4-1579-4d30-8f08-89d111f4566e"">

It would be nice to have something similar to facet by a field that contains a timestamp in datasette too - Which doesn't seem to do anything with timestamps right now...

<img width=""283"" alt=""image"" src=""https://github.com/simonw/datasette/assets/19492893/069083e4-13f5-4b28-9473-a7b9d48839ea"">
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2084/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1655860104,I_kwDOCGYnMM5ismuI,535,rows: --transpose or psql extended view-like functionality,7908073,closed,0,,,2,2023-04-05T15:37:33Z,2023-06-15T08:39:49Z,2023-06-14T22:05:28Z,CONTRIBUTOR,,"It would be nice if the rows subcommand had a flag, perhaps called `--transpose` which would print in long form instead of wide. Similar to extended display mode in psql (`\x`)

In other words instead of this:

```
sqlite-utils rows  --limit 5 --fmt github track_metadata.db songs
```

| track_id           | title             | song_id            | release                              | artist_id          | artist_mbid                          | artist_name      |   duration |   artist_familiarity |   artist_hotttnesss |   year |   track_7digitalid |   shs_perf |   shs_work |
|--------------------|-------------------|--------------------|--------------------------------------|--------------------|--------------------------------------|------------------|------------|----------------------|---------------------|--------|--------------------|------------|------------|
| TRMMMYQ128F932D901 | Silent Night      | SOQMMHC12AB0180CB8 | Monster Ballads X-Mas                | ARYZTJS1187B98C555 | 357ff05d-848a-44cf-b608-cb34b5701ae5 | Faster Pussy cat |    252.055 |             0.649822 |            0.394032 |   2003 |            7032331 |         -1 |          0 |
| TRMMMKD128F425225D | Tanssi vaan       | SOVFVAK12A8C1350D9 | Karkuteillä                          | ARMVN3U1187FB3A1EB | 8d7ef530-a6fd-4f8f-b2e2-74aec765e0f9 | Karkkiautomaatti |    156.551 |             0.439604 |            0.356992 |   1995 |            1514808 |         -1 |          0 |
| TRMMMRX128F93187D9 | No One Could Ever | SOGTUKN12AB017F4F1 | Butter                               | ARGEKB01187FB50750 | 3d403d44-36ce-465c-ad43-ae877e65adc4 | Hudson Mohawke   |    138.971 |             0.643681 |            0.437504 |   2006 |            6945353 |         -1 |          0 |
| TRMMMCH128F425532C | Si Vos Querés     | SOBNYVR12A8C13558C | De Culo                              | ARNWYLR1187B9B2F9C | 12be7648-7094-495f-90e6-df4189d68615 | Yerba Brava      |    145.058 |             0.448501 |            0.372349 |   2003 |            2168257 |         -1 |          0 |
| TRMMMWA128F426B589 | Tangle Of Aspens  | SOHSBXH12A8C13B0DF | Rene Ablaze Presents Winter Sessions | AREQDTE1269FB37231 |                                      | Der Mystic       |    514.298 |             0        |            0        |      0 |            2264873 |         -1 |          0 |


The output would look something like this:

```
$ for col in (sqlite-columns track_metadata.db songs)
    sqlite-utils --fmt github track_metadata.db ""select $col from songs order by rowid desc limit 5""
end
```

| track_id           |
|--------------------|
| TRYYYVU12903CD01E3 |
| TRYYYDJ128F9310A21 |
| TRYYYMG128F4260ECA |
| TRYYYJO128F426DA37 |
| TRYYYUS12903CD2DF0 |
| title                               |
|-------------------------------------|
| Fernweh feat. Sektion Kuchikäschtli |
| Faraday                             |
| Novemba                             |
| Jago Chhadeo                        |
| O Samba Da Vida                     |
| song_id            |
|--------------------|
| SOWXJXQ12AB0189F43 |
| SOLXGOR12A81C21EB7 |
| SOHODZI12A8C137BB3 |
| SOXQYIQ12A8C137FBB |
| SOTXAME12AB018F136 |
| release                         |
|---------------------------------|
| So Oder So                      |
| The Trance Collection Vol. 2    |
| Dub_Connected: electronic music |
| Naale Baba Lassi Pee Gya        |
| Pacha V.I.P.                    |
| artist_id          |
|--------------------|
| AR7PLM21187B990D08 |
| ARCMCOK1187B9B1073 |
| ARZ3R6M1187B9AF750 |
| ART5FZD1187B9A7FCF |
| AR7Z4J81187FB3FC59 |
| artist_mbid                          |
|--------------------------------------|
| 3af2b07e-c91c-4160-9bda-f0b9e3144ed3 |
| 4ac5f3de-c5ad-475e-ad50-41f1ef9dba20 |
| 8b97e9c8-61f5-4615-9a96-276f24204e34 |
| 2357c400-9109-42b6-b3fe-9e2d9f8e3872 |
| 9d50cb20-7e42-45cc-b0dd-154c3e92a577 |
| artist_name    |
|----------------|
| Texta          |
| Elude          |
| Gabriel Le Mar |
| Kuldeep Manak  |
| Kiko Navarro   |
|   duration |
|------------|
|    295.079 |
|    484.519 |
|    553.038 |
|    244.166 |
|    217.443 |
|   artist_familiarity |
|----------------------|
|             0.552977 |
|             0.403668 |
|             0.556918 |
|             0.4015   |
|             0.528617 |
|   artist_hotttnesss |
|---------------------|
|            0.454869 |
|            0.256935 |
|            0.336914 |
|            0.374866 |
|            0.411595 |
|   year |
|--------|
|   2004 |
|      0 |
|      0 |
|      0 |
|      0 |
|   track_7digitalid |
|--------------------|
|            8486723 |
|            5472456 |
|            2219291 |
|            1632096 |
|            7522478 |
|   shs_perf |
|------------|
|         -1 |
|         -1 |
|         -1 |
|         -1 |
|         -1 |
|   shs_work |
|------------|
|          0 |
|          0 |
|          0 |
|          0 |
|          0 |
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/535/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1581090327,I_kwDOCGYnMM5ePYYX,529,Microsoft line endings,7908073,closed,0,,,1,2023-02-12T02:20:48Z,2023-06-14T23:12:12Z,2023-06-14T23:11:47Z,CONTRIBUTOR,,"sqlite-utils prints `\r\n` but [it should probably](https://devblogs.microsoft.com/commandline/extended-eol-in-notepad/) print `\n` (unless the platform is detected as Windows?)

It has tripped me up a few times when piping the output of sqlite-utils to other programs:

```
$ sqlite-utils --no-headers --csv  ~/lb/fs/d.db 'select path from media limit 1' | cat -A
/mnt/d7/file^M$
$ sqlite-utils --no-headers --csv  ~/lb/fs/d.db 'select path from media limit 1' | tr -d '\r' | cat -A
/mnt/d7/file$
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/529/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1383646615,I_kwDOCGYnMM5SeMWX,491,Ability to merge databases and tables,8904453,open,0,,,7,2022-09-23T11:10:55Z,2023-06-14T22:14:24Z,,NONE,,"Hi! Let me firstly say that I am a big fan of your work -- I follow your tweets and blog posts with great interest 😄.

Now onto the matter at hand: I think it would be great if `sqlite-utils` included a `merge` or `combine` command, with the purpose of combining different SQLite databases into a single SQLite database. This way, the newly ""merged"" database would contain all differently named tables contained in the databases to be merged as-is, as well a concatenation of all tables of the same name.

This could look something like this:

```bash
sqlite-utils merge cats.db dogs.db > animals.db
```

I imagine this is rather straightforward if all databases involved in the merge contain differently named tables (i.e. no chance of conflicts), but things get slightly more complicated if two or more of the databases to be merged contain tables with the same name. Not only do you have to ""do something"" with the primary key(s), but these tables could also simply have different schemas (and therefore be incompatible for concatenation to begin with).

Anyhow, I would love your thoughts on this, and, if you are open to it, work together on the design and implementation!",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/491/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1733198948,I_kwDOCGYnMM5nToRk,555,Filter table by a large bunch of ids,10843208,open,0,,,1,2023-05-31T00:29:51Z,2023-06-14T22:01:57Z,,NONE,,"Hi! this might be a question related to both SQLite & sqlite-utils, and you might be more experienced with them.

I have a large bunch of ids, and I'm wondering which is the best way to query them in terms of performance, and simplicity if possible.

The naive approach would be something like `select * from table where rowid in (?, ?, ?...)` but that wouldn't scale if ids are >1k.

Another approach might be creating a temp table, or in-memory db table, insert all ids in that table and then join with the target one.

I failed to attach an in-memory db both using sqlite-utils, and plain sql's execute(), so my closest approach is something like,

```python
def filter_existing_video_ids(video_ids):
    db = get_db()  # contains a ""videos"" table
    db.execute(""CREATE TEMPORARY TABLE IF NOT EXISTS tmp (video_id TEXT NOT NULL PRIMARY KEY)"")
    db[""tmp""].insert_all([{""video_id"": video_id} for video_id in video_ids])
    for row in db[""tmp""].rows_where(""video_id not in (select video_id from videos)""):
        yield row[""video_id""]
    db[""tmp""].drop()
```

That kinda worked, I couldn't find an option in sqlite-utils's `create_table()` to tell it's a temporary table. Also, `tmp` table is not dropped finally, neither using `.drop()` despite being created with the keyword `TEMPORARY`. I believe it should be automatically dropped after connection/session ends though I read.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/555/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1740150327,I_kwDOCGYnMM5nuJY3,557,Aliased ROWID option for tables created from alter=True commands,7908073,closed,0,,,2,2023-06-04T05:29:28Z,2023-06-14T06:09:21Z,2023-06-05T19:26:26Z,CONTRIBUTOR,,"> If you use INTEGER PRIMARY KEY column, the VACUUM does not change the values of that column. However, if you use unaliased rowid, the VACUUM command will reset the rowid values.

ROWID should never be used with foreign keys but the simple act of aliasing rowid to id (which is what happens when one does `id integer primary key` DDL) makes it OK.

It would be convenient if there were more options to use a string column (eg. filepath) as the PK, and be able to use it during upserts, but when creating a foreign key, to create an integer column which aliases rowid

I made an attempt to switch to integer primary keys here but it is not going well... In my usecase the path column is a business key. Yes, it should be as simple as including the `id` column in any select statement where I plan on using `upsert` but it would be nice if this could be abstracted away somehow  https://github.com/chapmanjacobd/library/commit/788cd125be01d76f0fe2153335d9f6b21db1343c

https://github.com/chapmanjacobd/library/actions/runs/5173602136/jobs/9319024777",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/557/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1751214236,I_kwDOC8SPRc5oYWic,36,Getting sqlite_master may not be modified when creating dogsheep index,8711912,open,0,,,0,2023-06-11T03:21:53Z,2023-06-11T03:21:53Z,,NONE,,"When creating a `dogsheep` index from `config.yml` file on pocket.db (created using pocket-to-sqlite), I am getting this error

```
Traceback (most recent call last):
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/bin/dogsheep-beta"", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py"", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/click/core.py"", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/dogsheep_beta/cli.py"", line 36, in index
    run_indexer(
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/dogsheep_beta/utils.py"", line 32, in run_indexer
    ensure_table_and_indexes(db, tokenize)
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/dogsheep_beta/utils.py"", line 91, in ensure_table_and_indexes
    table.add_foreign_key(*fk)
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/sqlite_utils/db.py"", line 2155, in add_foreign_key
    self.db.add_foreign_keys([(self.name, column, other_table, other_column)])
  File ""/Users/khushmeeet/.pyenv/versions/3.11.2/lib/python3.11/site-packages/sqlite_utils/db.py"", line 1116, in add_foreign_keys
    cursor.execute(
sqlite3.OperationalError: table sqlite_master may not be modified
```

Command I ran to get this error
```
dogsheep-beta index pocket.db config.yml
```

Dogsheep version
```
dogsheep-beta, version 0.10.2
```

Python version 
```
Python 3.11.2
```",197431109,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/36/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1740026046,I_kwDOCGYnMM5ntrC-,556,Support storing incrementally piped values,601708,open,0,,,1,2023-06-04T00:45:23Z,2023-06-04T01:21:15Z,,CONTRIBUTOR,,"I'm trying to use sqlite-utils to data generated incrementally. There are a few
aspects of this that I don't currently know how to handle. I would like an option
to apply writes incrementally, line-by-line as they are received. I would like an
option to echo incremental progress. And, it would be nice to have 

In particular, I'm using CoreLocationCLI -w -j to generate, newline-delimited JSON.

One variant of the command 

`stdbuf -oL CoreLocationCLI -w -j | pee 'sqlite-utils insert loc.db loc -' nl`

`pee`, from `moreutils`, is like `tee` but spawns and pipes to the processes
created by invoking each of its arguments, so, for gratuitous demonstration,
`pee 'sponge out.log' cat` would behave like `tee`.

It looks like I can get what I want with:
`stdbuf -oL CoreLocationCLI -w -j | while read line; do <<<""$line"" sqlite-utils insert loc.db loc -; echo ""$line""; done | nl`




",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/556/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1734786661,PR_kwDOBm6k_c5R0fcK,2082,Catch query interrupted on facet suggest row count,10843208,open,0,,,0,2023-05-31T18:42:46Z,2023-05-31T18:45:26Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2082,"Just like facet's `suggest()` is trapping `QueryInterrupted` for facet columns, we also need to trap `get_row_count()`, which can reach timeout if database tables are big enough. 

I've included `get_columns()` inside the block as that's just another query, despite it's a really cheap one and might never raise the exception.

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2082.org.readthedocs.build/en/2082/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2082/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1727478903,I_kwDOBm6k_c5m9zx3,2081,Update Endpoints defined in metadata throws 403 Forbidden after a while,15085007,open,0,,,0,2023-05-26T11:52:30Z,2023-05-26T11:52:30Z,,NONE,,"Hello. I expose an endpoint to update `tasks`:
```
{
    ""title"": ""My Datasette Instance"",
    ""databases"": {
        ""tasks"": {
            ""queries"": {
                ""update_task"": {
                    ""sql"": ""UPDATE tasks SET status = :status, result = :result, systemMessage = :systemMessage WHERE queueID = :queueID"",
                    ""write"": true,
                    ""on_success_message"": ""Task updated"",
                    ""on_success_redirect"": ""/tasks/tasks.json"",
                    ""on_error_message"": ""Task update failed"",
                    ""on_error_redirect"": ""/tasks.json"",
                    ""params"": [""queueID"", ""taskData"", ""status"", ""result"", ""systemMessage""]
                }
            }
        }
    }
}
```

This works really well! But after a while, the Datasette Instanz answers with **403 Forbidden**.
I have to delete the database and recreate it in order to work again.

Any help here? (´。＿。｀)",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2081/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1726236847,I_kwDOBm6k_c5m5Eiv,2078,Resolve the difference between `wrap_view()` and `BaseView`,9599,closed,0,,,16,2023-05-25T17:44:32Z,2023-05-26T00:18:46Z,2023-05-26T00:18:46Z,OWNER,,"There are two patterns for implementing views in Datasette at the moment. I want to combine those.

Part of:
- #2053",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2078/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1726603778,PR_kwDOBm6k_c5RYvTU,2080,New View base class,9599,closed,0,,,3,2023-05-25T23:22:55Z,2023-05-26T00:18:45Z,2023-05-26T00:18:44Z,OWNER,simonw/datasette/pulls/2080,"Refs:

- #2078

TODO:

- [x] Teach router layer how to handle this
- [x] Use it for something

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2080.org.readthedocs.build/en/2080/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2080/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1726531350,I_kwDOBm6k_c5m6McW,2079,Datasette should serve Access-Control-Max-Age,9599,closed,0,,,8,2023-05-25T21:50:50Z,2023-05-25T22:56:28Z,2023-05-25T22:08:35Z,OWNER,,"Currently the CORS headers served are:

https://github.com/simonw/datasette/blob/9584879534ff0556e04e4c420262972884cac87b/datasette/utils/__init__.py#L1139-L1143

Serving `Access-Control-Max-Age: 600` would allow browsers to cache that for 10 minutes, avoiding additional CORS pre-flight OPTIONS requests during that time.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2079/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1720096994,I_kwDOCGYnMM5mhpji,554,"`IndexError` when doing `.insert(..., pk='id')` after `insert_all`",1231935,open,0,,,1,2023-05-22T17:13:02Z,2023-05-22T17:18:33Z,,NONE,,"I believe this is related to https://github.com/simonw/sqlite-utils/issues/98.

When `pk` is specified by table A's `insert` call, it throws an index error if a different table has written a row with a higher rowid than exists in the first table. Here's a basic example:

```py
from sqlite_utils import Database


def test_pk_for_insert(fresh_db):
    user = {""id"": ""abc"", ""name"": ""david""}

    fresh_db[""users""].insert(user, pk=""id"")

    fresh_db[""comments""].insert_all(
        [
            {""id"": ""def"", ""text"": ""ok""},
            {""id"": ""ghi"", ""text"": ""great""},
        ],
    )

    fresh_db[""users""].insert(
        user,
        ignore=True,
        # BUG: when specifying pk on the second insert call 
        # db.py goes into a block it doesn't expect and we get the error
        pk=""id"",
    )


if __name__ == ""__main__"":
    db = Database(""bug.db"")
    if db[""users""].exists():
        raise ValueError(
            ""bug only shows on a new database - remove bug.db before running the script""
        )
    test_pk_for_insert(db)
```

The error is:

```py
  File ""/Users/david/projects/reddit-to-sqlite/.venv/lib/python3.11/site-packages/sqlite_utils/db.py"", line 2960, in insert_chunk
    row = list(self.rows_where(""rowid = ?"", [self.last_rowid]))[0]
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
```

The issue is in this block: 

https://github.com/simonw/sqlite-utils/blob/2747257a3334d55e890b40ec58fada57ae8cfbfd/sqlite_utils/db.py#L2954-L2958

relevant locals are:

- `pk`: `'id'`
- `result.lastrowid`: `2`

What's most interesting is the comment `# self.last_rowid will be 0 if a ""INSERT OR IGNORE"" happened`, which doesn't seem to be the case here. ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/554/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1718612569,I_kwDOCGYnMM5mb_JZ,552,Document how to setup shell auto-completion,9599,closed,0,,,1,2023-05-21T19:20:41Z,2023-05-21T21:05:16Z,2023-05-21T21:03:40Z,OWNER,,"https://click.palletsprojects.com/en/8.1.x/shell-completion/

This works for `zsh`:

    eval ""$(_SQLITE_UTILS_COMPLETE=zsh_source sqlite-utils)""

This will probably work for `bash`:

    eval ""$(_SQLITE_UTILS_COMPLETE=bash_source sqlite-utils)""

Need to add this to the installation docs here: https://sqlite-utils.datasette.io/en/stable/installation.html - along with the pattern for adding that to `.zshrc` or whatever.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/552/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718607907,I_kwDOCGYnMM5mb-Aj,551,Make as many examples in the CLI docs as possible copy-and-pastable,9599,closed,0,,,6,2023-05-21T19:04:10Z,2023-05-21T21:04:04Z,2023-05-21T20:57:24Z,OWNER,,"e.g. in this section:

https://sqlite-utils.datasette.io/en/stable/cli.html#running-queries-directly-against-csv-or-json

<img width=""771"" alt=""image"" src=""https://github.com/simonw/sqlite-utils/assets/9599/3935111d-441e-4edb-9100-ce210117dbe5"">

The little copy button will also copy the `$ ` which breaks the examples when copied.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/551/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718635018,PR_kwDOCGYnMM5Q9lY4,553,Reformatted CLI examples in docs,9599,closed,0,,,2,2023-05-21T20:44:34Z,2023-05-21T20:57:27Z,2023-05-21T20:57:23Z,OWNER,simonw/sqlite-utils/pulls/553,"Refs:
- #551

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--553.org.readthedocs.build/en/553/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/553/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1718517882,I_kwDOCGYnMM5mboB6,545,Try out Trogon for a tui interface,9599,closed,0,,,6,2023-05-21T14:08:25Z,2023-05-21T19:33:13Z,2023-05-21T18:41:58Z,OWNER,,https://github.com/Textualize/trogon,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/545/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718595700,I_kwDOCGYnMM5mb7B0,550,AttributeError: 'EntryPoints' object has no attribute 'get' for flake8 on Python 3.7,9599,closed,0,,,3,2023-05-21T18:24:39Z,2023-05-21T18:42:25Z,2023-05-21T18:41:58Z,OWNER,,"https://github.com/simonw/sqlite-utils/actions/runs/5039064797/jobs/9036965488

```
Traceback (most recent call last):
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/bin/flake8"", line 8, in <module>
    sys.exit(main())
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/cli.py"", line 22, in main
    app.run(argv)
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py"", line 363, in run
    self._run(argv)
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py"", line 350, in _run
    self.initialize(argv)
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py"", line 330, in initialize
    self.find_plugins(config_finder)
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/main/application.py"", line 153, in find_plugins
    self.check_plugins = plugin_manager.Checkers(local_plugins.extension)
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/plugins/manager.py"", line 357, in __init__
    self.namespace, local_plugins=local_plugins
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/plugins/manager.py"", line 238, in __init__
    self._load_entrypoint_plugins()
  File ""/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/flake8/plugins/manager.py"", line 254, in _load_entrypoint_plugins
    eps = importlib_metadata.entry_points().get(self.namespace, ())
AttributeError: 'EntryPoints' object has no attribute 'get'
Error: Process completed with exit code 1.
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/550/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718586377,PR_kwDOCGYnMM5Q9cAv,549,TUI powered by Trogon,9599,closed,0,,,3,2023-05-21T17:55:42Z,2023-05-21T18:42:00Z,2023-05-21T18:41:56Z,OWNER,simonw/sqlite-utils/pulls/549,"Refs:
- #545

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--549.org.readthedocs.build/en/549/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/549/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1718576761,I_kwDOCGYnMM5mb2Z5,548,analyze-tables should validate provide --column names,9599,closed,0,,,1,2023-05-21T17:20:24Z,2023-05-21T17:35:52Z,2023-05-21T17:35:52Z,OWNER,,"Noticed this while testing:
- #547 

If you pass a non-existent column to `-c/--column` you don't get an error message.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/548/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718572201,I_kwDOCGYnMM5mb1Sp,547,No need to show common values if everything is null,9599,closed,0,,,1,2023-05-21T17:05:07Z,2023-05-21T17:19:21Z,2023-05-21T17:19:21Z,OWNER,,"Noticed this:

```
% sqlite-utils analyze-tables content.db repos -c delete_branch_on_merge --common-limit 20 --no-least
repos.delete_branch_on_merge: (1/1)

  Total rows: 158
  Null rows: 158
  Blank rows: 0

  Distinct values: 0

  Most common:
    158: None
```

The `158: None` there is duplicate information considering we already know there are 158/158 null rows.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/547/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718515590,I_kwDOCGYnMM5mbneG,544,New options for analyze-tables --common-limit --no-most and --no-least,9599,closed,0,,,2,2023-05-21T14:03:19Z,2023-05-21T17:03:06Z,2023-05-21T16:19:31Z,OWNER,,"The ""least common"" section is frequently uninteresting, especially for huge tables with a large number of repeated-once values.

    sqlite-utils analyze-tables content.db repos --common-limit 20 --no-least",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/544/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718550688,PR_kwDOCGYnMM5Q9VH0,546,"Analyze tables options: --common-limit, --no-most, --no-least",9599,closed,0,,,2,2023-05-21T15:54:39Z,2023-05-21T16:19:30Z,2023-05-21T16:19:30Z,OWNER,simonw/sqlite-utils/pulls/546,"Refs #544

- [x] Documentation for CLI options
- [x] Documentation for new Python API parameters:  `most_common: bool` and `least_common: bool`
- [x] Tests for CLI
- [x] Tests for Python API",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/546/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1715468032,PR_kwDOBm6k_c5QzEAM,2076,Datsette gpt plugin,130708713,open,0,,,0,2023-05-18T11:22:30Z,2023-05-18T11:22:45Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2076,"

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2076.org.readthedocs.build/en/2076/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2076/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1124731464,I_kwDOCGYnMM5DCgpI,399,"Make it easier to insert geometries, with documentation and maybe code",9599,open,0,,,25,2022-02-05T00:11:26Z,2023-05-16T03:11:52Z,,OWNER,,"In playing with the new SpatiaLite helpers from #385 I noticed that actually populating geometry columns is still a little bit tricky. Here's what I ended up doing:

```python
import httpx, sqlite_utils
db = sqlite_utils.Database(""/tmp/spatial.db"")
attractions = httpx.get(""https://latest.datasette.io/fixtures/roadside_attractions.json?_shape=array"").json()
db[""attractions""].insert_all(attractions, pk=""pk"")

# Schema of that table is now:
# CREATE TABLE [attractions] (
#    [pk] INTEGER PRIMARY KEY,
#    [name] TEXT,
#    [address] TEXT,
#    [latitude] FLOAT,
#    [longitude] FLOAT
# )

db.init_spatialite()
db[""attractions""].add_geometry_column(""point"", ""POINT"")

db.execute(""""""
    update attractions set point = GeomFromText(
      'POINT(' || longitude || ' ' || latitude || ')', 4326
    )
"""""")
```
That last line took some figuring out - especially the need for the SRID of `4326`, without which I got this error:

> `IntegrityError: attractions.point violates Geometry constraint [geom-type or SRID not allowed]`

It would be good to both document this in more detail, but ideally also to come up with a more obvious pattern for inserting common types of spatial data.

Also related:
- #398
- #79",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/399/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1690842199,PR_kwDOBm6k_c5PgNaA,2068,Bump sphinx from 6.1.3 to 7.0.0,49699333,closed,0,,,1,2023-05-01T13:58:46Z,2023-05-15T13:59:38Z,2023-05-15T13:59:36Z,CONTRIBUTOR,simonw/datasette/pulls/2068,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 7.0.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/releases"">sphinx's releases</a>.</em></p>
<blockquote>
<h2>v7.0.0</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>v7.0.0rc1</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>v6.2.1</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>v6.2.0</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/blob/master/CHANGES"">sphinx's changelog</a>.</em></p>
<blockquote>
<h1>Release 7.0.0 (released Apr 29, 2023)</h1>
<h2>Incompatible changes</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11359"">#11359</a>: Remove long-deprecated aliases for <code>MecabSplitter</code> and
<code>DefaultSplitter</code> in <code>sphinx.search.ja</code>.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11360"">#11360</a>: Remove deprecated <code>make_old_id</code> functions in domain object
description classes.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11363"">#11363</a>: Remove the Setuptools integration (<code>build_sphinx</code> hook in
<code>setup.py</code>).</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11364"">#11364</a>: Remove deprecated <code>sphinx.ext.napoleon.iterators</code> module.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11365"">#11365</a>: Remove support for the <code>jsdump</code> format in <code>sphinx.search</code>.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11366"">#11366</a>: Make <code>locale</code> a required argument to
<code>sphinx.util.i18n.format_date()</code>.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11370"">#11370</a>: Remove deprecated <code>sphinx.util.stemmer</code> module.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11371"">#11371</a>: Remove deprecated <code>sphinx.pycode.ast.parse()</code> function.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11372"">#11372</a>: Remove deprecated <code>sphinx.io.read_doc()</code> function.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11373"">#11373</a>: Removed deprecated <code>sphinx.util.get_matching_files()</code> function.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11378"">#11378</a>: Remove deprecated <code>sphinx.util.docutils.is_html5_writer_available()</code>
function.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11379"">#11379</a>: Make the <code>env</code> argument to <code>Builder</code> subclasses required.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11380"">#11380</a>: autosummary: Always emit grouped import exceptions.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11381"">#11381</a>: Remove deprecated <code>style</code> key for HTML templates.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11382"">#11382</a>: Remove deprecated <code>sphinx.writers.latex.LaTeXTranslator.docclasses</code>
attribute.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11383"">#11383</a>: Remove deprecated <code>sphinx.builders.html.html5_ready</code> and
<code>sphinx.builders.html.HTMLTranslator</code> attributes.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11385"">#11385</a>: Remove support for HTML 4 output.</li>
</ul>
<h1>Release 6.2.1 (released Apr 25, 2023)</h1>
<h2>Bugs fixed</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11355"">#11355</a>: Revert the default type of :confval:<code>nitpick_ignore</code> and
:confval:<code>nitpick_ignore_regex</code> to <code>list</code>.</li>
</ul>
<h1>Release 6.2.0 (released Apr 23, 2023)</h1>
<h2>Dependencies</h2>
<ul>
<li>Require Docutils 0.18.1 or greater.</li>
</ul>
<h2>Incompatible changes</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/d568b2f4f7cca743fcbf70814d15602d8129b790""><code>d568b2f</code></a> Bump to 7.0.0 final</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/ff79edf353f5cc6e02036f58e0295dc704c5e681""><code>ff79edf</code></a> Remove <code>jsdump</code> references post removal</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/1a5133ac391931d8e0e2a271fe8ea9c505176908""><code>1a5133a</code></a> Bump to 7.0.0rc1 final</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/5795fc7f9f7a5fe968cd11b961fc33fbd952a152""><code>5795fc7</code></a> Update <code>sphinx.deprecation</code> for Sphinx 7.0 (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11386"">#11386</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/6202087f912a2ae32808865072f884ea44cd31cb""><code>6202087</code></a> Add a note to CHANGES for PR 11385</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/ad473730a3763f241e9aaea1a87d1893f01b86fd""><code>ad47373</code></a> Remove HTML 4 support (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11385"">#11385</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/3e3251d3ba4f1e5fa6d34d995505f4d3ace5acad""><code>3e3251d</code></a> Remove <code>HTMLTranslator</code> and <code>html5_ready</code> from <code>sphinx.builders.html</code> (...</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/77fd819a09881102bdd48593cc69ec164b2495c8""><code>77fd819</code></a> Remove deprecated <code>LaTeXTranslator.docclasses</code> attribute (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11382"">#11382</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/4be56f3b7ddf44265d2e7a8f26a33ba46be52ab9""><code>4be56f3</code></a> Remove deprecated <code>style</code> key for HTML templates (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11381"">#11381</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/49027a9ae61d0d00016182025f68239a50ba4c09""><code>49027a9</code></a> Autosummary: Always emit grouped <code>ImportError</code> exceptions (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11380"">#11380</a>)</li>
<li>Additional commits viewable in <a href=""https://github.com/sphinx-doc/sphinx/compare/v6.1.3...v7.0.0"">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=7.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2068.org.readthedocs.build/en/2068/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2068/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1708981860,PR_kwDOBm6k_c5QdMea,2074,sort files by mtime,3919561,open,0,,,0,2023-05-14T15:25:15Z,2023-05-14T15:25:29Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2074,"serving multiple database files and getting tired by the default sort, changes so the sort order puts the latest changed databases to be on top of the list so don't have to scroll down, lazy as i am ;)

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2074.org.readthedocs.build/en/2074/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2074/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1708030220,I_kwDOBm6k_c5lznkM,2073,Faceting doesn't work against integer columns in views,9599,open,0,,,2,2023-05-12T18:20:10Z,2023-05-12T18:24:07Z,,OWNER,,"Spotted this issue here: https://til.simonwillison.net/datasette/baseline

I had to do this workaround:
```sql
create view baseline as select
  _key,
  spec,
  '' || json_extract(status, '$.is_baseline') as is_baseline,
  json_extract(status, '$.since') as baseline_since,
  json_extract(status, '$.support.chrome') as baseline_chrome,
  json_extract(status, '$.support.edge') as baseline_edge,
  json_extract(status, '$.support.firefox') as baseline_firefox,
  json_extract(status, '$.support.safari') as baseline_safari,
  compat_features,
  caniuse,
  usage_stats,
  status
from
  [index]
```
I think the core issue here is that, against a table, `select * from x where integer_column = '1'` works correctly, due to some kind of column type conversion mechanism... but this mechanism doesn't work against views.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2073/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1578790070,I_kwDOCGYnMM5eGmy2,527,`Table.convert()` skips falsey values,167893,closed,0,,,5,2023-02-10T00:00:52Z,2023-05-09T21:15:05Z,2023-05-08T21:03:24Z,CONTRIBUTOR,,"# Summary

By design, `Table.convert()` does [not attempt](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L2663) conversion of falsey values (`None`, `""""`, `0`, ...). This is surprising (directly contradicts the docstring) and `convert()` may quietly skip cells where the user assumed a conversion would take place. 

# Example
Increment a column of integers by one

``` python
from sqlite_utils import Database

db = Database(memory=True)
table = db['table']
col = 'x'
table.insert_all([{col: 0}, {col:1}])
print(table.get(1)) # 0
print(table.get(2)) # 1
print()

table.convert(col, lambda x: x+1)
print(table.get(1)) # got 0, expected 1 ⚠⚠⚠
print(table.get(2)) # got 2, expected 2
```

Another example might be, say, transforming cells containing empty string to `NULL`.

# Discussion

This was, I think, a pragmatic choice so that consumers can skip writing guard clauses for these falsey values (particularly from the CLI). But this surprising undocumented behavior can lead to incorrect data. I don't think this is a good trade-off between convenience and correctness.

In the absence of this convenience users will either have to write guard clauses into their conversion expressions (or adapt the called function to do the same), so: 
``` python
    fn(value) if value else value
```
instead of:
``` python
    fn(value)
```
This is more typing and sometimes I will forget, and there will be errors. (But they will be noisy errors, which is a good thing).

Such a change will certainly inconvenience some existing consumers; there will be some breakage. But I think this is worth it to avoid quietly not converting some values by default, which can lead to quietly bad data.

I have a PR that I will attach, please take a look and see what you think.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/527/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1702354223,I_kwDOBm6k_c5ld90v,2070,Mechanism for deploying a preview of a branch using Vercel,9599,closed,0,,,2,2023-05-09T16:21:45Z,2023-05-09T16:25:00Z,2023-05-09T16:24:31Z,OWNER,,"I prototyped that here: https://github.com/simonw/one-off-actions/blob/main/.github/workflows/deploy-datasette-branch-preview.yml

It deployed the `json-extras-query` branch here: https://datasette-preview-json-extras-query.vercel.app/",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2070/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1665200812,PR_kwDOCGYnMM5OKveS,537,Support self-referencing FKs in `Table.create`,544011,closed,0,,,3,2023-04-12T20:26:59Z,2023-05-08T22:45:33Z,2023-05-08T21:10:01Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/537,"

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--537.org.readthedocs.build/en/537/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/537/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1701018909,I_kwDOCGYnMM5lY30d,543,Tests broken on Windows due to new convert() lambda names,9599,closed,0,,,0,2023-05-08T22:11:29Z,2023-05-08T22:19:04Z,2023-05-08T22:19:04Z,OWNER,,"https://github.com/simonw/sqlite-utils/actions/runs/4920084038/jobs/8788501314
```python
sql = 'update [example] set [dt] = lambda_-9223371942137158589([dt]);'
```
From:
- #526",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/543/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1516644980,I_kwDOCGYnMM5aZip0,520,rows_from_file() raises confusing error if file-like object is not in binary mode,9599,closed,0,,,3,2023-01-02T19:00:14Z,2023-05-08T22:08:07Z,2023-05-08T22:08:07Z,OWNER,,"I got this error:

```
  File ""/Users/simon/Dropbox/Development/openai-to-sqlite/openai_to_sqlite/cli.py"", line 27, in embeddings
    rows, _ = rows_from_file(input)
              ^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/simon/.local/share/virtualenvs/openai-to-sqlite-jt4obeb2/lib/python3.11/site-packages/sqlite_utils/utils.py"", line 305, in rows_from_file
    first_bytes = buffered.peek(2048).strip()
                  ^^^^^^^^^^^^^^^^^^^
```
From this code:
```python

@cli.command()
@click.argument(
    ""db_path"",
    type=click.Path(file_okay=True, dir_okay=False, allow_dash=False),
)
@click.option(
    ""-i"",
    ""--input"",
    type=click.File(""r""),
    default=""-"",
)
def embeddings(db_path, input):
    ""Store embeddings for one or more text documents""
    click.echo(""Here is some output"")
    db = sqlite_utils.Database(db_path)
    rows, _ = rows_from_file(input)
    print(list(rows))
```
The error went away when I changed it to `type=click.File(""rb"")`.

This should either be called out in the documentation or `rows_from_file()` should be fixed to handle text-mode files in addition to binary files.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/520/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1279144769,I_kwDOCGYnMM5MPjNB,448,Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto',236907,closed,0,,,5,2022-06-21T21:48:27Z,2023-05-08T22:01:00Z,2023-05-08T22:01:00Z,NONE,,"Attempting to run the example given here (without extra bracket ;-):
https://sqlite-utils.datasette.io/en/stable/python-api.html#reading-rows-from-a-file
```
from sqlite_utils.utils import rows_from_file
import io

rows, format = rows_from_file(io.StringIO(""id,name\n1,Cleo""))
print(list(rows), format)
# Outputs [{'id': '1', 'name': 'Cleo'}] Format.CSV
```

Gives error
```
>""c:\Program Files\Python37\python.exe"" test2.py
Traceback (most recent call last):
  File ""test2.py"", line 4, in <module>
    rows, format = rows_from_file(io.StringIO(""id,name\n1,Cleo""))
  File ""C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main\sqlite_utils\utils.py"", line 300, in rows_from_file
    first_bytes = buffered.peek(2048).strip()
AttributeError: '_io.StringIO' object has no attribute 'readinto'
```

I am running Python on Windows.
```
>""c:\Program Files\Python37\python.exe""
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type ""help"", ""copyright"", ""credits"" or ""license"" for more information.
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/448/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1575131737,I_kwDOCGYnMM5d4ppZ,525,Repeated calls to `Table.convert()` fail,167893,closed,0,,,4,2023-02-07T22:40:47Z,2023-05-08T21:59:41Z,2023-05-08T21:54:02Z,CONTRIBUTOR,,"## Summary
When using the API, repeated calls to `Table.convert()`  do not work correctly since all conversions quietly use the callable (function, lambda) from the first call to `convert()` only. Subsequent invocations with different callables use the callable from the first invocation only.

## Example
```python
from sqlite_utils import Database

db = Database(memory=True)
table = db['table']
col = 'x'
table.insert_all([{col: 1}])
print(table.get(1))

table.convert(col, lambda x: x*2)
print(table.get(1))

def zeroize(x):
    return 0
#zeroize = lambda x: 0
#zeroize.__name__ = 'zeroize'
table.convert(col, zeroize)
print(table.get(1))
```

Output:
```
{'x': 1}
{'x': 2}
{'x': 4}
```
Expected:
```
{'x': 1}
{'x': 2}
{'x': 0}
```

## Explanation
This is some relevant [documentation](https://github.com/simonw/sqlite-utils/blob/1491b66dd7439dd87cd5cd4c4684f46eb3c5751b/docs/python-api.rst#registering-custom-sql-functions:~:text=By%20default%20registering%20a%20function%20with%20the%20same%20name%20and%20number%20of%20arguments%20will%20have%20no%20effect).

 * `Table.convert()` takes a `Callable` to perform data conversion on a column
 * The `Callable` is passed to `Database.register_function()`
 * `Database.register_function()` uses the callable's `__name__` attribute for registration
 * (Aside: all lambdas have a `__name__` of `<lambda>`: I thought this was the problem, and it was close, but not quite)
 * However `convert()` first wraps the callable by local function [`convert_value()`](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L2661)
 * Consequently `register_function()` sees name `convert_value` for all invocations from `convert()`
 * `register_function()` silently ignores registrations using the same name, retaining only the first such registration

There's a mismatch between the comments and the code: https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L404

but actually the existing function is returned/used instead (as the ""registering custom sql functions"" doc I linked above says too). Seems like this can be rectified to match the comment?

## Suggested fix
I think there are four things:
1. The call to `register_function()` from `convert()`should have an explicit `name=` parameter (to continue using `convert_value()` and the progress bar).
2. For functions, this name can be the real function name. (I understand the sqlite api needs a name, and it's nice if those are recognizable names where possible). For lambdas would `'lambda-{uuid}'` or similar be acceptable? 
3. `register_function()` really should throw an error on repeated attempts to register a duplicate (function, arity)-pair.
4. A test? I haven't looked at the test framework here but seems this should be testable.

## See also 
- #458 ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/525/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1576990618,PR_kwDOCGYnMM5JkkED,526,Fix repeated calls to `Table.convert()`,167893,closed,0,,,0,2023-02-09T00:14:49Z,2023-05-08T21:56:05Z,2023-05-08T21:53:58Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/526,"Fixes #525. All tests pass.

There's perhaps a better way to name lambdas? There could be a collision if a caller passes a function with name like `lambda_123456`.

SQLite [documentation](https://www.sqlite.org/appfunc.html) is a little, ah, lite on function name specs. If there is a character that can be used in place of underscore in a SQLite function name that is not permitted in a Python function identifier then that could be a good way to prevent accidental collisions. (I tried dash, colon, dot, no joy).

Otherwise, there is little chance of this happening and if it should happen the risk is mitigated by now throwing an exception in the case of a (name, arity) collision without `replace=True`.

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--526.org.readthedocs.build/en/526/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/526/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1465194249,I_kwDOCGYnMM5XVRcJ,514,upsert of new row with check constraints fails,193185,closed,0,,,5,2022-11-26T16:12:23Z,2023-05-08T21:50:52Z,2023-05-08T21:50:51Z,NONE,,"(I originally opened this in https://github.com/simonw/datasette-insert/issues/20, but I see that that library depends on sqlite-utils)

In the case of a new row, upsert first adds the row, specifying only its pkeys: https://github.com/simonw/sqlite-utils/blob/965ca0d5f5bffe06cc02cd7741344d1ddddf9d56/sqlite_utils/db.py#L2783-L2787

This means that a table with NON NULL (or other constraint) columns that aren't part of the pkey can't have new rows upserted.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/514/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1465194930,PR_kwDOCGYnMM5DvZxa,515,"upsert new rows with constraints, fixes #514",193185,closed,0,,,1,2022-11-26T16:15:21Z,2023-05-08T21:27:11Z,2023-05-08T21:27:10Z,NONE,simonw/sqlite-utils/pulls/515,"This fixes #514 by making the initial insert for upserts include all columns, so that new rows can be added to tables with non-pkey columns that have constraints.

(aside: I'm not a python programmer. `pip`? `pipenv`? `venv`? These are mystical incantations to me. The process to set up this repo for local development and testing was _so easy_. Thank you for the excellent contributing documentation!)

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--515.org.readthedocs.build/en/515/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/515/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1044267332,I_kwDOCGYnMM4-PkFE,336,"sqlite-util tranform --column-order mangles columns of type ""timestamp""",536941,closed,0,,,1,2021-11-04T01:15:38Z,2023-05-08T21:13:38Z,2023-05-08T21:13:38Z,CONTRIBUTOR,,"Reproducible code below:

```bash
> echo 'create table bar (baz text, created_at timestamp default CURRENT_TIMESTAMP)' | sqlite3 foo.db
> sqlite3 foo.db
SQLite version 3.36.0 2021-06-18 18:36:39
Enter "".help"" for usage hints.
sqlite> .schema bar
CREATE TABLE bar (baz text, created_at timestamp default CURRENT_TIMESTAMP);
sqlite> .exit
> sqlite-utils transform foo.db bar --column-order baz
sqlite3 foo.db
SQLite version 3.36.0 2021-06-18 18:36:39
Enter "".help"" for usage hints.
sqlite> .schema bar
CREATE TABLE IF NOT EXISTS ""bar"" (
   [baz] TEXT,
   [created_at] FLOAT DEFAULT 'CURRENT_TIMESTAMP'
);
sqlite> .exit
> sqlite-utils transform foo.db bar --column-order baz
> sqlite3 foo.db
SQLite version 3.36.0 2021-06-18 18:36:39
Enter "".help"" for usage hints.
sqlite> .schema bar
CREATE TABLE IF NOT EXISTS ""bar"" (
   [baz] TEXT,
   [created_at] FLOAT DEFAULT '''CURRENT_TIMESTAMP'''
);
```

",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/336/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1432377191,I_kwDOCGYnMM5VYFdn,509,`sqlite-utils transform` breaks DEFAULT string values and STRFTIME(),2199875,closed,0,,,0,2022-11-02T02:32:23Z,2023-05-08T21:13:38Z,2023-05-08T21:13:38Z,NONE,,"Very nice library! Our team found sqlite-utils through @simonw's [comment on the ""Simple declarative schema migration for SQLite"" article](https://news.ycombinator.com/item?id=31249823), and we were excited to use it, but unfortunately `sqlite-utils transform` seems to break our DB. 

Running `sqlite-utils transform` to modify a column mangles their DEFAULT values:

- Default string values are wrapped in extra single quotes
- Function expressions such as [`STRFTIME()`](https://www.sqlite.org/lang_datefunc.html) are turned into strings!

------

Here are steps to reproduce:

**Original database**

```
$ sqlite3 test.db << EOF
CREATE TABLE mytable (
    col1 TEXT DEFAULT 'foo',
    col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))
)
EOF

$ sqlite3 test.db ""SELECT sql FROM sqlite_master WHERE name = 'mytable';""
CREATE TABLE mytable (
    col1 TEXT DEFAULT 'foo',
    col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))
)
```

**Modified database after sqlite-utils**

```
$ sqlite3 test.db ""INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;""
foo|2022-11-02 02:26:58.038

$ sqlite-utils transform test.db mytable --rename col1 renamedcol1

$ sqlite3 test.db ""SELECT sql FROM sqlite_master WHERE name = 'mytable';""
CREATE TABLE ""mytable"" (
   [renamedcol1] TEXT DEFAULT '''foo''',
   [col2] TEXT DEFAULT 'STRFTIME(''%Y-%m-%d %H:%M:%f'', ''NOW'')'
)

$ sqlite3 test.db ""INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;""
foo|2022-11-02 02:26:58.038
'foo'|STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')
```

(Related: #336)",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/509/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1505568103,PR_kwDOCGYnMM5F609a,519,Fixes breaking DEFAULT values,13819005,closed,0,,,1,2022-12-21T01:27:52Z,2023-05-08T21:13:37Z,2023-05-08T21:13:37Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/519,"Fixes #509, Fixes #336

Thanks for the great library!
I fixed a bug that `sqlite-utils transform` breaks DEFAULT values.
All tests already present passed  with no changes, and I added some tests for this PR.

In #509 case, fixed here.

```shell
$ sqlite3 test.db << EOF
CREATE TABLE mytable (
    col1 TEXT DEFAULT 'foo',
    col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))
)
EOF

$ sqlite3 test.db ""SELECT sql FROM sqlite_master WHERE name = 'mytable';""
CREATE TABLE mytable (
    col1 TEXT DEFAULT 'foo',
    col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))
)

$ sqlite3 test.db ""INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;""
foo|2022-12-21 01:15:39.669

$ sqlite-utils transform test.db mytable --rename col1 renamedcol1
$ sqlite3 test.db ""SELECT sql FROM sqlite_master WHERE name = 'mytable';""
CREATE TABLE ""mytable"" (
   [renamedcol1] TEXT DEFAULT 'foo',
   [col2] TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW'))  # ← Non-String Value
)

$ sqlite3 test.db ""INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;""
foo|2022-12-21 01:15:39.669
foo|2022-12-21 01:15:56.432
```

And #336 case also fixed.
Special values are described [here](https://www.sqlite.org/lang_createtable.html).

> 3.2. The DEFAULT clause
> ... A default value may also be one of the special case-independent keywords CURRENT_TIME, CURRENT_DATE or CURRENT_TIMESTAMP.

```shell
$ echo 'create table bar (baz text, created_at timestamp default CURRENT_TIMESTAMP)' | sqlite3 foo.db
$ sqlite3 foo.db
SQLite version 3.39.5 2022-10-14 20:58:05
Enter "".help"" for usage hints.
sqlite> .schema bar
CREATE TABLE bar (baz text, created_at timestamp default CURRENT_TIMESTAMP);
sqlite> .exit

$ sqlite-utils transform foo.db bar --column-order baz
$ sqlite3 foo.db
SQLite version 3.39.5 2022-10-14 20:58:05
Enter "".help"" for usage hints.
sqlite> .schema bar
CREATE TABLE IF NOT EXISTS ""bar"" (
   [baz] TEXT,
   [created_at] FLOAT DEFAULT CURRENT_TIMESTAMP
);
sqlite> .exit

$ sqlite-utils transform foo.db bar --column-order baz
$ sqlite3 foo.db
SQLite version 3.39.5 2022-10-14 20:58:05
Enter "".help"" for usage hints.
sqlite> .schema bar
CREATE TABLE IF NOT EXISTS ""bar"" (
   [baz] TEXT,
   [created_at] FLOAT DEFAULT CURRENT_TIMESTAMP  # ← Non-String Value
);
```


<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--519.org.readthedocs.build/en/519/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/519/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1578793661,PR_kwDOCGYnMM5Jqn1u,528,Enable `Table.convert()` on falsey values,167893,closed,0,,,1,2023-02-10T00:04:09Z,2023-05-08T21:08:23Z,2023-05-08T21:08:23Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/528,"Fixes #527

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--528.org.readthedocs.build/en/528/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/528/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1700936245,I_kwDOCGYnMM5lYjo1,542,Remove `skip_false=True` and `--no-skip-false` in `sqlite-utils` 4.0,9599,open,0,,9374594,1,2023-05-08T21:04:28Z,2023-05-08T21:07:41Z,,OWNER,,"Following:
- #527

The only reason I didn't remove fix this mis-feature entirely is that it represents a backwards incompatible change. I'll make that change in 4.0.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/542/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1595340692,I_kwDOCGYnMM5fFveU,530,"add ability to configure ""on delete"" and ""on update"" attributes of foreign keys:",536941,open,0,,,2,2023-02-22T15:44:14Z,2023-05-08T20:39:01Z,,CONTRIBUTOR,,"sqlite supports these, and it would be quite nice to be able to add them with sqlite-utils.

https://www.sqlite.org/foreignkeys.html#fk_actions",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/530/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1620254998,I_kwDOCGYnMM5gkyEW,532,Show more information when JSON can't be imported with sqlite-utils insert,83080728,closed,0,,,2,2023-03-12T06:41:44Z,2023-05-08T20:32:16Z,2023-05-08T20:32:02Z,NONE,,"I am currently trying to import the [JSON export of my data from Discord](https://support.discord.com/hc/en-us/articles/360004027692-Requesting-a-Copy-of-your-Data), specifically `activity/reporting/events-*.json`

```
sqlite-utils.exe insert test.db reporting events-2023-00000-of-00001.json
  [###################################-]   99%  00:00:00
Error: Invalid JSON - use --csv for CSV or --tsv for TSV files
```

Please show more information as to *why* this is invalid, if possible.

I am using version 3.30 with Python 3.10 on Windows 11.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/532/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1695428235,I_kwDOCGYnMM5lDi6L,538,`table.upsert_all` fails to write rows when `not_null` is present,1231935,closed,0,,,9,2023-05-04T07:30:38Z,2023-05-08T20:06:35Z,2023-05-08T19:27:02Z,NONE,,"I found an odd bug today, where calls to `table.upsert_all` don't write rows if you include the `not_null` kwarg.

## Repro Example

```py
from sqlite_utils import Database

db = Database(""upsert-test.db"")

db[""comments""].upsert_all(
    [{""id"": 1, ""name"": ""david""}],
    pk=""id"",
    not_null=[""name""],
)

assert list(db[""comments""].rows) # err!
```

The schema is correctly created:

```sql
CREATE TABLE [comments] (
   [id] INTEGER PRIMARY KEY,
   [name] TEXT NOT NULL
)
```

But no rows are created. Removing either the `not_null` kwargs works as expected, as does an `insert_all` call.

## Version Info

- Python: `3.11.0`
- sqlite-utils: `3.30`
- sqlite: `3.39.5 2022-10-14`",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/538/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1700840265,I_kwDOCGYnMM5lYMNJ,541,Get tests to pass with `pytest -Werror`,9599,open,0,,,1,2023-05-08T19:57:23Z,2023-05-08T19:59:35Z,,OWNER,,"Inspired by:
- #534",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/541/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1622640374,I_kwDOCGYnMM5gt4b2,534, ResourceWarning: unclosed file,1244826,closed,0,,,1,2023-03-14T03:02:18Z,2023-05-08T19:56:29Z,2023-05-08T19:56:29Z,NONE,,"Issuing either

```
py -Wdefault -m sqlite_utils insert dogs.db dogs dogs0.csv --csv
  [#############-----------------------]   36%
  [####################################]  100%C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sqlite_utils\cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs0.csv' encoding='utf-8-sig'>
  insert_upsert_implementation(
ResourceWarning: Enable tracemalloc to get the object allocation traceback
```
or
```
set pythonwarnings=default
sqlite-utils insert dogs.db dogs dogs0.csv --csv
  [#############-----------------------]   36%
  [####################################]  100%C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sqlite_utils\cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs0.csv' encoding='utf-8-sig'>
  insert_upsert_implementation(
ResourceWarning: Enable tracemalloc to get the object allocation traceback
```

exhibits a ResourceWarning indicating that the CSV file being loaded is not closed.

sqlite-utils --version
sqlite-utils, version 3.30
py --version
Python 3.11.2
Windows Version 10.0.19045 Build 19045
SQLite version 3.41.0 2023-02-21 18:09:37
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/534/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1699184583,I_kwDOCGYnMM5lR3_H,540,sphinx.builders.linkcheck build error,9599,closed,0,,,4,2023-05-07T18:37:09Z,2023-05-08T04:56:13Z,2023-05-07T18:42:36Z,OWNER,,"https://readthedocs.org/projects/sqlite-utils/builds/20512693/
```
 Running Sphinx v6.2.1

Traceback (most recent call last):
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/registry.py"", line 442, in load_extension
    mod = import_module(extname)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/importlib/__init__.py"", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File ""<frozen importlib._bootstrap>"", line 1014, in _gcd_import
  File ""<frozen importlib._bootstrap>"", line 991, in _find_and_load
  File ""<frozen importlib._bootstrap>"", line 975, in _find_and_load_unlocked
  File ""<frozen importlib._bootstrap>"", line 671, in _load_unlocked
  File ""<frozen importlib._bootstrap_external>"", line 783, in exec_module
  File ""<frozen importlib._bootstrap>"", line 219, in _call_with_frames_removed
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/builders/linkcheck.py"", line 20, in <module>
    from requests import Response
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/requests/__init__.py"", line 43, in <module>
    import urllib3
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/urllib3/__init__.py"", line 38, in <module>
    raise ImportError(
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n  7 Dec 2017. See: https://github.com/urllib3/urllib3/issues/2168

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/cmd/build.py"", line 280, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/application.py"", line 225, in __init__
    self.setup_extension(extension)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/application.py"", line 404, in setup_extension
    self.registry.load_extension(self, extname)
  File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/latest/lib/python3.8/site-packages/sphinx/registry.py"", line 445, in load_extension
    raise ExtensionError(__('Could not import extension %s') % extname,
sphinx.errors.ExtensionError: Could not import extension sphinx.builders.linkcheck (exception: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n  7 Dec 2017. See: https://github.com/urllib3/urllib3/issues/2168)

Extension error:
Could not import extension sphinx.builders.linkcheck (exception: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n  7 Dec 2017. See: https://github.com/urllib3/urllib3/issues/2168)
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/540/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1699174055,I_kwDOCGYnMM5lR1an,539,"`--raw-lines` option, like `--raw` for multiple lines",9599,closed,0,,,4,2023-05-07T18:07:46Z,2023-05-07T18:43:24Z,2023-05-07T18:26:18Z,OWNER,,I wanted to output newline-separated output of the first column of every row in the results - like `--row` but for more than one line.,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/539/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1698865182,I_kwDOBm6k_c5lQqAe,2069,[BUG] Cannot insert new data to deployed instance,31861128,open,0,,,1,2023-05-07T02:59:42Z,2023-05-07T03:17:35Z,,NONE,,"## Summary

Recently, I deployed an instance of datasette to Vercel with the following plugins:

- datasette-auth-tokens
- datasette-insert

With the above plugins, I was able to insert new data to local sqlite db. However, when it comes to the deployment on Vercel, things behave differently. I observed some errors from the logs console on Vercel:

```console
File ""/var/task/datasette/database.py"", line 179, in _execute_writes
conn = self.connect(write=True)
File ""/var/task/datasette/database.py"", line 93, in connect
assert not (write and not self.is_mutable)
AssertionError
``` 

<img width=""1380"" alt=""image"" src=""https://user-images.githubusercontent.com/31861128/236655274-3fd5e16f-4e06-4ba8-87ff-41130b8ebe90.png"">

I think it is a potential bug.

## Reproduce

<details><summary>metadata.json</summary>
</br>

```json
{
  ""plugins"": {
    ""datasette-insert"": {
      ""allow"": {
        ""id"": ""*""
      }
    },
    ""datasette-auth-tokens"": {
      ""tokens"": [
        {
          ""token"": {
            ""$env"": ""INSERT_TOKEN""
          },
          ""actor"": {
            ""id"": ""repeater""
          }
        }
      ],
      ""param"": ""_auth_token""
    }
  }
}
```

</details>

<details><summary>commands</summary>
</br>

```bash
# deploy
datasette publish vercel remote.db \
  --project=repeater-bot-sqlite \
  --metadata metadata.json \
  --install datasette-auth-tokens \
  --install datasette-insert \
  --vercel-json=vercel.json

# test insert
cat fixtures/dogs.json | curl --request POST -d @- -H ""Authorization: Bearer <token>"" \
  'https://repeater-bot-sqlite.vercel.app/-/insert/remote/dogs?pk=id'
```

</details>

<details><summary>logs</summary>
</br>

```console
Traceback (most recent call last):
File ""/var/task/datasette/app.py"", line 1354, in route_path
response = await view(request, send)
File ""/var/task/datasette/app.py"", line 1500, in async_view_fn
response = await async_call_with_supported_arguments(
File ""/var/task/datasette/utils/__init__.py"", line 1005, in async_call_with_supported_arguments
return await fn(*call_with)
File ""/var/task/datasette_insert/__init__.py"", line 14, in insert_or_upsert
response = await insert_or_upsert_implementation(request, datasette)
File ""/var/task/datasette_insert/__init__.py"", line 91, in insert_or_upsert_implementation
table_count = await db.execute_write_fn(write_in_thread, block=True)
File ""/var/task/datasette/database.py"", line 167, in execute_write_fn
raise result
File ""/var/task/datasette/database.py"", line 179, in _execute_writes
conn = self.connect(write=True)
File ""/var/task/datasette/database.py"", line 93, in connect
assert not (write and not self.is_mutable)
AssertionError
```

</details>",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2069/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1393202060,I_kwDOCGYnMM5TCpOM,496,devrel/python api: Pylance type hinting,7908073,open,0,,,4,2022-10-01T03:03:34Z,2023-05-03T05:53:27Z,,CONTRIBUTOR,,"Pylance is generally pretty good at figuring out stuff but `sqlite-utils` has some quirks which make type hinting kinda useless. Maybe you don't care but I thought I would bring it to your attention.

For example:

```
db[""subs""].insert_all(subs, pk=""index"")
```

```
Cannot access member ""insert_all"" for type ""View""
  Member ""insert_all"" is unknown
```

`insert_all` and all the other methods show up as a type issues because the program can't know whether something is a View or a Table. Fair enough. But that basically throws all type checking out the window.

`pk=""index""` also shows up as a type issue:

```
Argument of type ""Literal['index']"" cannot be assigned to parameter ""pk"" of type ""Default"" in function ""insert_all""
  ""Literal['index']"" is incompatible with ""Default""
```

I think this is because DEFAULT is an empty class? 

maybe a few small changes could be made to make the library more type-friendly

The interim solution is of course to turn off type hints completely for the line
```
db[""subs""].insert_all(subs, pk=""index"")  # type: ignore
```
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/496/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1690765434,I_kwDOBm6k_c5kxwh6,2067,Litestream-restored db: errors on 3.11 and 3.10.8; but works on py3.10.7 and 3.10.6,39538958,open,0,,,1,2023-05-01T12:42:28Z,2023-05-03T00:16:03Z,,NONE,,"Hi! Wondering if this issue is limited to my local system or if it affects others as well. 

It seems like 3.11 errors out on a ""litestream-restored"" database. On further investigation, it also appears to conk out on 3.10.8 but works on 3.10.7 and 3.10.6.

To demo issue I created a test database, replicated it to an aws s3 bucket, then restored the same under various .pyenv-versioned shells where I test whether I can read the database via the sqlite3 cli.

```sh
# create new shell with 3.11.3
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
sqlite3 data/db.sqlite  
# SQLite version 3.41.2 2023-03-22 11:56:21
# Enter "".help"" for usage hints.
# sqlite> .tables
# _litestream_lock  _litestream_seq   movie           
# sqlite> 
```

However this get me an `OperationalError` when reading via datasette:

<details>
<summary>Error on 3.11.3 and 3.10.8</summary>

```sh
datasette data/db.sqlite
```

```console
/tester/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
  warnings.warn(""pkg_resources is deprecated as an API"", DeprecationWarning)
Traceback (most recent call last):
  File ""/tester/.venv/bin/datasette"", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 143, in wrapped
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 615, in serve
    asyncio.get_event_loop().run_until_complete(check_databases(ds))
  File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/asyncio/base_events.py"", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 660, in check_databases
    await database.execute_fn(check_connection)
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 213, in execute_fn
    return await asyncio.get_event_loop().run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 211, in in_thread
    return fn(conn)
           ^^^^^^^^
  File ""/tester/.venv/lib/python3.11/site-packages/datasette/utils/__init__.py"", line 951, in check_connection
    for r in conn.execute(
             ^^^^^^^^^^^^^
sqlite3.OperationalError: unable to open database file
```

</details>


<details>
<summary>Works on 3.10.7, 3.10.6</summary>

```sh
# create new shell with 3.10.7 / 3.10.6
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
datasette data/db.sqlite
# ...
# INFO:     Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
```

</details>

In both scenarios, the only dependencies were the pinned python version and the latest Datasette version 0.64.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2067/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1683229834,PR_kwDOBm6k_c5PG0wF,2064,Bump sphinx from 6.1.3 to 6.2.1,49699333,closed,0,,,1,2023-04-25T13:57:49Z,2023-05-01T13:58:53Z,2023-05-01T13:58:52Z,CONTRIBUTOR,simonw/datasette/pulls/2064,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 6.2.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/releases"">sphinx's releases</a>.</em></p>
<blockquote>
<h2>v6.2.1</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>v6.2.0</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/blob/master/CHANGES"">sphinx's changelog</a>.</em></p>
<blockquote>
<h1>Release 6.2.1 (released Apr 25, 2023)</h1>
<h2>Bugs fixed</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11355"">#11355</a>: Revert the default type of :confval:<code>nitpick_ignore</code> and
:confval:<code>nitpick_ignore_regex</code> to <code>list</code>.</li>
</ul>
<h1>Release 6.2.0 (released Apr 23, 2023)</h1>
<h2>Dependencies</h2>
<ul>
<li>Require Docutils 0.18.1 or greater.</li>
</ul>
<h2>Incompatible changes</h2>
<ul>
<li>LaTeX: removal of some internal TeX <code>\dimen</code> registers (not previously
publicly documented) as per 5.1.0 code comments in <code>sphinx.sty</code>:
<code>\sphinxverbatimsep</code>, <code>\sphinxverbatimborder</code>, <code>\sphinxshadowsep</code>,
<code>\sphinxshadowsize</code>, and <code>\sphinxshadowrule</code>. (refs: <a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11105"">#11105</a>)</li>
<li>Remove <code>.egg</code> support from pycode <code>ModuleAnalyser</code>; Python eggs are a
now-obsolete binary distribution format</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11089"">#11089</a>: Remove deprecated code in <code>sphinx.builders.linkcheck</code>.
Patch by Daniel Eades</li>
<li>Remove internal-only <code>sphinx.locale.setlocale</code></li>
</ul>
<h2>Deprecated</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11247"">#11247</a>: Deprecate the legacy <code>intersphinx_mapping</code> format</li>
<li><code>sphinx.util.osutil.cd</code> is deprecated in favour of <code>contextlib.chdir</code>.</li>
</ul>
<h2>Features added</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11277"">#11277</a>: :rst:dir:<code>autoproperty</code> allows the return type to be specified as
a type comment (e.g., <code># type: () -&gt; int</code>). Patch by Bénédikt Tran</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/10811"">#10811</a>: Autosummary: extend <code>__all__</code> to imported members for template rendering
when option <code>autosummary_ignore_module_all</code> is set to <code>False</code>. Patch by
Clement Pinard</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11147"">#11147</a>: Add a <code>content_offset</code> parameter to <code>nested_parse_with_titles()</code>,
allowing for correct line numbers during nested parsing.
Patch by Jeremy Maitin-Shepard</li>
<li>Update to Unicode CLDR 42</li>
<li>Add a <code>--jobs</code> synonym for <code>-j</code>. Patch by Hugo van Kemenade</li>
<li>LaTeX: a command <code>\sphinxbox</code> for styling text elements with a (possibly</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/ec993dda3690f260345133c47a4a0f6ef0b18493""><code>ec993dd</code></a> Bump to 6.2.1 final</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/d2aa91f63ce2fa749e4d8835ea7b771ead29e54a""><code>d2aa91f</code></a> Revert the default type of <code>nitpick_ignore[_regex]</code> to <code>list</code></li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/60d8fa10b881308c97012d478fb13770a41b469f""><code>60d8fa1</code></a> Bump version</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/70102ac2ee4d89e1293d8efa7763dcdd104ad548""><code>70102ac</code></a> Bump to 6.2.0 final</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/4e27a5fba146bf85b6e2e0ba8b9f5ac30b1c1e29""><code>4e27a5f</code></a> Remove unneeded JavaScript from <code>sphinx13</code> theme</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/bffb54734f64e88c95b1d5558ec3fb7cb44ceda7""><code>bffb547</code></a> Note correct deprecation version for <code>sphinx.util.osutil.cd</code></li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/59de8d52029deca803d38b5d084a3a32c7b7f994""><code>59de8d5</code></a> Revert &quot;Support and prefer <code>.jinja</code> to <code>_t</code> for static templates (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11165"">#11165</a>)...</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/aee3c0ab75974790adf359a9c5089d1d781a6b21""><code>aee3c0a</code></a> Partially revert &quot;Disable localisation when SOURCE_DATE_EPOCH is set (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/10949"">#10949</a>)...</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/186d596f3323f3141c34618e2e788e5cdff8fec3""><code>186d596</code></a> Use <code>overwrite_file</code> context manager in <code>test_ext_autodoc_configs</code> (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11320"">#11320</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/77483f28244dac1e7459d09397180812cfb36946""><code>77483f2</code></a> Add missing test decorator for <code>test_util_inspect</code> (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11321"">#11321</a>)</li>
<li>Additional commits viewable in <a href=""https://github.com/sphinx-doc/sphinx/compare/v6.1.3...v6.2.1"">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=6.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2064.org.readthedocs.build/en/2064/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2064/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1686033652,I_kwDOBm6k_c5kftT0,2065,Datasette cannot be installed with Rye,9599,closed,0,,,4,2023-04-27T03:35:42Z,2023-04-27T05:09:36Z,2023-04-27T05:09:36Z,OWNER,,"https://github.com/mitsuhiko/rye

I tried this:

    rye install datasette

But now:

```
% ~/.rye/shims/datasette
Traceback (most recent call last):
  File ""/Users/simon/.rye/shims/datasette"", line 5, in <module>
    from datasette.cli import cli
  File ""/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/cli.py"", line 17, in <module>
    from .app import (
  File ""/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/app.py"", line 14, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
```
I think that's because `setuptools` is not included in Rye.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2065/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1686042269,I_kwDOBm6k_c5kfvad,2066,Failing test: httpx.InvalidURL: URL too long,9599,closed,0,,,10,2023-04-27T03:48:47Z,2023-04-27T04:27:50Z,2023-04-27T04:27:50Z,OWNER,,"https://github.com/simonw/datasette/actions/runs/4815723640/jobs/8574667731
```
    def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult:
        # Initial basic checks on allowable URLs.
        # ---------------------------------------
    
        # Hard limit the maximum allowable URL length.
        if len(url) > MAX_URL_LENGTH:
>           raise InvalidURL(""URL too long"")
E           httpx.InvalidURL: URL too long

/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/httpx/_urlparse.py:155: InvalidURL
=========================== short test summary info ============================
FAILED tests/test_csv.py::test_max_csv_mb - httpx.InvalidURL: URL too long
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2066/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1681339696,PR_kwDOBm6k_c5PAcGt,2063,Bump sphinx from 6.1.3 to 6.2.0,49699333,closed,0,,,1,2023-04-24T13:58:21Z,2023-04-25T13:57:55Z,2023-04-25T13:57:53Z,CONTRIBUTOR,simonw/datasette/pulls/2063,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 6.2.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/releases"">sphinx's releases</a>.</em></p>
<blockquote>
<h2>v6.2.0</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/blob/master/CHANGES"">sphinx's changelog</a>.</em></p>
<blockquote>
<h1>Release 6.2.0 (released Apr 23, 2023)</h1>
<h2>Dependencies</h2>
<ul>
<li>Require Docutils 0.18.1 or greater.</li>
</ul>
<h2>Incompatible changes</h2>
<ul>
<li>LaTeX: removal of some internal TeX <code>\dimen</code> registers (not previously
publicly documented) as per 5.1.0 code comments in <code>sphinx.sty</code>:
<code>\sphinxverbatimsep</code>, <code>\sphinxverbatimborder</code>, <code>\sphinxshadowsep</code>,
<code>\sphinxshadowsize</code>, and <code>\sphinxshadowrule</code>. (refs: <a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11105"">#11105</a>)</li>
<li>Remove <code>.egg</code> support from pycode <code>ModuleAnalyser</code>; Python eggs are a
now-obsolete binary distribution format</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11089"">#11089</a>: Remove deprecated code in <code>sphinx.builders.linkcheck</code>.
Patch by Daniel Eades</li>
<li>Remove internal-only <code>sphinx.locale.setlocale</code></li>
</ul>
<h2>Deprecated</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11247"">#11247</a>: Deprecate the legacy <code>intersphinx_mapping</code> format</li>
<li><code>sphinx.util.osutil.cd</code> is deprecated in favour of <code>contextlib.chdir</code>.</li>
</ul>
<h2>Features added</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11277"">#11277</a>: :rst:dir:<code>autoproperty</code> allows the return type to be specified as
a type comment (e.g., <code># type: () -&gt; int</code>). Patch by Bénédikt Tran</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/10811"">#10811</a>: Autosummary: extend <code>__all__</code> to imported members for template rendering
when option <code>autosummary_ignore_module_all</code> is set to <code>False</code>. Patch by
Clement Pinard</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11147"">#11147</a>: Add a <code>content_offset</code> parameter to <code>nested_parse_with_titles()</code>,
allowing for correct line numbers during nested parsing.
Patch by Jeremy Maitin-Shepard</li>
<li>Update to Unicode CLDR 42</li>
<li>Add a <code>--jobs</code> synonym for <code>-j</code>. Patch by Hugo van Kemenade</li>
<li>LaTeX: a command <code>\sphinxbox</code> for styling text elements with a (possibly
rounded) box, optional background color and shadow, has been added.
See :ref:<code>sphinxbox</code>. (refs: <a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11224"">#11224</a>)</li>
<li>LaTeX: add <code>\sphinxstylenotetitle</code>, ..., <code>\sphinxstylewarningtitle</code>, ...,
for an extra layer of mark-up freeing up <code>\sphinxstrong</code> for other uses.
See :ref:<code>latex-macros</code>. (refs: <a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11267"">#11267</a>)</li>
<li>LaTeX: :dudir:<code>note</code>, :dudir:<code>hint</code>, :dudir:<code>important</code> and :dudir:<code>tip</code> can
now each be styled as the other admonitions, i.e. possibly with a background
color, individual border widths and paddings, possibly rounded corners, and
optional shadow.  See :ref:<code>additionalcss</code>. (refs: <a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11234"">#11234</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/e7d4c3669cec32721ee7fc25d9cb8dc3497095ed""><code>e7d4c36</code></a> Bump to 6.2.0 final</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/4e27a5fba146bf85b6e2e0ba8b9f5ac30b1c1e29""><code>4e27a5f</code></a> Remove unneeded JavaScript from <code>sphinx13</code> theme</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/bffb54734f64e88c95b1d5558ec3fb7cb44ceda7""><code>bffb547</code></a> Note correct deprecation version for <code>sphinx.util.osutil.cd</code></li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/59de8d52029deca803d38b5d084a3a32c7b7f994""><code>59de8d5</code></a> Revert &quot;Support and prefer <code>.jinja</code> to <code>_t</code> for static templates (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11165"">#11165</a>)...</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/aee3c0ab75974790adf359a9c5089d1d781a6b21""><code>aee3c0a</code></a> Partially revert &quot;Disable localisation when SOURCE_DATE_EPOCH is set (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/10949"">#10949</a>)...</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/186d596f3323f3141c34618e2e788e5cdff8fec3""><code>186d596</code></a> Use <code>overwrite_file</code> context manager in <code>test_ext_autodoc_configs</code> (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11320"">#11320</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/77483f28244dac1e7459d09397180812cfb36946""><code>77483f2</code></a> Add missing test decorator for <code>test_util_inspect</code> (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11321"">#11321</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/d8f15c7361c4766667db65252b685f19d4a08b56""><code>d8f15c7</code></a> Increase timeout threshold for <code>linkcheck</code> tests (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11326"">#11326</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/b430e052858c490a4e90f3143f159ee6240eb948""><code>b430e05</code></a> Create a 'search field' component for themes (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11045"">#11045</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/e2f66cea4997b6d8c588d3509adb68d4e9108ee6""><code>e2f66ce</code></a> Update CHANGES for PR <a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11333"">#11333</a></li>
<li>Additional commits viewable in <a href=""https://github.com/sphinx-doc/sphinx/compare/v6.1.3...v6.2.0"">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=6.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2063.org.readthedocs.build/en/2063/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2063/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1674322631,PR_kwDOBm6k_c5OpEz_,2061,"Add ""Packaging a plugin using Poetry"" section in docs",1238873,open,0,,,0,2023-04-19T07:23:28Z,2023-04-19T07:27:18Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2061,"This PR adds a new section about packaging a plugin using `poetry` within the ""Writing plugins"" page of the documentation.

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2061.org.readthedocs.build/en/2061/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2061/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1203842656,I_kwDOCGYnMM5HwS5g,425,`sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher,9599,closed,0,,,5,2022-04-13T22:16:53Z,2023-04-15T20:14:58Z,2022-04-13T22:48:57Z,OWNER,,"Got this error while investigating:
- #421

Even though I was using the `LD_PRELOAD` trick from https://til.simonwillison.net/sqlite/ld-preload to use a newer version of SQLite.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1098531354_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1661860507,PR_kwDOBm6k_c5N_bMw,2056,GitHub Action to lint Python code with ruff,3709715,open,0,,,6,2023-04-11T06:41:27Z,2023-04-15T14:24:46Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2056,"[Ruff](https://beta.ruff.rs/) supports [over 500 lint rules](https://beta.ruff.rs/docs/rules) and can be used to replace [Flake8](https://pypi.org/project/flake8/) (plus dozens of plugins), [isort](https://pypi.org/project/isort/), [pydocstyle](https://pypi.org/project/pydocstyle/), [yesqa](https://github.com/asottile/yesqa), [eradicate](https://pypi.org/project/eradicate/), [pyupgrade](https://pypi.org/project/pyupgrade/), and [autoflake](https://pypi.org/project/autoflake/), all while executing (in Rust) tens or hundreds of times faster than any individual tool.

The ruff Action uses minimal steps to run in ~5 seconds, rapidly providing intuitive GitHub Annotations to contributors.

![image](https://user-images.githubusercontent.com/3709715/223758136-afc386d2-70aa-4eff-953a-2c2d82ceea23.png)

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2056.org.readthedocs.build/en/2056/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2056/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1617602868,I_kwDOJHON9s5gaqk0,6,Character encoding problem,9599,open,0,,,2,2023-03-09T16:44:34Z,2023-04-14T15:22:09Z,,MEMBER,,"I ran against a recent note with this in it:

> Or just ""Actions ⚙️ ""

And got back:

<img width=""367"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/224092586-e2475d8f-868c-489d-b495-2bfeb1a7db99.png"">

> `Actions ⚙️`

Pasting that into https://ftfy.vercel.app/?s=Actions+%E2%80%9A%C3%B6%C3%B4%C3%94%E2%88%8F%C3%A8+ gives this:

```python
s = 'Actions â\x80\x9aöôÃ\x94â\x88\x8fè'
s = s.encode('latin-1')
s = s.decode('utf-8')
s = s.encode('macroman')
s = s.decode('utf-8')
print(s)
```
<img width=""816"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/224092864-2b796c40-7c68-40b1-ae38-d0c61721b742.png"">
",611552758,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/6/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1663399821,I_kwDOBm6k_c5jJXeN,2058,"500 ""attempt to write a readonly database"" error caused by ""PRAGMA schema_version""",9599,open,0,,,9,2023-04-11T23:57:50Z,2023-04-13T16:35:21Z,,OWNER,,"I've not been able to replicate this myself yet, but I've seen log files from a user affected by it.

```
File ""/usr/local/lib/python3.11/site-packages/datasette/views/base.py"", line 89, in dispatch_request
await self.ds.refresh_schemas()
File ""/usr/local/lib/python3.11/site-packages/datasette/app.py"", line 371, in refresh_schemas
await self._refresh_schemas()
File ""/usr/local/lib/python3.11/site-packages/datasette/app.py"", line 386, in _refresh_schemas
schema_version = (await db.execute(""PRAGMA schema_version"")).first()[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 267, in execute
results = await self.execute_fn(sql_operation_in_thread)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 213, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 211, in in_thread
return fn(conn)
^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 237, in sql_operation_in_thread
cursor.execute(sql, params if params is not None else {})
sqlite3.OperationalError: attempt to write a readonly database
```
That's running the official Datasette Docker image on https://fly.io/ - it's causing 500 errors on every page of their site.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2058/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1659525418,PR_kwDOCGYnMM5N35VZ,536,Add paths for homebrew on Apple silicon,25778,closed,0,,,1,2023-04-08T13:34:21Z,2023-04-13T01:44:43Z,2023-04-13T01:44:43Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/536,"Does what it says and nothing else. This is the same set of paths as Datasette uses.

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--536.org.readthedocs.build/en/536/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/536/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1665510265,I_kwDOBm6k_c5jRat5,2060,Clean up a bunch of warnings from ruff,9599,open,0,,,0,2023-04-13T01:23:02Z,2023-04-13T01:23:02Z,,OWNER,,"See:
- #2056

`ruff` spots a bunch of warnings about things like unused variables - would be good to clean up as many of these as possible.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2060/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1665053646,I_kwDOBm6k_c5jPrPO,2059,"""Deceptive site ahead"" alert on Heroku deployment",1186275,open,0,,,1,2023-04-12T18:34:51Z,2023-04-13T01:13:01Z,,NONE,,"I deployed a fairly basic instance of Datasette (`datasette-auth-passwords` is the only plugin) using Heroku. The deployed URL now gives a ""Deceptive site ahead"" warning to users.

Is there way around this? Maybe a way to add ownership verification [through Google's search console](https://search.google.com/search-console/welcome)? ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2059/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1373210675,I_kwDODD6af85R2Ygz,13,fails before generating views. ERR:  table sqlite_master may not be modified,116795,open,0,,,4,2022-09-14T15:41:50Z,2023-04-11T03:46:17Z,,NONE,,"generates checkins.db but seems to fail before generating views 

note: it worked on an Ubuntu WSL but fails on macOS 12.5.1

later edit: I suspect this is a problem with my local set-up, `dogsheep-beta index` also throws the same error

full error:

    Importing 2591 checkins  [###################################-]   98%  00:00:00
    Traceback (most recent call last):
      File ""/Users/pax/devbox/envAll/bin/swarm-to-sqlite"", line 8, in <module>
        sys.exit(cli())
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
        return self.main(*args, **kwargs)
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py"", line 782, in main
        rv = self.invoke(ctx)
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
        return callback(*args, **kwargs)
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/swarm_to_sqlite/cli.py"", line 77, in cli
        ensure_foreign_keys(db)
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/swarm_to_sqlite/utils.py"", line 145, in ensure_foreign_keys
        db[fk.table].add_foreign_key(fk.column, fk.other_table, fk.other_column)
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/sqlite_utils/db.py"", line 2123, in add_foreign_key
        self.db.add_foreign_keys([(self.name, column, other_table, other_column)])
      File ""/Users/pax/devbox/envAll/lib/python3.8/site-packages/sqlite_utils/db.py"", line 1086, in add_foreign_keys
        cursor.execute(
    sqlite3.OperationalError: table sqlite_master may not be modified",205429375,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1661617056,I_kwDODD6af85jCkOg,15,ambiguous column name: createdAt - on checkin_details view,9599,closed,0,,,0,2023-04-11T01:07:47Z,2023-04-11T03:16:37Z,2023-04-11T03:16:37Z,MEMBER,,"It looks like Swarm changed their schema and now both `venues` and `checkins` have `createdAt` fields.

Which breaks this view: https://github.com/dogsheep/swarm-to-sqlite/blob/719b6e96a016d0ca8b316d3bed9c2a7a0cb499ee/swarm_to_sqlite/utils.py#L171-L188",205429375,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/15/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1620164673,PR_kwDOCGYnMM5L08O8,531,Add paths for homebrew on Apple silicon,25778,closed,0,,,4,2023-03-11T22:27:52Z,2023-04-09T01:49:44Z,2023-04-09T01:49:43Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/531,"This also passes in the extension path when specified in GIS methods. Wherever we know an extension path, we use `db.init_spatialite(find_spatialite() or load_extension)`.

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--531.org.readthedocs.build/en/531/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/531/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1657861026,I_kwDOBm6k_c5i0POi,2054,"Make detailed notes on how table, query and row views work right now",9599,open,0,,,13,2023-04-06T18:21:09Z,2023-04-07T20:14:38Z,,OWNER,,"Research to help influence the following:
- #2049 
- #2053 
- #2050 
- #262 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2054/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1646734246,I_kwDOBm6k_c5iJyum,2049,Custom SQL queries should use new JSON ?_extra= format,9599,open,0,,8755003,4,2023-03-30T00:42:53Z,2023-04-05T23:29:27Z,,OWNER,,"Related:
- #262

I've made the change to the table view, now I need the new format to work for arbitrary SQL queries too.

Note that this incorporates both arbitrary SQL queries and canned queries.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2049/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1650984552,PR_kwDOJHON9s5NbyYN,13,use universal command,14314871,open,0,,,0,2023-04-02T15:10:54Z,2023-04-02T15:37:34Z,,FIRST_TIME_CONTRIBUTOR,dogsheep/apple-notes-to-sqlite/pulls/13,,611552758,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/13/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1650981564,I_kwDOJHON9s5iZ_q8,12,Error running pytest,14314871,open,0,,,0,2023-04-02T15:02:36Z,2023-04-02T15:07:10Z,,NONE,,"`______________________________________________________ ERROR collecting tests/test_apple_notes_to_sqlite.py _______________________________________________________
ImportError while importing test module '/Users/lol/development/apple-notes-to-sqlite/tests/test_apple_notes_to_sqlite.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_apple_notes_to_sqlite.py:2: in <module>
    from apple_notes_to_sqlite.cli import cli, COUNT_SCRIPT, FOLDERS_SCRIPT
E   ModuleNotFoundError: No module named 'apple_notes_to_sqlite'`

Solution:
This is likely a PYTHONPATH issue due to having pytest installed both globally and in the venv. We can guarantee the tests run by adding the current directory to sys.path automatically using

`python -m pytest`

The alternative is to activate the venv, install pytest, deactivate, then activate the venv again (https://stackoverflow.com/questions/35045038/how-do-i-use-pytest-with-virtualenv)",611552758,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
907795562,MDU6SXNzdWU5MDc3OTU1NjI=,265,Using enable_fts before search term,36287,open,0,,,1,2021-06-01T01:43:34Z,2023-04-01T17:27:18Z,,NONE,,"Many thanks for the sqlite-utils suite of utilities. Has made my life much much easier. 
I used this to create a table and enable FTS. All works fine. The datasette utility detects FTS and shows a text box. Searching for a term using that interface works well.

However, when I start to use features by following https://www.sqlite.org/fts5.html section **""3. Full-text Query Syntax""** I seem to run into issues that I suspect is due to `escape_fts` wrapper function.

As an example, if i search for the term `""^குகை"" `on the text box in datasette it produces 140 results. However, when i tweak the query produced by datasette to not use ""escape_fts"" it produces 5 results.

Similarly, when I try to restrict the search to a single column in FTS using a spec like `{title : ^குகை}` it returns no rows. The same thing pulls results when used without `escape_fts`. The text in the table is in Tamil language and the search term is a Tamil word.

```
   ...
    where
      posts_fts match escape_fts(:search)
```
vs

```
    ...
    where
      posts_fts match (:search)
```

Any ideas why? How can I get the benefits of both escaping as well as utilizing different facets of providing / controlling search terms? Thanks.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/265/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
702386948,MDU6SXNzdWU3MDIzODY5NDg=,159,.delete_where() does not auto-commit (unlike .insert() or .upsert()),11712349,open,0,,,9,2020-09-16T01:55:52Z,2023-04-01T17:21:05Z,,NONE,,"When you use the delete_where() function on a table, it never commits....

Is that intentional?",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/159/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,