id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
1977004379,PR_kwDOCGYnMM5elFZf,600,Add spatialite arm64 linux path,37802088,closed,0,,,5,2023-11-03T22:23:26Z,2023-11-04T00:34:33Z,2023-11-04T00:31:49Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/600,"According to both [Debian](https://packages.debian.org/bookworm/arm64/libsqlite3-mod-spatialite/filelist) and [Ubuntu](https://packages.ubuntu.com/mantic/arm64/libsqlite3-mod-spatialite/filelist), the correct “target triple” for arm64 is `aarch64-linux-gnu`, so we should be looking in `/usr/lib/aarch64-linux-gnu` for `mod_spatialite.so`.

I can confirm that on both of my Debian arm64 SBCs, `libsqlite3-mod-spatialite` installs to that path.

```
$ ls -l /usr/lib/*/*spatial*
lrwxrwxrwx 1 root root      23 Dec  1  2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so -> mod_spatialite.so.7.1.0
lrwxrwxrwx 1 root root      23 Dec  1  2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7 -> mod_spatialite.so.7.1.0
-rw-r--r-- 1 root root 7348584 Dec  1  2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7.1.0
```

This is a set of before and after snippets of pytest’s output for this PR.

### Before

```
$ pytest
tests/test_get.py ......                                                 [ 73%]
tests/test_gis.py ssssssssssss                                           [ 75%]
tests/test_hypothesis.py ....                                            [ 75%]
```

### After

```
$ pytest
tests/test_get.py ......                                                 [ 73%]
tests/test_gis.py ............                                           [ 75%]
tests/test_hypothesis.py ....                                            [ 75%]
```


Issue: #599

<!-- readthedocs-preview sqlite-utils start -->
----
:books: Documentation preview :books:: https://sqlite-utils--600.org.readthedocs.build/en/600/

<!-- readthedocs-preview sqlite-utils end -->",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/600/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1901483874,PR_kwDOBm6k_c5amULw,2190,"Raise an exception if a ""plugins"" block exists in metadata.json",15178711,closed,0,,,5,2023-09-18T18:08:56Z,2023-10-12T16:20:51Z,2023-10-12T16:20:51Z,CONTRIBUTOR,simonw/datasette/pulls/2190,"refs #2183 #2093

From [this comment](https://github.com/simonw/datasette/pull/2183#issuecomment-1714699724) in #2183: If a `""plugins""` block appears in `metadata.json`, it means that a user hasn't migrated over their plugin configuration from `metadata.json` to `datasette.yaml`, which is a breaking change in Datasette 1.0. 

This PR will ensure that an error is raised whenever that happens.

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2190.org.readthedocs.build/en/2190/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2190/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1869807874,PR_kwDOBm6k_c5Y8AN0,2160,"Bump sphinx, furo, blacken-docs dependencies",49699333,closed,0,,,5,2023-08-28T13:49:31Z,2023-08-29T00:38:33Z,2023-08-29T00:38:32Z,CONTRIBUTOR,simonw/datasette/pulls/2160,"Bumps the python-packages group with 3 updates: [sphinx](https://github.com/sphinx-doc/sphinx), [furo](https://github.com/pradyunsg/furo) and [blacken-docs](https://github.com/asottile/blacken-docs).

Updates `sphinx` from 7.1.2 to 7.2.4
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/releases"">sphinx's releases</a>.</em></p>
<blockquote>
<h2>Sphinx 7.2.4</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>Sphinx 7.2.3</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>Sphinx 7.2.2</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>Sphinx 7.2.1</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
<h2>Sphinx 7.2.0</h2>
<p>Changelog: <a href=""https://www.sphinx-doc.org/en/master/changes.html"">https://www.sphinx-doc.org/en/master/changes.html</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href=""https://github.com/sphinx-doc/sphinx/blob/master/CHANGES"">sphinx's changelog</a>.</em></p>
<blockquote>
<h1>Release 7.2.4 (released Aug 28, 2023)</h1>
<h2>Bugs fixed</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11618"">#11618</a>: Fix a regression in the MoveModuleTargets transform,
introduced in <a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/10478"">#10478</a> (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/9662"">#9662</a>).</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11649"">#11649</a>: linkcheck: Resolve hanging tests for timezones west of London
and incorrect conversion from UTC to offsets from the UNIX epoch.
Patch by Dmitry Shachnev and Adam Turner.</li>
</ul>
<h1>Release 7.2.3 (released Aug 23, 2023)</h1>
<h2>Dependencies</h2>
<ul>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11576"">#11576</a>: Require sphinxcontrib-serializinghtml 1.1.9.</li>
</ul>
<h2>Bugs fixed</h2>
<ul>
<li>Fix regression in <code>autodoc.Documenter.parse_name()</code>.</li>
<li>Fix regression in JSON serialisation.</li>
<li><a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11543"">#11543</a>: autodoc: Support positional-only parameters in <code>classmethod</code> methods
when <code>autodoc_preserve_defaults</code> is <code>True</code>.</li>
<li>Restore support string methods on path objects.
This is deprecated and will be removed in Sphinx 8.
Use :py:func:<code>os.fspath</code> to convert :py:class:<code>~pathlib.Path</code> objects to strings,
or :py:class:<code>~pathlib.Path</code>'s methods to work with path objects.</li>
</ul>
<h1>Release 7.2.2 (released Aug 17, 2023)</h1>
<h2>Bugs fixed</h2>
<ul>
<li>Fix the signature of the <code>StateMachine.insert_input()</code> patch,
for when calling with keyword arguments.</li>
<li>Fixed membership testing (<code>in</code>) for the :py:class:<code>str</code> interface
of the asset classes (<code>_CascadingStyleSheet</code> and <code>_JavaScript</code>),
which several extensions relied upon.</li>
<li>Fixed a type error in <code>SingleFileHTMLBuilder._get_local_toctree</code>,
<code>includehidden</code> may be passed as a string or a boolean.</li>
<li>Fix <code>:noindex:</code> for <code>PyModule</code> and <code>JSModule</code>.</li>
</ul>
<h1>Release 7.2.1 (released Aug 17, 2023)</h1>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/3256f1f22092a3e9c2925472f6f0b41e4f09e902""><code>3256f1f</code></a> Bump to 7.2.4 final</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/2f025a4b226610a170d008a24eeba4f90719fa76""><code>2f025a4</code></a> linkcheck: Fix conversion from UTC time to the UNIX epoch (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11649"">#11649</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/1567281287f780d83d5c1b1202a7e551fd0b3998""><code>1567281</code></a> autodoc: Fix UnboundLocalError in <code>filter_members</code> (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11651"">#11651</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/5e88b9f88665dd8782b9feb381b499ae580cfa1a""><code>5e88b9f</code></a> Fix the MoveModuleTargets transform (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11647"">#11647</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/694fcee13d4298ca39ff08837c28b12b2485fbfd""><code>694fcee</code></a> Fix markup in CHANGES (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11639"">#11639</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/c503c90090efb54eab81621dde73e4c57e1af1e6""><code>c503c90</code></a> Improve <code>pathlib</code> type annotations (<a href=""https://redirect.github.com/sphinx-doc/sphinx/issues/11646"">#11646</a>)</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/bf339b123e9f01bfc62663849563d52a7099701a""><code>bf339b1</code></a> Bump version</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/2f6ea1422b08f9494f34b80c2ff6c96b5f396fdc""><code>2f6ea14</code></a> Bump to 7.2.3 final</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/511e4070aa429a41eac04e88d96806c8fd0a5cc7""><code>511e407</code></a> Implement <code>bool()</code> for string paths</li>
<li><a href=""https://github.com/sphinx-doc/sphinx/commit/494de7327f23e1b8a09f4e327f6d4cdf4e351013""><code>494de73</code></a> Implement <code>hash()</code> for string paths</li>
<li>Additional commits viewable in <a href=""https://github.com/sphinx-doc/sphinx/compare/v7.1.2...v7.2.4"">compare view</a></li>
</ul>
</details>
<br />

Updates `furo` from 2023.7.26 to 2023.8.19
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href=""https://github.com/pradyunsg/furo/blob/main/docs/changelog.md"">furo's changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<!-- raw HTML omitted -->
<h2>2023.08.19 -- Xenolithic Xanadu</h2>
<ul>
<li>Fix missing search context with Sphinx 7.2, for dirhtml builds.</li>
<li>Drop support for Python 3.7.</li>
<li>Present configuration errors in a better format -- thanks <a href=""https://github.com/AA-Turner""><code>@​AA-Turner</code></a>!</li>
<li>Bump <code>require_sphinx()</code> to Sphinx 6.0, in line with dependency changes in Unassuming Ultramarine.</li>
</ul>
<h2>2023.08.17 -- Wonderous White</h2>
<ul>
<li>Fix compatiblity with Sphinx 7.2.0 and 7.2.1.</li>
</ul>
<h2>2023.07.26 -- Vigilant Volt</h2>
<ul>
<li>Fix compatiblity with Sphinx 7.1.</li>
<li>Improve how content overflow is handled.</li>
<li>Improve how literal blocks containing inline code are handled.</li>
</ul>
<h2>2023.05.20 -- Unassuming Ultramarine</h2>
<ul>
<li>✨ Add support for Sphinx 7.</li>
<li>Drop support for Sphinx 5.</li>
<li>Improve the screen-reader label for sidebar collapse.</li>
<li>Make it easier to create derived themes from Furo.</li>
<li>Bump all JS dependencies (NodeJS and npm packages).</li>
</ul>
<h2>2023.03.27 -- Tasty Tangerine</h2>
<ul>
<li>Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.</li>
<li>Add missing class to Font Awesome examples</li>
</ul>
<h2>2023.03.23 -- Sassy Saffron</h2>
<ul>
<li>Update Python version classifiers.</li>
<li>Increase the icon size in mobile header.</li>
<li>Increase admonition title bg opacity.</li>
<li>Change the default API background to transparent.</li>
<li>Transition the API background change.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href=""https://github.com/pradyunsg/furo/commit/0766bb2d6d0392bf90d4e4ac447ae721fa673ee9""><code>0766bb2</code></a> Prepare release: 2023.08.19</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/807d73cb1080a1c9b8321a1dcffd84aee73e859a""><code>807d73c</code></a> Update changelog</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/364b2611f27e463cd1f879bf348975c01294ffc0""><code>364b261</code></a> Accomodate for the required <code>data-content_root</code> for search</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/0d38bc6cffc528ce3cf742fe2bd20e83c46d7891""><code>0d38bc6</code></a> Simplify retrieval of <code>pygments_dark_style</code> value (<a href=""https://redirect.github.com/pradyunsg/furo/issues/699"">#699</a>)</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/3631ffc4ea2f241b89bbd9b6c9d76e52a4226c56""><code>3631ffc</code></a> Use <code>sphinx.errors.ConfigError</code> (<a href=""https://redirect.github.com/pradyunsg/furo/issues/697"">#697</a>)</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/d2e2448852752ec570313bc1daea3bda56069816""><code>d2e2448</code></a> Remove duplicate HTML builder check (<a href=""https://redirect.github.com/pradyunsg/furo/issues/698"">#698</a>)</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/7b4f1308832e99d8e49c80f24665e938673c1644""><code>7b4f130</code></a> Drop Python 3.7 (<a href=""https://redirect.github.com/pradyunsg/furo/issues/701"">#701</a>)</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/e322b71b10c959cce0af80d3ac6934725ce728bb""><code>e322b71</code></a> Remove pointless assert (<a href=""https://redirect.github.com/pradyunsg/furo/issues/702"">#702</a>)</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/ee2097a797e92f1155a17287d063b3181f137115""><code>ee2097a</code></a> Bump <code>require_sphinx()</code> to Sphinx 6.0 (<a href=""https://redirect.github.com/pradyunsg/furo/issues/700"">#700</a>)</li>
<li><a href=""https://github.com/pradyunsg/furo/commit/c1ff10b768aaa2a2e479bbf8d58187e12a2731ac""><code>c1ff10b</code></a> Back to development</li>
<li>Additional commits viewable in <a href=""https://github.com/pradyunsg/furo/compare/2023.07.26...2023.08.19"">compare view</a></li>
</ul>
</details>
<br />

Updates `blacken-docs` from 1.15.0 to 1.16.0
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href=""https://github.com/adamchainz/blacken-docs/blob/main/CHANGELOG.rst"">blacken-docs's changelog</a>.</em></p>
<blockquote>
<h2>1.16.0 (2023-08-16)</h2>
<ul>
<li>
<p>Allow Markdown fence options.</p>
<p>Thanks to initial work from Matthew Anderson in <code>PR [#246](https://github.com/asottile/blacken-docs/issues/246) &lt;https://github.com/adamchainz/blacken-docs/pull/246&gt;</code>__.</p>
</li>
<li>
<p>Expand Markdown detection to all Python language names from Pygments: <code>py</code>, <code>sage</code>, <code>python3</code>, <code>py3</code>, and <code>numpy</code>.</p>
</li>
<li>
<p>Preserve leading whitespace lines in reStructuredText code blocks.</p>
<p>Thanks to Julianus Pfeuffer for the report in <code>Issue [#217](https://github.com/asottile/blacken-docs/issues/217) &lt;https://github.com/adamchainz/blacken-docs/issues/217&gt;</code>__.</p>
</li>
<li>
<p>Use exit code 2 to indicate errors from Black, whilst exit code 1 remains for “files have been formatted”.</p>
<p>Thanks to Julianus Pfeuffer for the report in <code>Issue [#218](https://github.com/asottile/blacken-docs/issues/218) &lt;https://github.com/adamchainz/blacken-docs/issues/218&gt;</code>__.</p>
</li>
<li>
<p>Support passing the <code>--preview</code> option through to Black, to select the future style.</p>
</li>
<li>
<p>Remove <code>language_version</code> from <code>.pre-commit-hooks.yaml</code>.
This change allows <code>default_language_version</code> in ``.pre-commit-config.yaml` to take precedence.</p>
<p>Thanks to Aneesh Agrawal in <code>PR [#258](https://github.com/asottile/blacken-docs/issues/258) &lt;https://github.com/adamchainz/blacken-docs/pull/258&gt;</code>__.</p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/960ead214cd1184149d366c6d27ca6c369ce46b6""><code>960ead2</code></a> Version 1.16.0</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/8f0ed18ce69d35e7e631d2b01983ecf8248667a0""><code>8f0ed18</code></a> Support passing --preview through to Black (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/273"">#273</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/4eb4e4cdbff31cde3d9ffdd7693d5b7dce4dc506""><code>4eb4e4c</code></a> Tweak changelog note</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/6c7450cfa90581b482d71b451b2d007361aa47c1""><code>6c7450c</code></a> Use exit code 2 to indicate errors (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/272"">#272</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/99dfc8d0b402b9b7b0f0e723fc1d030063249eac""><code>99dfc8d</code></a> Preserve leading whitespace lines in rST (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/271"">#271</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/94465e83e7253b4caef1e00fcb30b74ced33f08e""><code>94465e8</code></a> Reformat markdown tests with dedent() (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/270"">#270</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/7cd5f30f0f9e014bda566a1893df3b9c7f9a2c48""><code>7cd5f30</code></a> Use .md in glob example</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/f97e56994178e079ab807cebe54c2044f7b2ae55""><code>f97e569</code></a> Document applying to many files (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/269"">#269</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/ae612b095b5005eee8c3ab9fe65a248d7654582d""><code>ae612b0</code></a> Expand Markdown detection to all Python language names (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/268"">#268</a>)</li>
<li><a href=""https://github.com/adamchainz/blacken-docs/commit/da9b4557afc9b820c40b8d6bdb8f515c6cdec0f5""><code>da9b455</code></a> Replace NamedTuple with plain class (<a href=""https://redirect.github.com/asottile/blacken-docs/issues/267"">#267</a>)</li>
<li>Additional commits viewable in <a href=""https://github.com/asottile/blacken-docs/compare/1.15.0...1.16.0"">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions


</details>

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2160.org.readthedocs.build/en/2160/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2160/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1605481359,PR_kwDOBm6k_c5LDwrF,2031,Expand foreign key references in row view as well,82332573,open,0,,,5,2023-03-01T18:43:09Z,2023-03-24T18:35:25Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2031,"Unlike the table view, the single row view does not resolve foreign key references into labels. This patch extracts the foreign key reference expansion code from TableView.data() into a standalone function that is then called by both TableView.data() and RowView.data().

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2031.org.readthedocs.build/en/2031/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2031/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1033678984,PR_kwDOBm6k_c4tjgJ8,1495,Allow routes to have extra options,536941,open,0,,,5,2021-10-22T15:00:45Z,2021-11-19T15:36:27Z,,CONTRIBUTOR,simonw/datasette/pulls/1495,"Right now, datasette routes can only be a 2-tuple of `(regex, view_fn)`. 

If it was possible for datasette to handle extra options, like [standard Django does](https://docs.djangoproject.com/en/3.2/topics/http/urls/#passing-extra-options-to-view-functions), it would add flexibility for plugin authors.

For example, if extra options were enabled, then it would be easy to make a single table the home page (#1284). This plugin would accomplish it.

```python
from datasette import hookimpl
from datasette.views.table import TableView

@hookimpl
def register_routes(datasette):
        return [
        (r""^/$"", TableView.as_view(datasette), {'db_name': 'DB_NAME',
                                                'table': 'TABLE_NAME'})
    ]
```
",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1495/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
377518499,MDU6SXNzdWUzNzc1MTg0OTk=,374,Get Datasette working with Zeit Now v2's 100MB image size limit,9599,closed,0,,,5,2018-11-05T18:08:29Z,2018-12-19T01:35:59Z,2018-12-19T01:35:59Z,OWNER,,"Follow-on from #366

Zeit Now's v2 cloud has a 100MB size limit on Docker images, in order to support much faster wake-ups of new instances.

Fitting Datasette AND the SQLite database it is hosting in here is going to be a challenge.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/374/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
333238932,MDU6SXNzdWUzMzMyMzg5MzI=,316,datasette inspect takes a very long time on large dbs,132230,closed,0,,,5,2018-06-18T11:56:27Z,2019-05-11T18:26:25Z,2019-05-11T18:26:25Z,NONE,,"Hi,

I want to expose data in a very large sqlite database (~600Gb) to the web.  I have used datasette with success on smaller test databases with the same schema - it works very well (thanks!).  However, using the full db, both `datasette inspect` and `datasette serve` seem to hang or pause for a very long time (tens of minutes) on startup.  Is this expected behaviour?

(I noticed that the output of `datasette inspect` includes row counts for each table.   Simply counting the rows in this db will take a long time (tens of millions of rows across each of ~10 tables), so I wondered if this is the source of the problem.)

Any help on a workaround would be appreciated.
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/316/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
329661905,MDU6SXNzdWUzMjk2NjE5MDU=,306,Custom URL routing with independent tests,9599,closed,0,,,5,2018-06-05T23:40:08Z,2018-06-07T15:29:28Z,2018-06-07T15:29:28Z,OWNER,,"The more I think about #303 the more I feel that Datasette's URL routing needs go beyond Django-style regex matching.

If we go custom, tests should live in `test_routing.py`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/306/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326182814,MDU6SXNzdWUzMjYxODI4MTQ=,284,Ability to enable/disable specific features via --config,9599,closed,0,,,5,2018-05-24T15:47:56Z,2018-05-25T06:05:02Z,2018-05-25T05:51:09Z,OWNER,,"`--config` settings from #274 can currently only be integers.

I'd like them to be available as boooeans too. Then we can use them to have that are turned on by default but can be turned off.

First features to get this treatment:

- [x] `allow_sql` - whether or not the `?sql=` parameter is allowed and form is displayed
- [X] `allow_facet` - is `?_facet=` allowed or do we only run facets defined in `metadata.json`
- [X] `allow_download` - do we let users download the full SQLite database file?
- [X] `suggest_facets` - do we attempt to calculate suggested facets?

Refs #275 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/284/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323677499,MDU6SXNzdWUzMjM2Nzc0OTk=,265,Add links to example Datasette instances to appropiate places in docs,9599,closed,0,,,5,2018-05-16T15:40:20Z,2018-06-18T15:52:15Z,2018-06-18T15:52:15Z,OWNER,,"Links to working examples would really help, especially on these pages:

* http://datasette.readthedocs.io/en/latest/json_api.html
* http://datasette.readthedocs.io/en/latest/sql_queries.html
* http://datasette.readthedocs.io/en/latest/facets.html
* http://datasette.readthedocs.io/en/latest/full_text_search.html",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/265/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275125805,MDU6SXNzdWUyNzUxMjU4MDU=,124,Option to open readonly but not immutable,9599,closed,0,,,5,2017-11-19T02:11:03Z,2019-06-24T06:43:46Z,2019-06-24T06:43:46Z,OWNER,,Immutable assumes no other process can modify the file. An option to open reqdonly instead would enable other processes to update the file in place.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/124/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275092453,MDU6SXNzdWUyNzUwOTI0NTM=,122,"Redesign JSON output, ditch jsono, offer variants controlled by parameter instead",9599,closed,0,,,5,2017-11-18T16:52:28Z,2018-04-08T14:54:09Z,2018-04-08T14:54:09Z,OWNER,,"I want to support three variants for the rows output:

* a list of lists, with a columns key saying what they are
* a list of dictionaries
* a single dictionary where the keys are the primary keys of the rows and the values are the row dictionaries themselves

I also want to make the various bits of metadata opt-in - so you don't get the SQL statement unless you ask for it.

These output options should be controlled by query string arguments.

I will set the .jsono URL to redirect to .json with the corresponding options. ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/122/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
275814941,MDU6SXNzdWUyNzU4MTQ5NDE=,141,datasette publish can fail if /tmp is on a different device,21148,closed,0,,2949431,5,2017-11-21T18:28:05Z,2020-04-29T03:27:54Z,2017-12-08T16:06:36Z,CONTRIBUTOR,,"`datasette publish` uses hard links to avoid copying the db into a tmp directory. This can fail if `/tmp` is on another device, because hardlinks can't cross devices. You'll see something like this:

```
$ datasette publish heroku whatever.db
...
OSError: [Errno 18] Invalid cross-device link: '/mnt/c/Users/jacob/c/datasette/whatever.db' -> '/tmp/tmpvxq2yof6/whatever.db'
```
[In my case this is failing because I'm on a Windows machine, using WSL, so my code's on a different virtual filesystem from the Linux subsystem, Because Reasons.]

I'm not sure if it's possible to detect this (can you figure out which device `/tmp` is on?), or what the fallback should be (soft link? copy?).",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/141/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
271831408,MDU6SXNzdWUyNzE4MzE0MDg=,47,Create neat example database,9599,closed,0,,,5,2017-11-07T13:29:38Z,2017-11-14T03:08:13Z,2017-11-14T03:08:13Z,OWNER,,How about data from open elections  eg https://github.com/openelections/openelections-data-ca?files=1,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/47/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
268462768,MDU6SXNzdWUyNjg0NjI3Njg=,38,Experiment with patterns for concurrent long running queries,9599,closed,0,,,5,2017-10-25T16:23:42Z,2018-05-28T20:47:31Z,2018-05-28T20:47:31Z,OWNER,,I want to understand how the system could perform under load with many concurrent long-running queries. Can we serve these without blocking the event loop?,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/38/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
977323133,MDU6SXNzdWU5NzczMjMxMzM=,1445,Ability to search for text across all columns in a table,9599,open,0,,,5,2021-08-23T18:50:48Z,2021-08-23T19:10:17Z,,OWNER,,"When I'm working with new data I often find myself wanting to run a search for text embedded in ANY of the columns of a table, without having to even fully understand the schema first.

I figured out a trick for doing that using a SQL-generated SQL query here: https://til.simonwillison.net/datasette/search-all-columns-trick

But maybe this should be a core Datasette feature? Or a plugin?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1445/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
959898166,MDU6SXNzdWU5NTk4OTgxNjY=,1420,`datasette publish cloudrun --cpu X` option,9599,closed,0,,,5,2021-08-04T05:04:31Z,2021-08-05T00:54:59Z,2021-08-04T05:33:48Z,OWNER,,"For setting the number of vCPUs - current valid values are 1, 2 or 4: https://cloud.google.com/run/docs/configuring/cpu

Pass that through to `gcloud run deploy --image IMAGE_URL --cpu CPU`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1420/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
941300946,MDU6SXNzdWU5NDEzMDA5NDY=,1391,Stop using generated columns in fixtures.db,9599,closed,0,,,5,2021-07-10T18:26:11Z,2021-07-10T19:26:58Z,2021-07-10T19:26:00Z,OWNER,,"Refs #1376 - but I also keep running into this myself, where I try to run something against `fixtures.db` and get this confusing error:

    sqlite3.DatabaseError: malformed database schema (generated_columns) - near ""AS"": syntax error

I'm going to stop using generated columns in `fixtures.db` and instead dynamically generate the generated column table for the duration of the relevant test.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1391/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
925320167,MDU6SXNzdWU5MjUzMjAxNjc=,284,.transform(types=) turns rowid into a concrete column,9599,closed,0,,,5,2021-06-19T05:25:27Z,2021-06-19T15:28:30Z,2021-06-19T15:28:30Z,OWNER,,"Noticed this in the tests for `sqlite-utils memory` in #282 - is it possible to fix this?

https://github.com/simonw/sqlite-utils/commit/ec5174ed40fa283cb06f25ee0c0136297ec313ae",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/284/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
924748955,MDU6SXNzdWU5MjQ3NDg5NTU=,1380,Serve all db files in a folder,193463,open,0,,,5,2021-06-18T10:03:32Z,2021-11-13T08:09:11Z,,NONE,,"I tried to get the `serve` command to serve all the .db files in the `/mnt` folder but is seems that the server does not refresh the list of files.

In more detail:

* Starting datasette as a docker container with: 
```
docker run -p 8001:8001 -v `pwd`:/mnt \
    datasetteproject/datasette \
    datasette -p 8001 -h 0.0.0.0 /mnt
```

* Datasette correctly serves all the *.db files found in the /mnt folder 
* When the server is running, if I copy a new file in the $PWD folder, Datasette does not seem to see the new files, forcing me to restart Docker.

Is there an option/setting that I overlooked, or is this something missing?

BTW, the `--reload` setting, although at first glance is what you think you need, does not seem to do anything in regards of seeing all *.db files.

Thanks!",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1380/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
920636216,MDU6SXNzdWU5MjA2MzYyMTY=,64,"feature: support ""events""",231498,open,0,,,5,2021-06-14T17:42:49Z,2021-06-15T00:48:37Z,,NONE,,"the GitHub API provides the ability to fetch all events for a given user, organization, or repository: https://docs.github.com/en/rest/reference/activity#list-events-for-the-authenticated-user

this would allow users to export all of the issue comments, new issues, etc. that they created. something which is currently missing from the GitHub takeout exports.",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/64/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
906345899,MDU6SXNzdWU5MDYzNDU4OTk=,261,`table.xindexes` using `PRAGMA index_xinfo(table)`,9599,closed,0,,,5,2021-05-29T04:23:48Z,2021-06-03T03:54:14Z,2021-06-03T03:51:32Z,OWNER,,"> `PRAGMA index_xinfo(table)` DOES return that data:
> ```
> (Pdb) [c[0] for c in fresh_db.execute(""PRAGMA > index_xinfo('idx_dogs_age_name')"").description]
> ['seqno', 'cid', 'name', 'desc', 'coll', 'key']
> (Pdb) fresh_db.execute(""PRAGMA index_xinfo('idx_dogs_age_name')"").fetchall()
> [(0, 2, 'age', 1, 'BINARY', 1), (1, 0, 'name', 0, 'BINARY', 1), (2, -1, None, 0, 'BINARY', 0)]
> ```
> See https://sqlite.org/pragma.html#pragma_index_xinfo
> 
> Example output: https://covid-19.datasettes.com/covid?sql=select+*+from+pragma_index_xinfo%28%27idx_ny_times_us_counties_date%27%29
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/260#issuecomment-850766552_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/261/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
893890496,MDU6SXNzdWU4OTM4OTA0OTY=,1332,?_facet_size=X to increase number of facets results on the page,192568,closed,0,,,5,2021-05-18T02:40:16Z,2021-05-27T16:13:07Z,2021-05-23T00:34:37Z,CONTRIBUTOR,,"Is there a way to add a parameter to the URL to modify default_facet_size?

LIkewise, a way to produce a link on the three dots to expand to all items (or match previous number of items, or add x more)?

",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1332/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
860625833,MDU6SXNzdWU4NjA2MjU4MzM=,1300,Make row available to `render_cell` plugin hook,3243482,closed,0,,,5,2021-04-18T10:14:37Z,2022-07-07T16:34:05Z,2022-07-07T16:31:22Z,CONTRIBUTOR,,"*Original title: **Generating URL for a row inside `render_cell` hook***

Hey,
I am using Datasette to view a database that contains video metadata. It has BLOB columns that contain video thumbnails in JPG format (around 100-500KB per row). 

I've registered an output formatter that extends `datasette.blob_renderer.render_blob` function and serves the column with `image/jpeg` content type.

```python
from datasette.blob_renderer import render_blob

async def render_jpg(datasette, database, rows, columns, request, table, view_name):
    response = await render_blob(datasette, database, rows, columns, request, table, view_name)
    response.content_type = ""image/jpeg""
    response.headers[""Content-Disposition""] = f'inline; filename=""image.jpg""'
    return response


@hookimpl
def register_output_renderer():
    return {
        ""extension"": ""jpg"",
        ""render"": render_jpg,
        ""can_render"": lambda: True,
    }
```

This works well. I can visit `http://localhost:8001/mydb/videos/1.jpg?_blob_column=thumbnail` and view the image.

I want to display the image directly with an `<img>` tag (lazy-loaded of course). So, I need a URL, because embedding base64 would increase the page size too much (each image > 100KB). 

Datasette generates a link with `.blob` extension for blob columns. It does this by calling `datasette.urls.row_blob`

https://github.com/simonw/datasette/blob/7a2ed9f8a119e220b66d67c7b9e07cbab47b1196/datasette/views/table.py#L169-L179

But I have no way of getting the row inside the `render_cell` hook. 

```python
@hookimpl
def render_cell(value, column, table, database, datasette):
    if isinstance(value, bytes) and imghdr.what(None, value):
        # generate url
        return '$renderedLink'
```

Any pointers?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1300/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
849396758,MDU6SXNzdWU4NDkzOTY3NTg=,1287,Upgrade to Python 3.9.4,9599,open,0,,,5,2021-04-02T18:43:15Z,2021-04-03T22:38:39Z,,OWNER,,Has some security fixes https://pythoninsider.blogspot.com/2021/04/python-393-and-389-are-now-available.html,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
849220154,MDU6SXNzdWU4NDkyMjAxNTQ=,1286,Better default display of arrays of items,192568,open,0,,,5,2021-04-02T13:31:40Z,2021-06-12T12:36:15Z,,CONTRIBUTOR,,"Would be great to have template filters that convert array fields to bullets and/or delimited lists upon table display:
```
|to_bullets
|to_comma_delimited
|to_semicolon_delimited
```
or maybe: 
```
|join_array(""bullet"")
|join_array(""bullet"",""square"")
|join_array("";"")
|join_array("","")
```
Keeping in mind that bullets show up in html as \<li\> while other delimiting characters appear after the value.

Of course, the fields themselves would remain as facetable arrays. ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1286/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
830567275,MDU6SXNzdWU4MzA1NjcyNzU=,1259,Research using CTEs for faster facet counts,9599,open,0,,,5,2021-03-12T22:19:49Z,2021-03-21T22:55:31Z,,OWNER,,"https://www.sqlite.org/changes.html#version_3_35_0

> Add support for the [MATERIALIZED](https://www.sqlite.org/lang_with.html#mathint) and [NOT MATERIALIZED](https://www.sqlite.org/lang_with.html#mathint) hints when specifying [common table expressions](https://www.sqlite.org/lang_with.html). The default behavior was formerly NOT MATERIALIZED, but is now changed to MATERIALIZED for CTEs that are used more than once.

If a CTE creates a table that is used multiple time in that query, SQLite will now default to creating a materialized table for the duration of that query.

This could be a big performance boost when applying faceting multiple times against the same query. Consider this example query:
```sql
WITH data as (
  select
    *
  from
    [global-power-plants]
),
country_long as (select 
  'country_long' as col, country_long as value, count(*) as c from data group by country_long
  order by c desc limit 10
),
primary_fuel as (
select
  'primary_fuel' as col, primary_fuel as value, count(*) as c from data group by primary_fuel
  order by c desc limit 10
)
select * from primary_fuel union select * from country_long order by col, c desc
```
https://global-power-plants.datasettes.com/global-power-plants?sql=WITH+data+as+%28%0D%0A++select%0D%0A++++*%0D%0A++from%0D%0A++++%5Bglobal-power-plants%5D%0D%0A%29%2C%0D%0Acountry_long+as+%28select+%0D%0A++%27country_long%27+as+col%2C+country_long+as+value%2C+count%28*%29+as+c+from+data+group+by+country_long%0D%0A++order+by+c+desc+limit+10%0D%0A%29%2C%0D%0Aprimary_fuel+as+%28%0D%0Aselect%0D%0A++%27primary_fuel%27+as+col%2C+primary_fuel+as+value%2C+count%28*%29+as+c+from+data+group+by+primary_fuel%0D%0A++order+by+c+desc+limit+10%0D%0A%29%0D%0Aselect+*+from+primary_fuel+union+select+*+from+country_long+order+by+col%2C+c+desc

Outputs:

col | value | c
-- | -- | --
country_long | United States of America | 8688
country_long | China | 4235
country_long | United Kingdom | 2603
country_long | Brazil | 2360
country_long | France | 2155
country_long | India | 1590
country_long | Germany | 1309
country_long | Canada | 1159
country_long | Spain | 829
country_long | Russia | 545
primary_fuel | Solar | 9662
primary_fuel | Hydro | 7155
primary_fuel | Wind | 5188
primary_fuel | Gas | 3922
primary_fuel | Coal | 2390
primary_fuel | Oil | 2290
primary_fuel | Biomass | 1396
primary_fuel | Waste | 1087
primary_fuel | Nuclear | 198
primary_fuel | Geothermal | 189
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1259/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
828858421,MDU6SXNzdWU4Mjg4NTg0MjE=,1258,Allow canned query params to specify default values,1385831,open,0,,,5,2021-03-11T07:19:02Z,2023-02-20T23:39:58Z,,NONE,,"If I call a canned query that includes named parameters, without passing any parameters, datasette runs the query anyway, resulting in an HTTP status code 400, and a visible error in the browser, with only a link back to home.  This means that one of the default links on https://site/database/ will lead to a broken page with no apparent way out.

![image](https://user-images.githubusercontent.com/1385831/110748683-13e72300-820e-11eb-855c-32e03dfef5bf.png)

Is there any way to skip performing the query when parameters aren't supplied, but otherwise render the usual canned query page?  Alternatively, can I supply default values for my parameters, either when defining my canned queries or when linking to the canned query page from the default database template.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1258/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
810397025,MDU6SXNzdWU4MTAzOTcwMjU=,1228,500 error caused  by faceting if a column called `n` exists,7107523,closed,0,,,5,2021-02-17T17:41:20Z,2022-03-19T06:44:40Z,2022-03-19T01:38:04Z,NONE,,"I recently discovered `datasette` thanks to your great talk at FOSDEM and would like to use it for some projects. However, when trying to use it on databases created from some csv ot tsv files, I am sometimes getting this issue when going to http://127.0.0.1:8001/databasetest/databasetest and I don't exactly understand what it refers to.

So far, I couldn't find anything relevant when reviewing the raw text files that could explain this issue, nor could I find something obvious between the files that generate this issue and those that don't. Does the error ring a bell and, if so, could you please point me to the right direction?

```
$ datasette databasetest.db 
INFO:     Started server process [1408482]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
INFO:     127.0.0.1:56394 - ""GET / HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56394 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56396 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56398 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
Traceback (most recent call last):
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/app.py"", line 1099, in route_path
    response = await view(request, send)
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 147, in view
    request, **request.scope[""url_route""][""kwargs""]
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 121, in dispatch_request
    return await handler(request, *args, **kwargs)
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 260, in get
    request, database, hash, correct_hash_provided, **kwargs
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 434, in view_get
    request, database, hash, **kwargs
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/table.py"", line 782, in data
    suggested_facets.extend(await facet.suggest())
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in suggest
    and any(r[""n""] > 1 for r in distinct_values)
  File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in <genexpr>
    and any(r[""n""] > 1 for r in distinct_values)
TypeError: '>' not supported between instances of 'str' and 'int'
INFO:     127.0.0.1:56402 - ""GET /databasetest/databasetest HTTP/1.1"" 500 Internal Server Error
INFO:     127.0.0.1:56402 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET / HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56406 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /databasetest HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56410 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56414 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /databasetest?sql=select+*+from+databasetest HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56410 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56404 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56414 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56408 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK
INFO:     127.0.0.1:56410 - ""GET /databasetest.json?sql=select+*+from+databasetest&_shape=array&_shape=array HTTP/1.1"" 200 OK
^CINFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [1408482]
```

Note that there is no error if I go to http://127.0.0.1:8001/databasetest and then click on `Run SQL`.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1228/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
792904595,MDU6SXNzdWU3OTI5MDQ1OTU=,1201,Release notes for Datasette 0.54,9599,closed,0,,6346396,5,2021-01-24T21:22:28Z,2021-01-25T17:42:21Z,2021-01-25T17:42:21Z,OWNER,,"These will incorporate the release notes from the alpha, much expanded: https://github.com/simonw/datasette/releases/tag/0.54a0",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1201/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
779088071,MDU6SXNzdWU3NzkwODgwNzE=,54,Archive import appears to be broken on recent exports,21148,open,0,,,5,2021-01-05T14:18:01Z,2023-01-04T11:06:55Z,,CONTRIBUTOR,,"I requested a Twitter export yesterday, and unfortunately they seem to have changed it such that `twitter-to-sqlite import` can't handle it anymore 😢 

So far I've ran into two issues. The first was easy to work around, but the second will take more investigation. If I can find the time I'll keep working on it and update this issue accordingly.

The issues (so far):

### 1. Data seems to have moved to a `data/` subdirectory

Running `twitter-to-sqlite import` on the raw zip file reports a bunch of ""not yet implemented"" errors, and then exits without actually importing anything:

```
❯ twitter-to-sqlite import tarchive.db twitter.zip
...
data/manifest: not yet implemented
data/account-creation-ip: not yet implemented
data/account-suspension: not yet implemented
... (dozens of more lines like this, including critical stuff like data/tweets) ...
```

(`tarchive.db` now exists, but is empty)

Workaround: unpack the zip file, and run `twitter-to-sqlite import tarchive.db path/to/archive/data`

That gets further, but:

### 2. Some schema(s?) have changed

At least, the `blocks` schema seems different now:

```
❯ twitter-to-sqlite import tarchive.db archive/data
direct-messages-group: not yet implemented
branch-links: not yet implemented
periscope-expired-broadcasts: not yet implemented
direct-messages: not yet implemented
mute: not yet implemented
Traceback (most recent call last):
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/bin/twitter-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/cli.py"", line 772, in import_
    archive.import_from_file(db, filepath.name, open(filepath, ""rb"").read())
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py"", line 215, in import_from_file
    to_insert = transformer(data)
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py"", line 115, in lists_member
    return {""lists-member"": _list_from_common(data)}
  File ""/Users/jacob/Library/Caches/pypoetry/virtualenvs/jacobian-dogsheep-4AXaN4tu-py3.8/lib/python3.8/site-packages/twitter_to_sqlite/archive.py"", line 200, in _list_from_common
    for url in block[""userListInfo""][""urls""]:
KeyError: 'urls'
```

That's as far as I got before I needed to work on something else. I'll report back if I get further!",206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
778380836,MDU6SXNzdWU3NzgzODA4MzY=,4,Feature Request: Gmail,203343,open,0,,,5,2021-01-04T21:31:09Z,2021-03-04T20:54:44Z,,NONE,,"From takeout, I only exported my Gmail account.  Ideally I could parse this into sqlite via this tool.",206649770,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/4/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
771608692,MDU6SXNzdWU3NzE2MDg2OTI=,14,UNIQUE constraint failed: workouts.id,1234956,open,0,,,5,2020-12-20T15:11:20Z,2023-07-10T14:46:52Z,,NONE,,"I'm getting an error on my initial attempt to import data:

```console
$ healthkit-to-sqlite 20201119\ healthkit\ export.zip healthkit.db
Importing from HealthKit  [###################################-]   98%  00:00:01
Traceback (most recent call last):
  File ""venv/bin/healthkit-to-sqlite"", line 8, in <module>
    sys.exit(cli())
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""venv/lib/python3.9/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/cli.py"", line 57, in cli
    convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py"", line 34, in convert_xml_to_sqlite
    workout_to_db(el, db, zipfile)
  File ""venv/lib/python3.9/site-packages/healthkit_to_sqlite/utils.py"", line 57, in workout_to_db
    pk = db[""workouts""].insert(record, alter=True, hash_id=""id"").last_pk
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1660, in insert
    return self.insert_all(
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1778, in insert_all
    self.insert_chunk(
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 1588, in insert_chunk
    result = self.db.execute(query, params)
  File ""venv/lib/python3.9/site-packages/sqlite_utils/db.py"", line 213, in execute
    return self.conn.execute(sql, parameters)
sqlite3.IntegrityError: UNIQUE constraint failed: workouts.id
```",197882382,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/14/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
756622648,MDU6SXNzdWU3NTY2MjI2NDg=,1125,Show pysqlite3 version on /-/versions,9599,closed,0,,,5,2020-12-03T21:57:23Z,2020-12-04T04:16:57Z,2020-12-04T04:16:57Z,OWNER,,"This code can use `pysqlite3` if available. The version should be exposed on `/-/versions`.

https://github.com/simonw/datasette/blob/4cce5516661b24afeddaf35bee84b00fbf5c7f89/datasette/utils/sqlite.py#L1-L4",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1125/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
750079085,MDU6SXNzdWU3NTAwNzkwODU=,1107,Rename datasette.config() method to datasette.setting(),9599,closed,0,,6055094,5,2020-11-24T21:24:11Z,2020-11-24T22:09:11Z,2020-11-24T22:06:38Z,OWNER,,Part of #1105. Thankfully this isn't yet part of the documented public API on https://docs.datasette.io/en/stable/internals.html,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1107/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
743370900,MDU6SXNzdWU3NDMzNzA5MDA=,1098,Foreign key links break for compound foreign keys,9599,closed,0,,,5,2020-11-15T23:22:14Z,2020-11-29T19:50:31Z,2020-11-29T19:30:23Z,OWNER,,"Reported on Twitter here: https://twitter.com/ZaneSelvans/status/1328093641395548161

> Maybe I'm doing something wrong here but the automatically generated links based foreign key relationships seem to be working here for utility_id_eia, but not for plant_id_eia & generator_id which seems odd: https://pudl-datasette-xl7xwcpe2a-uc.a.run.app/pudl/generators_eia860
>
> Right now it seems like they're trying to, but with only one of the two keys, so it gives ""Error 500. You did not supply a value for binding 2."" Maybe only create the links when it's a simple foreign key?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1098/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
737153927,MDU6SXNzdWU3MzcxNTM5Mjc=,197,Rethink how table.search() method works,9599,closed,0,,6079500,5,2020-11-05T18:04:34Z,2020-11-08T17:07:37Z,2020-11-08T17:07:37Z,OWNER,,"I need to improve this method to help build `sqlite-utils search` in #192 (PR is #195).

The challenge is deciding how it should handle sorting by relevance - especially since that is easy in FTS5 but not at all easy in FTS4.

> Latest test failure:
> ```
> 114  ->	        assert [(""racoons are biting trash pandas"", ""USA"", ""bar"")] == table.search(
> 115  	            ""bite"", order=""rowid""
> 116  	        )
> 117  	
> 118  	
> 119  	def test_optimize_fts(fresh_db):
> (Pdb) table.search(""bite"")
> [(2, 'racoons are biting trash pandas', 'USA', 'bar', -9.641434262948206e-07)]
> ```
> The problem here is that the `table.search()` method now behaves differently for FTS4 v.s. FTS5 tables.
> 
> With FTS4 you get back just the table columns.
>
> With FTS5 you also get back the `rowid` as the first column and the `rank` score as the last column.
> 
> This is weird. It also makes me question whether having `.search()` return a list of tuples is the right API design.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/pull/195#issuecomment-722542895_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/197/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
732939921,MDU6SXNzdWU3MzI5Mzk5MjE=,1068,Default menu links should check a real permission ,9599,closed,0,,6026070,5,2020-10-30T07:08:34Z,2020-10-30T15:44:13Z,2020-10-30T15:42:11Z,OWNER,,"https://github.com/simonw/datasette/blob/18a64fbb29271ce607937110bbdb55488c43f4e0/datasette/default_menu_links.py#L4-L6

This should check a named permission so that it can be customized by permission plugins.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1068/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
732674148,MDU6SXNzdWU3MzI2NzQxNDg=,1062,Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows,9599,open,0,,3268330,5,2020-10-29T21:25:02Z,2022-09-28T14:09:54Z,,OWNER,,This can drive the upgrade of the `register_output_renderer` hook to be able to handle streaming all rows in a large query.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1062/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
725099777,MDU6SXNzdWU3MjUwOTk3Nzc=,1033,datasette.urls.static_plugins(...) method,9599,closed,0,,6026070,5,2020-10-20T01:16:32Z,2020-10-24T22:58:33Z,2020-10-24T20:03:52Z,OWNER,,"Follow-on from #904. For constructing URLs like this:

    /-/static-plugins/NAME_OF_PLUGIN_PACKAGE/yourfile.js

Should be documented on https://docs.datasette.io/en/latest/writing_plugins.html#static-assets and https://docs.datasette.io/en/latest/internals.html#datasette-urls",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1033/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
724759588,MDU6SXNzdWU3MjQ3NTk1ODg=,29,Add search highlighting snippets,9599,open,0,,,5,2020-10-19T16:00:48Z,2021-08-26T20:23:11Z,,MEMBER,,Like on https://til.simonwillison.net/til/search?q=Snippet,197431109,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-beta/issues/29/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,
722758132,MDU6SXNzdWU3MjI3NTgxMzI=,1027,Add documentation on serving Datasette behind a proxy using base_url,9599,closed,0,,6026070,5,2020-10-15T23:46:29Z,2020-10-31T21:14:05Z,2020-10-31T21:14:05Z,OWNER,,"This can go on this page: https://docs.datasette.io/en/stable/deploying.html

Refs #1023, #865",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1027/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
722738988,MDU6SXNzdWU3MjI3Mzg5ODg=,1026,How should datasette.client interact with base_url,9599,closed,0,,6026070,5,2020-10-15T23:07:11Z,2020-10-31T19:29:52Z,2020-10-31T19:29:51Z,OWNER,,"Refs #1023. If Datasette is running with a `base_url` setting and a plugin calls e.g. `datasette.client.get(""/-/plugins.json"")` what should happen?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1026/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
718540751,MDU6SXNzdWU3MTg1NDA3NTE=,1012,For 1.0 update trove classifier in setup.py,9599,open,0,,3268330,5,2020-10-10T05:52:08Z,2021-11-16T13:18:36Z,,OWNER,,    Development Status :: 5 - Production/Stable,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1012/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
711649325,MDU6SXNzdWU3MTE2NDkzMjU=,182,"Better handling of encodings other than utf-8 for ""sqlite-utils insert""",765871,closed,0,,,5,2020-09-30T05:43:48Z,2020-10-16T17:20:41Z,2020-10-16T17:18:52Z,NONE,,"Makefile:
```
data.db:
        curl -O http://maps.natalian.org/data.txt
        go run csv-write.go > data.csv
        sqlite-utils insert data.db travels data.csv --csv

clean:
        rm data*
```
[csv-write.go](https://gist.github.com/kaihendry/dff2442de20d73f900026d13bf7a11d9)


Error message is:

```
sqlite-utils insert data.db travels data.csv --csv
Traceback (most recent call last):
  File ""/home/hendry/.local/bin/sqlite-utils"", line 8, in <module>
    sys.exit(cli())
  File ""/home/hendry/.local/lib/python3.8/site-packages/click/core.py"", line 829, in __call__
    return self.main(*args, **kwargs)
  File ""/home/hendry/.local/lib/python3.8/site-packages/click/core.py"", line 782, in main
    rv = self.invoke(ctx)
  File ""/home/hendry/.local/lib/python3.8/site-packages/click/core.py"", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File ""/home/hendry/.local/lib/python3.8/site-packages/click/core.py"", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File ""/home/hendry/.local/lib/python3.8/site-packages/click/core.py"", line 610, in invoke
    return callback(*args, **kwargs)
  File ""/home/hendry/.local/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 614, in insert
    insert_upsert_implementation(
  File ""/home/hendry/.local/lib/python3.8/site-packages/sqlite_utils/cli.py"", line 553, in insert_upsert_implementation
    headers = next(reader)
  File ""/usr/lib/python3.8/codecs.py"", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe3 in position 1234: invalid continuation byte
make: *** [Makefile:4: data.db] Error 1
[hendry@t14s datasette-map]$ sqlite-utils --version
sqlite-utils, version 2.19
```

Little bit surprised if Go is spewing out bad Unicode, but I'm not sure how to grok `position 1234`..
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/182/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
684961449,MDU6SXNzdWU2ODQ5NjE0NDk=,949,Try out CodeMirror SQL hints,9599,closed,0,,,5,2020-08-24T20:58:21Z,2023-11-03T05:28:58Z,2020-11-01T03:29:48Z,OWNER,,"> It would also be interesting to try out the SQL hint mode, which can autocomplete against tables and columns. This demo shows how to configure that: https://codemirror.net/mode/sql/
> 
> Some missing documentation: https://stackoverflow.com/questions/20023381/codemirror-how-add-tables-to-sql-hint
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/948#issuecomment-679355426_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/949/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
681516976,MDU6SXNzdWU2ODE1MTY5NzY=,944,Path parameters for custom pages,9599,closed,0,,5818042,5,2020-08-19T03:25:17Z,2020-09-14T03:21:45Z,2020-09-14T02:34:58Z,OWNER,,"[Custom pages](https://docs.datasette.io/en/stable/custom_templates.html#custom-pages) let you e.g. create a `templates/pages/about.html` page and have it automatically served at `/about`.

It would be useful if these pages could capture path patterns. I like the Python format string syntax for this (also used by Starlette): `/foo/bar/{slug}`.

So... how about embedding those patterns in the filenames themselves?

    templates/pages/museums/{slug}.html

Would capture any hits to `/museums/something` and use that page to serve them.

",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/944/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
677037043,MDU6SXNzdWU2NzcwMzcwNDM=,923,Add homebrew installation to documentation,9599,closed,0,,,5,2020-08-11T16:54:31Z,2020-08-11T22:53:07Z,2020-08-11T22:52:46Z,OWNER,,"> ```
> $ brew tap simonw/datasette
> $ brew install simonw/datasette/datasette
> $ datasette --version
> datasette, version 0.46
> ```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/335#issuecomment-672088880_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/923/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
675839512,MDU6SXNzdWU2NzU4Mzk1MTI=,132,Features for enabling and disabling WAL mode,9599,closed,0,,,5,2020-08-10T03:25:44Z,2020-08-10T18:59:35Z,2020-08-10T18:59:35Z,OWNER,,I finally figured out how to enable WAL - turns out it's a property of the database file itself: https://github.com/simonw/til/blob/master/sqlite/enabling-wal-mode.md,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
675753042,MDU6SXNzdWU2NzU3NTMwNDI=,131,sqlite-utils insert: options for column types,9599,open,0,,,5,2020-08-09T18:59:11Z,2022-03-15T13:21:42Z,,OWNER,,"The `insert` command currently results in string types for every column - at least when used against CSV or TSV inputs.

It would be useful if you could do the following:

- automatically detects the column types based on eg the first 1000 records
- explicitly state the rule for specific columns

`--detect-types` could work for the former - or it could do that by default and allow opt-out using `--no-detect-types`

For specific columns maybe this:

    sqlite-utils insert db.db images images.tsv \
      --tsv \
      -c id int \
      -c score float",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/131/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
670209331,MDU6SXNzdWU2NzAyMDkzMzE=,913,Mechanism for passing additional options to `datasette my.db` that affect plugins,9599,open,0,,,5,2020-07-31T20:38:26Z,2021-01-04T20:04:11Z,,OWNER,,"> It's a shame there's no obvious mechanism for passing additional options to `datasette my.db` that affect how plugins work.
>
>The only way I can think of at the moment is via environment variables:
>
>     DATASETTE_INSERT_UNSAFE=1 datasette my.db
>
>This will have to do for the moment - it's ugly enough that people will at least know they are doing something unsafe, which is the goal here.
_Originally posted by @simonw in https://github.com/simonw/datasette-insert/issues/15#issuecomment-667346438_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/913/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
660429601,MDU6SXNzdWU2NjA0Mjk2MDE=,45,Fix the demo - it breaks because of the tags table change,9599,closed,0,,,5,2020-07-18T22:49:32Z,2020-07-18T23:03:14Z,2020-07-18T23:03:13Z,MEMBER,,"https://github.com/dogsheep/github-to-sqlite/runs/885773677
```
  File ""/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py"", line 476, in save_tags
    db[""tags""].insert_all(
  File ""/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/sqlite_utils/db.py"", line 1145, in insert_all
    result = self.db.conn.execute(query, params)
sqlite3.OperationalError: table tags has no column named repo
```
That's because I changed the name in #44. I thought this would be safe since no-one else could possibly be using this yet (it hadn't shipped in a release) but turns out I broke my demo!",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
653529088,MDU6SXNzdWU2NTM1MjkwODg=,891,Consider using enable_callback_tracebacks(True),9599,closed,0,,5971510,5,2020-07-08T19:07:16Z,2020-10-08T23:54:23Z,2020-09-15T21:59:27Z,OWNER,,"From https://docs.python.org/3/library/sqlite3.html#sqlite3.enable_callback_tracebacks

> `sqlite3.``enable_callback_tracebacks`(*flag*)[¶](https://docs.python.org/3/library/sqlite3.html#sqlite3.enable_callback_tracebacks ""Permalink to this definition"")
> 
> By default you will not get any tracebacks in user-defined functions, aggregates, converters, authorizer callbacks etc. If you want to debug them, you can call this function with *flag* set to `True`. Afterwards, you will get tracebacks from callbacks on `sys.stderr`. Use [`False`](https://docs.python.org/3/library/constants.html#False ""False"") to disable the feature again.

Maybe turn this on for all of Datasette? Are there any disadvantages to doing that?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/891/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
628003707,MDU6SXNzdWU2MjgwMDM3MDc=,784,Ability to sign in to Datasette as a root account,9599,closed,0,,5512395,5,2020-05-31T17:10:15Z,2020-07-06T19:31:53Z,2020-06-01T01:18:20Z,OWNER,,"> I'm going to draw the line here: default Datasette supports authentication but only for a single user account (""admin""). Plugins can then add support for multiple user accounts, social auth, SSO etc.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/699#issuecomment-636498770_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/784/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
625980317,MDU6SXNzdWU2MjU5ODAzMTc=,771,Unit test that checks that all plugin hooks have corresponding unit tests,9599,closed,0,,5471110,5,2020-05-27T19:42:35Z,2020-05-27T20:21:36Z,2020-05-27T20:17:13Z,OWNER,,Turns out some hooks are missing unit test coverage: https://github.com/simonw/datasette/issues/581#issuecomment-634893744_,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/771/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
621332242,MDU6SXNzdWU2MjEzMzIyNDI=,25,Create a public demo,9599,closed,0,,,5,2020-05-19T22:47:20Z,2020-05-21T22:26:16Z,2020-05-20T05:54:18Z,MEMBER,,"So I can show people what this does, using some of my photos.",256834907,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
608512747,MDU6SXNzdWU2MDg1MTI3NDc=,14,Annotate photos using the Google Cloud Vision API,9599,open,0,,,5,2020-04-28T18:09:03Z,2020-04-28T18:19:06Z,,MEMBER,,"It can detect faces, run OCR, do image labeling (it knows what a lemur is!) and do object localization where it identifies objects and returns bounding polygons for them.",256834907,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/dogsheep-photos/issues/14/reactions"", ""total_count"": 3, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,
601330277,MDU6SXNzdWU2MDEzMzAyNzc=,27,Repos have a big blob of JSON in the organization column,9599,closed,0,,,5,2020-04-16T18:43:14Z,2020-04-18T00:19:16Z,2020-04-18T00:18:52Z,MEMBER,,"e.g. https://github-to-sqlite.dogsheep.net/github/repos

![github__repos__11_rows_where_sorted_by_updated_at_descending](https://user-images.githubusercontent.com/9599/79494124-5640b980-7fd7-11ea-99a2-17ffbd82f9ce.png)

This appears to be obsolete because the `owner` column already links to that record, albeit in the `users` table with `type` set to `Organization`: https://github-to-sqlite.dogsheep.net/github/users/53015001",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
600583271,MDU6SXNzdWU2MDA1ODMyNzE=,727,Custom CSS class on body for styling canned queries,9599,closed,0,,,5,2020-04-15T20:57:32Z,2020-04-15T21:14:58Z,2020-04-15T21:07:50Z,OWNER,,"https://latest.datasette.io/fixtures/neighborhood_search is a canned query page.

One of the templates scanned is `query-fixtures-neighborhood_search.html`

BUT... the body CSS class just looks like this:
```html
<body class=""query db-fixtures"">
```
I would be useful if that included a class that can be used to style that specific canned query page.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/727/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
590669793,MDU6SXNzdWU1OTA2Njk3OTM=,40,Feature: record history of follower counts,9599,closed,0,,,5,2020-03-30T23:32:28Z,2020-04-01T04:13:05Z,2020-04-01T04:13:05Z,MEMBER,,"We currently over-write the follower count every time we import a tweet (when we import that user profile again):

https://github.com/dogsheep/twitter-to-sqlite/blob/810cb2af5a175837204389fd7f4b5721f8b325ab/twitter_to_sqlite/utils.py#L293-L294

It would be neat if we noticed if that user's follower count (and maybe other counts?) had changed since we last saved them and recorded that change in a separate history table. This would be an inexpensive way of building up rough charts of follower count over time.",206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/40/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
585390482,MDU6SXNzdWU1ODUzOTA0ODI=,702,Option in metadata.json to set default sort order for a table,9599,closed,0,,5234079,5,2020-03-21T00:19:56Z,2020-03-25T04:19:36Z,2020-03-22T02:40:35Z,OWNER,,If you access the table page without any `?_sort` or `?_sort_desc` arguments it currently defaults to order by primary key - would be neat to be able to change that.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/702/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
585282212,MDU6SXNzdWU1ODUyODIyMTI=,35,twitter-to-sqlite user-timeline [screen_names] --sql / --attach,9599,closed,0,,,5,2020-03-20T19:26:07Z,2020-03-20T20:17:00Z,2020-03-20T20:16:35Z,MEMBER,,Split from #8.,206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/35/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
571805300,MDU6SXNzdWU1NzE4MDUzMDA=,88,"table.disable_fts() method and ""sqlite-utils disable-fts ..."" command",9599,closed,0,,,5,2020-02-27T04:00:50Z,2020-02-27T04:40:44Z,2020-02-27T04:40:44Z,OWNER,,This would make it easier to iterate on the FTS configuration for a database without having to wipe and recreate the database each time.,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/88/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
569275763,MDU6SXNzdWU1NjkyNzU3NjM=,680,Release automation: automate the bit that posts the GitHub release,9599,closed,0,,,5,2020-02-22T03:50:40Z,2020-09-12T18:18:50Z,2020-09-12T18:18:50Z,OWNER,,"The most manual part of [the release process](https://datasette.readthedocs.io/en/stable/contributing.html#release-process) right now is having to post a GitHub release that matches the updated changelog.

This is particularly annoying because the changelog is in `.rst` while the GitHub release needs markdown - so I currently manually translate between the two.

Having the release script automatically post a GitHub release at the end would be much more convenient.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/680/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
565552217,MDU6SXNzdWU1NjU1NTIyMTc=,674,Rethink how sanity checks work,9599,closed,0,,,5,2020-02-14T20:57:02Z,2020-03-26T17:19:23Z,2020-02-15T17:57:46Z,OWNER,,"If you specify a file to open using `files` or `-i` then Datasette should show a useful error message and fail to start.

Files found by scanning a directory #672 should just be skipped.

_Split off from comment by @simonw in https://github.com/simonw/datasette/issues/673#issuecomment-586455321_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/674/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
556814876,MDU6SXNzdWU1NTY4MTQ4NzY=,662,Escape_fts5_query-hookimplementation does not work with queries to standard tables,2181410,closed,0,,,5,2020-01-29T11:56:03Z,2020-01-30T00:30:20Z,2020-01-30T00:30:19Z,NONE,,"Hi Simon

Thank you for adding the escape_function, but it does not work on my datasette-installation (0.33). I've added the following file to my datasette-dir: /plugins/sql_functions.py:

`from datasette import hookimpl

def escape_fts_query(query):
    bits = query.split()
    return ' '.join('""{}""'.format(bit.replace('""', '')) for bit in bits)

@hookimpl
def prepare_connection(conn):
    conn.create_function(""escape_fts_query"", 1, escape_fts_query)`

It has no effect on the standard queries to the tables though, as they still produce errors when including any characters like '-', '/', '+' or '?'

Does the function only work when using costum queries, where I can include the escape_fts-function explicitly in the sql-query?

PS. I'm calling datasette with --plugins=plugins, and my other plugins work just fine.
PPS. The fts5 virtual table is created with 'sqlite3' like so:

`CREATE VIRTUAL TABLE ""cases_fts"" USING FTS5(
  title,
  subtitle,
  resume,
  suggestion,
  presentation,
  detail = full,
  content_rowid = 'id',
  content = 'cases',
  tokenize='unicode61', 'remove_diacritics 2', 'tokenchars ""-_""'
);`

Thanks!

_Originally posted by @clausjuhl in https://github.com/simonw/datasette/issues/651#issuecomment-579675357_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/662/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
520756546,MDU6SXNzdWU1MjA3NTY1NDY=,12,Add this view for seeing new releases,9599,closed,0,,,5,2019-11-11T06:00:12Z,2020-05-02T18:58:18Z,2020-05-02T18:58:17Z,MEMBER,,"```sql
CREATE VIEW recent_releases AS select
  json_object(""label"", repos.full_name, ""href"", repos.html_url) as repo,
  json_object(
    ""href"",
    releases.html_url,
    ""label"",
    releases.name
  ) as release,
  substr(releases.published_at, 0, 11) as date,
  releases.body as body_markdown,
  releases.published_at
from
  releases
  join repos on repos.id = releases.repo
order by
  releases.published_at desc
```",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
508070977,MDU6SXNzdWU1MDgwNzA5Nzc=,597,If you have databases called foo.db and foo-bar.db you cannot visit /foo-bar,9599,closed,0,,,5,2019-10-16T20:07:41Z,2019-10-18T22:51:08Z,2019-10-18T22:51:08Z,OWNER,,"Weird bug I just came across.

It appears that if you have one database called `foo.db` and another called `foo-bar.db` any attempts to visit `/foo-bar` will redirect to `/foo`.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/597/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
502355384,MDU6SXNzdWU1MDIzNTUzODQ=,580,Testing utilities should be available to plugins,9599,closed,0,,,5,2019-10-03T23:58:26Z,2020-02-28T07:58:46Z,2020-02-28T07:58:46Z,OWNER,,"I'm trying to write a plugin at the moment ([datasette-atom](https://github.com/simonw/datasette-atom)) which needs to run unit tests against a full in-memory Datasette instance, in the same way that the Datasette test suite itself works.

I got it working by creating copies of the [TestClient and TestResponse classes](https://github.com/simonw/datasette/blob/a314b761866d250c16f1ff6dd682010cf4181eb4/tests/fixtures.py#L22-L96) within the plugin itself:

https://github.com/simonw/datasette-atom/commit/c0e3bd9556d7b31f253a8bf666d42205cd24f4fc#diff-33337525d2d877f7cc7f33737bfd2d7b

I had to do this because those classes are in the `tests/` directory within Datasette, so they don't get included in the package that ships to PyPI.

It would be better if these classes were included in the main package in a way that made it easy for plugins to reuse them to write their own tests.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/580/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
500783373,MDU6SXNzdWU1MDA3ODMzNzM=,62,[enhancement] Method to delete a row in python,4454869,closed,0,,,5,2019-10-01T09:45:47Z,2019-11-04T16:30:34Z,2019-11-04T16:18:18Z,NONE,,"Hi !
Thanks for the lib !

Obviously, every possible sql queries won't have a dedicated method.

But I was thinking : a method to delete a row (I'm terrible with names, maybe `delete_where()` or something, would be useful.

I have a Database, with primary key.

For the moment, I use :

```Python3
db.conn.execute(f""DELETE FROM table WHERE key = {key_id}"")
db.conn.commit()
```
to delete a row I don't need anymore, giving his primary key.

Works like a charm.

Just an idea :

```Python3
table.delete_where_pkey({'key': key_id})
```
or something (I know, I'm terrible at naming methods...).

Pros : well, no need to write SQL query.

Cons : WHERE normally allows to do many more things (operators =, <>, >, <, BETWEEN), not to mention AND, OR, etc...
Method is maybe to specific, and/or a pain to render more flexible.

Again, just a thought. Writing his own sql works too, so...

Thanks again.
See yah.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/62/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
492297930,MDU6SXNzdWU0OTIyOTc5MzA=,10,Rethink progress bars for various commands,9599,closed,0,,,5,2019-09-11T15:06:47Z,2020-04-01T03:45:48Z,2020-04-01T03:45:48Z,MEMBER,,"Progress bars and the `--silent` option are implemented inconsistently across commands at the moment.

This is made more challenging by the fact that for many operations the total length is not known.

https://click.palletsprojects.com/en/7.x/api/#click.progressbar",206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/10/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
480961330,MDU6SXNzdWU0ODA5NjEzMzA=,54,"Ability to list views, and to access db[""view_name""].rows / rows_where / etc",20264,closed,0,,,5,2019-08-15T02:00:28Z,2019-08-23T12:41:09Z,2019-08-23T12:20:15Z,NONE,,"The docs show me how to create a view via `db.create_view()` but I can't seem to get back to that view post-creation; if I query it as a table it returns `None`, and it doesn't appear in the table listing, even though querying the view works fine from inside the sqlite3 command-line.

It'd be great to have the view as a pseudo-table, or if the python/sqlite3 module makes that hard to pull off (I couldn't figure it out), to have that edge-case documented next to the `db.create_view()` docs.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/54/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
447408527,MDU6SXNzdWU0NDc0MDg1Mjc=,483,Option to facet by date using month or year,9599,open,0,,,5,2019-05-23T01:25:29Z,2019-05-29T21:38:27Z,,OWNER,,"Facet by date (from #481) can take datetimes and facet them by the day component.

https://latest.datasette.io/fixtures/facetable?_facet_date=created

I'd like to also be able to facet by month or year.

I'm not sure what the best way to achieve this is. Could be two more Facet classes (YearFacet and MonthFacet) but I think it might be nicer if the existing DateFacet could take an optional argument that changed its behaviour. But... if I do that, do I expose it in the UI somewhere or is it only available to URL-hackers?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/483/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
443020810,MDU6SXNzdWU0NDMwMjA4MTA=,460,Design changes to homepage to support mutable files,9599,closed,0,,4305096,5,2019-05-11T17:58:05Z,2019-05-16T03:34:09Z,2019-05-16T03:24:16Z,OWNER,,"Needed for #419 - since we can now start up Datasette with a whole bunch of large connected databases that are mutable we can no longer guarantee a quick count of rows across all of the tables.

New proposed homepage tweaks:

<img width=""872"" alt=""Datasette__blah__commits__events__fixtures__fixtures_modifyme__modme__nerds__out__russian-ads__salaries__sf-trees__sortable__this-db-will-change"" src=""https://user-images.githubusercontent.com/9599/57573373-a1237b80-73db-11e9-9b7c-1842729bcde6.png"">
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/460/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
440437037,MDU6SXNzdWU0NDA0MzcwMzc=,454,Plugin for allowing CORS from specified hosts,9599,closed,0,9599,,5,2019-05-05T12:05:02Z,2019-10-03T23:59:57Z,2019-10-03T23:59:56Z,OWNER,,"It would be useful if Datasette could be configured to allow CORS requests from one or more origins, as opposed to only allowing either none or `""*""`.

This is slightly tricky because the `Access-Control-Allow-Origin: https://foo.example` header is only allowed to return one value per request - and according to https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS ""The Access-Control-Allow-Origin header should contain the value that was sent in the request's Origin header.""

This means the application code needs to have a whitelist of allowed hosts and code that dynamically changes the outgoing `Access-Control-Allow-Origin` header based on the `Origin` header from the incoming request.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/454/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
421548881,MDU6SXNzdWU0MjE1NDg4ODE=,418,Hashed URLs should be optional,9599,closed,0,,4305096,5,2019-03-15T14:34:12Z,2019-05-16T15:12:26Z,2019-05-16T15:12:26Z,OWNER,,"The cute performance hack where a hash of the DB contents is included in the URL makes a lot less sense when serving files that frequently change. It's also difficult to explain to people.

It should be optional and default to ""off"".

Needed for #417 and #419",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/418/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
978537855,MDExOlB1bGxSZXF1ZXN0NzE5MTA5NzA5,321,"Ability to insert file contents as text, in addition to blob",9599,closed,0,,,5,2021-08-24T22:37:18Z,2021-08-24T23:31:17Z,2021-08-24T23:31:13Z,OWNER,simonw/sqlite-utils/pulls/321,Refs #319.,140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/321/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
913865304,MDExOlB1bGxSZXF1ZXN0NjYzODM2OTY1,1368,DRAFT: A new plugin hook for dynamic metadata,2670795,closed,0,,,5,2021-06-07T18:56:00Z,2021-06-26T22:24:54Z,2021-06-26T22:24:54Z,CONTRIBUTOR,simonw/datasette/pulls/1368,"Note that this is a WORK IN PROGRESS!

This PR adds the following plugin hook:

    get_metadata(
      datasette=self, key=key, database=database, table=table,
      fallback=fallback
    )

This gets called when we're building our metdata for the rest of the system to use. Datasette merges whatever the plugins return with any local metadata (from metadata.yml/yaml/json) allowing for a live-editable dynamic Datasette. __A major design consideration is this: should Datasette perform the metadata merge? Or should Datasette allow plugins to perform any modifications themselves?__

As a security precation, local meta is *not* overwritable by plugin hooks. The workflow for transitioning to live-meta would be to load the plugin with the full metadata.yaml and save. Then remove the parts of the metadata that you want to be able to change from the file.

I have a WIP dynamic configuration plugin here, for reference: https://github.com/next-LI/datasette-live-config/",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1368/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
881219362,MDExOlB1bGxSZXF1ZXN0NjM0ODIxMDY1,1319,Add Docker multi-arch support with Buildx,10801138,closed,0,,,5,2021-05-08T19:35:03Z,2021-05-27T16:49:24Z,2021-05-27T16:49:24Z,CONTRIBUTOR,simonw/datasette/pulls/1319,"This adds Docker support to extra CPU architectures (like arm) using [Docker's Buildx action](https://github.com/marketplace/actions/docker-setup-buildx)

You can see [what that looks like on Dockerhub](https://hub.docker.com/r/blairdrummond/datasette/tags?page=1&ordering=last_updated)

And how it lets Datasette run on a Raspberry Pi (top is my dockerhub, bottom is upstream)

![Screenshot from 2021-05-08 15-32-25](https://user-images.githubusercontent.com/10801138/117551210-a17a9f80-b012-11eb-966b-10e1590dd4a9.png)

The workflow log [here](https://github.com/blairdrummond/datasette/runs/2535743398?check_suite_focus=true) (I subbed `blairdrummond` for datasetteproject in my branch) ",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1319/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
688668680,MDExOlB1bGxSZXF1ZXN0NDc1ODc0NDkz,146,Handle case where subsequent records (after first batch) include extra columns,96218,closed,0,,,5,2020-08-30T07:13:58Z,2020-09-08T23:20:37Z,2020-09-08T23:20:37Z,CONTRIBUTOR,simonw/sqlite-utils/pulls/146,"Addresses #145.

I think this should do the job.  If it meets with your approval I'll update this PR to include an update to the documentation -- I came across this bug while preparing a PR to update the documentation around `batch_size` in any event.",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/146/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
677265716,MDExOlB1bGxSZXF1ZXN0NDY2NDEwNzU1,927,"'datasette --get' option, refs #926",9599,closed,0,,,5,2020-08-11T23:31:52Z,2020-08-12T00:24:42Z,2020-08-12T00:24:41Z,OWNER,simonw/datasette/pulls/927,"Refs #926, #898",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/927/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
631300342,MDExOlB1bGxSZXF1ZXN0NDI4MjEyNDIx,798,CSRF protection,9599,closed,0,,5512395,5,2020-06-05T04:22:35Z,2020-06-06T00:43:41Z,2020-06-05T19:05:58Z,OWNER,simonw/datasette/pulls/798,Refs #793,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/798/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
559522877,MDExOlB1bGxSZXF1ZXN0MzcwNjc1MDA3,664,Datasette.render_template() method,9599,closed,0,,,5,2020-02-04T06:53:59Z,2020-02-04T20:26:18Z,2020-02-04T20:26:18Z,OWNER,simonw/datasette/pulls/664,Refs #577,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/664/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
557892819,MDExOlB1bGxSZXF1ZXN0MzY5Mzk0MDQz,80,on_create mechanism for after table creation,9599,closed,0,,,5,2020-01-31T03:38:48Z,2020-01-31T05:08:04Z,2020-01-31T05:08:04Z,OWNER,simonw/sqlite-utils/pulls/80,"I need this for `geojson-to-sqlite`, in particular https://github.com/simonw/geojson-to-sqlite/issues/6",140912432,pull,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/80/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
516763727,MDExOlB1bGxSZXF1ZXN0MzM1OTgwMjQ2,8,"stargazers command, refs #4",9599,closed,0,,,5,2019-11-03T00:37:36Z,2020-05-02T20:00:27Z,2020-05-02T20:00:26Z,MEMBER,dogsheep/github-to-sqlite/pulls/8,Needs tests. Refs #4.,207052882,pull,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
509340359,MDExOlB1bGxSZXF1ZXN0MzI5OTQ3MTgw,601,Don't auto-format SQL on page load,9599,closed,0,,,5,2019-10-18T22:37:39Z,2019-10-20T02:29:49Z,2019-10-18T23:56:45Z,OWNER,simonw/datasette/pulls/601,Refs #600,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/601/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
464786717,MDExOlB1bGxSZXF1ZXN0Mjk0OTkyNTc4,542,extra_template_vars plugin hook,9599,closed,0,,,5,2019-07-05T22:19:17Z,2019-07-06T00:05:57Z,2019-07-06T00:05:56Z,OWNER,simonw/datasette/pulls/542,Refs #541,107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/542/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
322591993,MDExOlB1bGxSZXF1ZXN0MTg3NjY4ODkw,257,Refactor views,9599,closed,0,,,5,2018-05-13T13:00:50Z,2018-05-14T03:04:25Z,2018-05-14T03:04:24Z,OWNER,simonw/datasette/pulls/257,"* Split out view classes from main `app.py`
* Run [black](https://github.com/ambv/black) against resulting code to apply opinionated source code formatting
* Run [isort](https://github.com/timothycrosley/isort) to re-order my imports

Refs #256 ",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/257/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1308461063,I_kwDODFdgUs5N_YgH,74,500 error in github-to-sqlite demo,9599,closed,0,,,5,2022-07-18T19:39:32Z,2022-07-18T21:16:18Z,2022-07-18T21:14:22Z,MEMBER,,"https://github-to-sqlite.dogsheep.net/github/issue_comments throws a 500:

> `cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)`

https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/metrics?project=datasette-222320 suggests this started happening 3 days ago.",207052882,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1818838294,I_kwDOCGYnMM5saUUW,578,Plugin hook for adding new output formats,9599,open,0,,,5,2023-07-24T17:29:18Z,2023-08-07T15:41:49Z,,OWNER,,"> What would it take to add a format hook? I'm still thinking about my GIS workflow, and being able to do `sqlite-utils query ... --geojson` would be nice. It's the one place my Datasette workflow is messy, having to do `datasette . --get /path/to/query.geojson --setting max_rows_returned 10000 --load-extension spatialite`.
> I know the current pattern is `--csv`, but maybe `--format geojson` is more future-proof.

https://discord.com/channels/823971286308356157/997738192360964156/1133076679011602432",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/578/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1578790070,I_kwDOCGYnMM5eGmy2,527,`Table.convert()` skips falsey values,167893,closed,0,,,5,2023-02-10T00:00:52Z,2023-05-09T21:15:05Z,2023-05-08T21:03:24Z,CONTRIBUTOR,,"# Summary

By design, `Table.convert()` does [not attempt](https://github.com/simonw/sqlite-utils/blob/fc221f9b62ed8624b1d2098e564f525c84497969/sqlite_utils/db.py#L2663) conversion of falsey values (`None`, `""""`, `0`, ...). This is surprising (directly contradicts the docstring) and `convert()` may quietly skip cells where the user assumed a conversion would take place. 

# Example
Increment a column of integers by one

``` python
from sqlite_utils import Database

db = Database(memory=True)
table = db['table']
col = 'x'
table.insert_all([{col: 0}, {col:1}])
print(table.get(1)) # 0
print(table.get(2)) # 1
print()

table.convert(col, lambda x: x+1)
print(table.get(1)) # got 0, expected 1 ⚠⚠⚠
print(table.get(2)) # got 2, expected 2
```

Another example might be, say, transforming cells containing empty string to `NULL`.

# Discussion

This was, I think, a pragmatic choice so that consumers can skip writing guard clauses for these falsey values (particularly from the CLI). But this surprising undocumented behavior can lead to incorrect data. I don't think this is a good trade-off between convenience and correctness.

In the absence of this convenience users will either have to write guard clauses into their conversion expressions (or adapt the called function to do the same), so: 
``` python
    fn(value) if value else value
```
instead of:
``` python
    fn(value)
```
This is more typing and sometimes I will forget, and there will be errors. (But they will be noisy errors, which is a good thing).

Such a change will certainly inconvenience some existing consumers; there will be some breakage. But I think this is worth it to avoid quietly not converting some values by default, which can lead to quietly bad data.

I have a PR that I will attach, please take a look and see what you think.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/527/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1465194249,I_kwDOCGYnMM5XVRcJ,514,upsert of new row with check constraints fails,193185,closed,0,,,5,2022-11-26T16:12:23Z,2023-05-08T21:50:52Z,2023-05-08T21:50:51Z,NONE,,"(I originally opened this in https://github.com/simonw/datasette-insert/issues/20, but I see that that library depends on sqlite-utils)

In the case of a new row, upsert first adds the row, specifying only its pkeys: https://github.com/simonw/sqlite-utils/blob/965ca0d5f5bffe06cc02cd7741344d1ddddf9d56/sqlite_utils/db.py#L2783-L2787

This means that a table with NON NULL (or other constraint) columns that aren't part of the pkey can't have new rows upserted.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/514/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1373224657,I_kwDOCGYnMM5R2b7R,488,`sqlite-utils transform` should set empty strings to null when converting text columns to integer/float,9599,open,0,,,5,2022-09-14T15:51:30Z,2022-12-23T17:38:55Z,,OWNER,,"```
/tmp % echo ""id,age,weight\n1,3,2.5\n2,,"" | sqlite-utils insert test.db test - --csv
/tmp % sqlite-utils schema test.db                                                  
CREATE TABLE [test] (
   [id] TEXT,
   [age] TEXT,
   [weight] TEXT
);
/tmp % sqlite-utils transform test.db test --type age integer --type weight float   
/tmp % sqlite-utils schema test.db                                               
CREATE TABLE ""test"" (
   [id] TEXT,
   [age] INTEGER,
   [weight] FLOAT
);
/tmp % sqlite-utils rows test.db test
[{""id"": ""1"", ""age"": 3, ""weight"": 2.5},
 {""id"": ""2"", ""age"": """", ""weight"": """"}]
```
It would be neat if this resulted in the following instead:
```
 {""id"": ""2"", ""age"": null, ""weight"": null}
```
Related Discord discussion: https://discord.com/channels/823971286308356157/823971286941302908/1019635490833567794",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/488/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1338001039,I_kwDOCGYnMM5PwEaP,464,Link from documentation to source code,9599,closed,0,,,5,2022-08-13T16:19:57Z,2022-08-17T23:38:03Z,2022-08-17T23:38:03Z,OWNER,,Twitter conversation asking for ways to automate this here: https://twitter.com/simonw/status/1558260492015046656,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/464/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1310243385,I_kwDOCGYnMM5OGLo5,456,feature request: pivot command,536941,open,0,,,5,2022-07-20T00:58:08Z,2022-07-20T17:50:50Z,,CONTRIBUTOR,,pivoting long-format table to wide-format tables is pretty common and kind of pain. would love to see this feature in sqlite-utils!,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/456/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1279144769,I_kwDOCGYnMM5MPjNB,448,Reading rows from a file => AttributeError: '_io.StringIO' object has no attribute 'readinto',236907,closed,0,,,5,2022-06-21T21:48:27Z,2023-05-08T22:01:00Z,2023-05-08T22:01:00Z,NONE,,"Attempting to run the example given here (without extra bracket ;-):
https://sqlite-utils.datasette.io/en/stable/python-api.html#reading-rows-from-a-file
```
from sqlite_utils.utils import rows_from_file
import io

rows, format = rows_from_file(io.StringIO(""id,name\n1,Cleo""))
print(list(rows), format)
# Outputs [{'id': '1', 'name': 'Cleo'}] Format.CSV
```

Gives error
```
>""c:\Program Files\Python37\python.exe"" test2.py
Traceback (most recent call last):
  File ""test2.py"", line 4, in <module>
    rows, format = rows_from_file(io.StringIO(""id,name\n1,Cleo""))
  File ""C:\Users\swood\Downloads\sqlite-utils-main-20220621\sqlite-utils-main\sqlite_utils\utils.py"", line 300, in rows_from_file
    first_bytes = buffered.peek(2048).strip()
AttributeError: '_io.StringIO' object has no attribute 'readinto'
```

I am running Python on Windows.
```
>""c:\Program Files\Python37\python.exe""
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type ""help"", ""copyright"", ""credits"" or ""license"" for more information.
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/448/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1271426387,I_kwDOCGYnMM5LyG1T,444,CSV `extras_key=` and `ignore_extras=` equivalents for CLI tool,9599,open,0,,,5,2022-06-14T22:22:47Z,2022-07-07T16:39:18Z,,OWNER,,"> I forgot to add equivalents of `extras_key=` and `ignore_extras=` to the CLI tool - will do that in a separate issue.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1155767915_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/444/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1236693079,I_kwDOCGYnMM5JtnBX,432,"Support `rows_where()`, `delete_where()` etc for attached alias databases",11597658,open,0,,,5,2022-05-16T06:38:58Z,2022-06-14T22:16:48Z,,NONE,,"Hi,

I noticed `rows_where()` doesn't return any rows from tables which are from attached databases. The `exists()` function returns false. As far as I can see this is because the `table_names()` function only looks for table names in the current database and not in attached (or temp) databases.

Besides, `rows_where()`, also `insert_all()` and `delete_where()` didn't do what I was expecting because of this. For the moment I've patched `table_names()` for myself, see below but I'm not sure what the total impact is on the other functions like lookup truncate etc which all use `exists()`. Also `view_names()` doesn't look for views in attached or temp databases. 
```python
 def table_names(self, fts4: bool = False, fts5: bool = False) -> List[str]:
        ""A list of string table names in this database.""
        where = [""type = 'table'""]
        if fts4:
            where.append(""sql like '%USING FTS4%'"")
        if fts5:
            where.append(""sql like '%USING FTS5%'"")
        dbs = [x[1] for x in self.execute('pragma database_list').fetchall()]    
        lst=[]
        for db in dbs:    
            sql = ""select name from {} where {}"".format(db+"".sqlite_master"","" AND "".join(where))
            lst.extend(r[0] for r in self.execute(sql).fetchall())
        return lst
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/432/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1203842656,I_kwDOCGYnMM5HwS5g,425,`sqlite3.NotSupportedError`: deterministic=True requires SQLite 3.8.3 or higher,9599,closed,0,,,5,2022-04-13T22:16:53Z,2023-04-15T20:14:58Z,2022-04-13T22:48:57Z,OWNER,,"Got this error while investigating:
- #421

Even though I was using the `LD_PRELOAD` trick from https://til.simonwillison.net/sqlite/ld-preload to use a newer version of SQLite.

_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/421#issuecomment-1098531354_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1166587040,I_kwDOCGYnMM5FiLSg,413,Display autodoc type information more legibly,9599,closed,0,,,5,2022-03-11T15:58:20Z,2022-03-11T18:07:10Z,2022-03-11T18:07:10Z,OWNER,,"https://sqlite-utils.datasette.io/en/3.25/reference.html#sqlite_utils.db.Table.insert looks like this at the moment:

<img width=""703"" alt=""image"" src=""https://user-images.githubusercontent.com/9599/157902622-368935a8-93f2-42e9-98ad-94a45c818e80.png"">
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/413/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1128466114,I_kwDOCGYnMM5DQwbC,406,Creating tables with custom datatypes,82988,open,0,,,5,2022-02-09T12:16:31Z,2022-09-15T18:13:50Z,,NONE,,"Via https://stackoverflow.com/a/18622264/454773 I note the ability to register custom handlers for novel datatypes that can map into and out of things like sqlite `BLOB`s.

From a quick look and a quick play, I didn't spot a way to do this in `sqlite_utils`?

For example:

```python
# Via https://stackoverflow.com/a/18622264/454773
import sqlite3
import numpy as np
import io

def adapt_array(arr):
    """"""
    http://stackoverflow.com/a/31312102/190597 (SoulNibbler)
    """"""
    out = io.BytesIO()
    np.save(out, arr)
    out.seek(0)
    return sqlite3.Binary(out.read())

def convert_array(text):
    out = io.BytesIO(text)
    out.seek(0)
    return np.load(out)


# Converts np.array to TEXT when inserting
sqlite3.register_adapter(np.ndarray, adapt_array)

# Converts TEXT to np.array when selecting
sqlite3.register_converter(""array"", convert_array)
```

```python
from sqlite_utils import Database
db = Database('test.db')

# Reset the database connection to used the parsed datatype
# sqlite_utils doesn't seem to support eg:
#  Database('test.db', detect_types=sqlite3.PARSE_DECLTYPES)
db.conn = sqlite3.connect(db_name, detect_types=sqlite3.PARSE_DECLTYPES)

# Create a table the old fashioned way
# but using the new custom data type
vector_table_create = """"""
CREATE TABLE dummy 
    (title TEXT, vector array );
""""""

cur = db.conn.cursor()
cur.execute(vector_table_create)


# sqlite_utils doesn't appear to support custom types (yet?!)
# The following errors on the ""array"" datatype
""""""
db[""dummy""].create({
    ""title"": str,
    ""vector"": ""array"",
})
""""""
```

We can then add / retrieve records from the database where the datatype of the `vector` field is a custom registered `array` type (which is to say, a `numpy` array):

```python
import numpy as np

db[""dummy""].insert({'title':""test1"", 'vector':np.array([1,2,3])})

for row in db.query(""SELECT * FROM dummy""):
    print(row['title'], row['vector'], type(row['vector']))

""""""
test1 [1 2 3] <class 'numpy.ndarray'>
""""""
```

It would be handy to be able to do this idiomatically in `sqlite_utils`.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/406/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,