# issues

node_id: MDU6SXNzdWU5NjQzMjIxMzY=
title: Manage /robots.txt in Datasette core, block robots by default
state: open
created_at: 2021-08-09T19:56:56Z
updated_at: 2021-12-04T07:11:29Z
author_association: OWNER
type: issue
body:
  See accompanying Twitter thread: https://twitter.com/simonw/status/1424820203603431439

  > Datasette currently has a plugin for configuring robots.txt, but I'm beginning to think it should be part of core and crawlers should be blocked by default - having people explicitly opt-in to having their sites crawled and indexed feels a lot safer https://datasette.io/plugins/datasette-block-robots

  I have a lot of Datasettes deployed now, and tailing logs shows that they are being *hammered* by search engine crawlers even though many of them are not interesting enough to warrant indexing.

  I'm starting to think blocking crawlers would actually be a better default for most people, provided it was well documented and easy to understand how to allow them.

  Default-deny is usually a better policy than default-allow!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1426/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NjQ0MDA0ODI=
title: `sqlite-utils insert --flatten` option to flatten nested JSON
state: closed
created_at: 2021-08-09T21:23:08Z
updated_at: 2021-10-16T13:54:56Z
closed_at: 2021-08-09T21:44:06Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I had to do this with a `jq` recipe today: https://til.simonwillison.net/cloudrun/tailing-cloud-run-request-logs

  ```
  cat log.json | jq -c '[leaf_paths as $path | {
    "key": $path | join("_"), "value": getpath($path)
  }] | from_entries' \
  | sqlite-utils insert /tmp/logs.db logs - --nl --alter --batch-size 1
  ```
  That was to turn something like this:
  ```json
    {
      "httpRequest": {
        "latency": "0.112114537s",
        "requestMethod": "GET",
        "requestSize": "534",
        "status": 200,
      },
      "insertId": "6111722f000b5b4c4d4071e2",
      "labels": {
        "service": "datasette-io"
      }
    }
  ```
  Into this instead:
  ```json
  {
      "httpRequest_latency": "0.112114537s",
      "httpRequest_requestMethod": "GET",
      "httpRequest_requestSize": "534",
      "httpRequest_status": 200,
      "insertId": "6111722f000b5b4c4d4071e2",
      "labels_service": "datasette-io"
  }
  ```
  I have to do this often enough that I think it should be an option, `--flatten` - so I can do this instead:
  ```
  cat log.json | sqlite-utils insert /tmp/logs.db logs - --flatten
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/310/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NjUxMDI1MzQ=
title: Add reference documentation generated from docstrings
state: closed
created_at: 2021-08-10T16:04:00Z
updated_at: 2021-08-11T12:03:50Z
closed_at: 2021-08-11T12:03:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Using https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html

  I'm not a big fan of this kind of documentation because it so often comes in place of narrative documentation - but the library has great narrative documentation now, so the reference documentation can link to it in places.

  This will also encourage me to add good docstrings everywhere, useful for IDEs and suchlike.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/311/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzA3NDkwNzg5
title: Add reference page to documentation using Sphinx autodoc
state: closed
created_at: 2021-08-10T16:59:17Z
updated_at: 2021-08-10T23:09:32Z
closed_at: 2021-08-10T23:09:28Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/312
body: Refs #311.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/312/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NjUxNjYwNTg=
title: `.add_foreign_keys()` doesn't reject being called with a View
state: closed
created_at: 2021-08-10T17:22:17Z
updated_at: 2021-08-10T17:25:34Z
closed_at: 2021-08-10T17:25:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Spotted this bug using `mypy` while working on #311 / #312!

  ```
  % mypy sqlite_utils
  sqlite_utils/db.py:725: error: Item "View" of "Union[Table, View]" has no attribute "foreign_keys"
  Found 1 error in 1 file (checked 5 source files)
  ```
  Refers to this code: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L710-L720

  It's a bug! We run some checks earlier but none of them ensure that it's a view:

  https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L697-L709

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/313/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NjUyMTA5NjY=
state: closed
created_at: 2021-08-10T18:03:59Z
updated_at: 2021-08-18T22:25:21Z
closed_at: 2021-08-18T22:25:21Z
author_association: OWNER
type: issue
state_reason: completed
title:
  Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__`

body:
  > Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/312#issuecomment-896200682_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/314/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NjU0NDAwMTc=
title: `.delete_where()` returns `[]` when it should return self
state: closed
created_at: 2021-08-10T21:54:55Z
updated_at: 2021-08-10T23:09:29Z
closed_at: 2021-08-10T23:09:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  If the table doesn't exist it should still return `self`, not `[]`:

  https://github.com/simonw/sqlite-utils/blob/ee469e3122d6f5973ec2584c1580d930daca2e7c/sqlite_utils/db.py#L1676-L1683

  Spotted with `mypy` while working on #312.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/315/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5Njk1NDg5MzU=
title: UI for setting `?_size=max` on table page
state: open
created_at: 2021-08-12T20:52:09Z
updated_at: 2021-08-13T04:37:41Z
author_association: OWNER
type: issue
body:
  It defaults to 100 per page, but you can increase that to 1000 per page using `?_size=max` (or higher if `max_returned_rows` is set higher than that).

  But... that's only available to people who know how to hack URLs.

  Solution: add a link that sets that option to the pagination block at the bottom of the table:

  <img width="565" alt="data__states__28_999_rows" src="https://user-images.githubusercontent.com/9599/129268119-989dd835-3e04-4baa-8e9e-28e56ad59154.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1429/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzExNzgzNjE2
title: Column metadata
state: closed
created_at: 2021-08-12T23:34:39Z
updated_at: 2021-08-12T23:53:23Z
closed_at: 2021-08-12T23:53:23Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1430
body:
  Refs #942

  Still needs:

  - [x] Tests
  - [x] Documentation

type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1430/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5Njk4NDAzMDI=
title: `--help-config` should be called `--help-settings`
state: closed
created_at: 2021-08-13T00:46:48Z
updated_at: 2021-08-13T01:01:58Z
closed_at: 2021-08-13T01:01:58Z
author_association: OWNER
body: Follow-on from #1105 rebranding exercise.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1431/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5Njk4NTU3NzQ=
title: Rename Datasette.__init__(config=) parameter to settings=
state: open
created_at: 2021-08-13T01:00:27Z
updated_at: 2021-10-19T01:16:41Z
author_association: OWNER
type: issue
body:
  > While I'm doing this I should rename this internal variable to avoid confusion in the future:
  >
  > https://github.com/simonw/datasette/blob/e837095ef35ae155b4c78cc9a8b7133a48c94f03/datasette/app.py#L203

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1431#issuecomment-898072940_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1432/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzAzMjA2MTU=
title: Fix visible backticks on reference page
state: closed
created_at: 2021-08-13T11:37:46Z
updated_at: 2021-08-14T05:12:23Z
closed_at: 2021-08-14T05:10:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://sqlite-utils.datasette.io/en/latest/reference.html

  Search for backtick to reveal various minor markup bugs.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/316/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzEyMzQ2MTk5
title: Update trustme requirement from <0.9,>=0.7 to >=0.7,<0.10
state: closed
created_at: 2021-08-13T13:10:24Z
updated_at: 2021-08-25T01:29:27Z
closed_at: 2021-08-25T01:29:26Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1433
type: pull
body:
  Updates the requirements on [trustme](https://github.com/python-trio/trustme) to permit the latest version.
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/python-trio/trustme/commit/8fc5bf9c5afaf40843287e6eccca344713eddb50"><code>8fc5bf9</code></a> Bump version to 0.9.0</li>
  <li><a href="https://github.com/python-trio/trustme/commit/913e21dd94fc16a8b41742cc60338dcf638ae792"><code>913e21d</code></a> Bump types-cryptography from 3.3.3 to 3.3.5 (<a href="https://github-redirect.dependabot.com/python-trio/trustme/issues/342">#342</a>)</li>
  <li><a href="https://github.com/python-trio/trustme/commit/c66709d48bffc0d3a70d260313b752037ea71f8d"><code>c66709d</code></a> Bump types-pyopenssl from 20.0.4 to 20.0.5 (<a href="https://github-redirect.dependabot.com/python-trio/trustme/issues/343">#343</a>)</li>
  <li><a href="https://github.com/python-trio/trustme/commit/5131f79e9b7d73d993f901d8e48e6d744b0606ef"><code>5131f79</code></a> Add type annotations (<a href="https://github-redirect.dependabot.com/python-trio/trustme/issues/341">#341</a>)</li>
  <li><a href="https://github.com/python-trio/trustme/commit/a411dad9614f3fc2e746324d9a226e4947d55835"><code>a411dad</code></a> Bump charset-normalizer from 2.0.3 to 2.0.4 (<a href="https://github-redirect.dependabot.com/python-trio/trustme/issues/340">#340</a>)</li>
  <li><a href="https://github.com/python-trio/trustme/commit/be5ec8a3892ea218291ac93da8b382229b07f044"><code>be5ec8a</code></a> Bump sphinx from 4.1.1 to 4.1.2</li>
  <li><a href="https://github.com/python-trio/trustme/commit/d3b8865576a82189e06c735c64685fbad7ad02d0"><code>d3b8865</code></a> Bump charset-normalizer from 2.0.2 to 2.0.3</li>
  <li><a href="https://github.com/python-trio/trustme/commit/4503bef325bf0b23f4a0289654ee4c72d0c79fd1"><code>4503bef</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/python-trio/trustme/issues/334">#334</a> from python-trio/dependabot/pip/charset-normalizer-2.0.2</li>
  <li><a href="https://github.com/python-trio/trustme/commit/ce8099d9e2b59d610cf9858d095b4d7213341ba1"><code>ce8099d</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/python-trio/trustme/issues/335">#335</a> from python-trio/dependabot/pip/sphinx-4.1.1</li>
  <li><a href="https://github.com/python-trio/trustme/commit/8b6d3c660f4389b9d97f048f6eb2c11d201dda44"><code>8b6d3c6</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/python-trio/trustme/issues/336">#336</a> from python-trio/dependabot/pip/idna-3.2</li>
  <li>Additional commits viewable in <a href="https://github.com/python-trio/trustme/compare/v0.7.0...v0.9.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1433/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzEyNDEyODgz
title: Enrich arbitrary query results with foreign key links and column descriptions
state: open
created_at: 2021-08-13T14:43:01Z
updated_at: 2021-08-19T21:18:58Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1434
body: Refs #1293, follows #942.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1434/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzA2MjY2MjU=
title: Turn off suggest facets on tables with large numbers of columns
state: open
created_at: 2021-08-13T18:30:48Z
updated_at: 2021-08-13T18:30:48Z
author_association: OWNER
type: issue
body:
  If a table has 200 columns it will take multiple seconds to try and suggest facets. I should either quit after the first 20 or not suggest facets at all.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1435/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzI4MjczNDY=
title: Link to a better example on docs index
state: closed
created_at: 2021-08-17T15:43:40Z
updated_at: 2021-08-18T18:31:43Z
closed_at: 2021-08-18T18:31:43Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/simonw/sqlite-utils/blob/7a19822ac9ee24be2fbb4c2326a0bf2f3d2d9c4d/docs/index.rst#L39 Is a very old example

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/317/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzI5MTg1MzM=
state: open
created_at: 2021-08-17T17:35:36Z
updated_at: 2021-08-18T00:22:23Z
author_association: OWNER
type: issue
title:
  Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions

body:
  > Confirmed: https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A is a page where the URL correctly encoded `%` as `%25` - but then in the HTML on that page that links to the CSV and JSON versions we get this:
  >
  > ```html
  > <p class="export-links">This data as
  >   <a href="/thesession.json?sql=select * from tunes where name like &#34;%wise maid%&#34;">json</a>,
  >   <a href="/thesession.csv?sql=select * from tunes where name like &#34;%wise maid%&#34;&amp;_size=max">CSV</a>
  > </p>
  > ```
  Those CSV and JSON links are incorrect.

  _Originally posted by @simonw in https://github.com/simonw/datasette-publish-vercel/issues/48#issuecomment-900497579_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1438/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzMxMzkwNDc=
title: Rethink how .ext formats (v.s. ?_format=) works before 1.0
state: closed
created_at: 2021-08-17T23:32:51Z
updated_at: 2022-03-15T20:51:26Z
closed_at: 2022-03-15T20:51:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Datasette currently has surprising special behaviour for if a table name ends in `.csv` - which can happen when a tool like `csvs-to-sqlite` creates tables that match the filename that they were imported from.

  https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv illustrates this behaviour: it links to `.csv` and `.json` that look like this:

  - https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=json
  - https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=csv&_size=max

  Where normally Datasette would add the `.csv` or `.json` extension to the path component of the URL (as seen on other pages such as https://latest.datasette.io/fixtures/facet_cities) here the [path_with_format() function](https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/__init__.py#L710) notices that there is already a `.` in the path and instead adds `?_format=csv` to the query string instead.

  The problem with this mechanism is that it's pretty surprising. Anyone writing external code to Datasette who wants to get back the `.csv` or `.json` version giving the URL to a table page will need to know about and implement this behaviour themselves. That's likely to cause all kinds of bugs in the future.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1439/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzQwNjcxNTY=
title: Research: handle gzipped CSV directly
state: open
created_at: 2021-08-18T21:23:04Z
updated_at: 2021-08-18T21:25:30Z
author_association: OWNER
type: issue
body:
  Would it be worthwhile for the `sqlite-utils` command-line tool to grow features to efficiently directly interact with gzipped CSV data?

  Maybe add `--gz` options to both `insert` and to the various commands that output query results.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/318/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzQ5ODc4NTY=
title: Mechanism to cause specific branches to deploy their own demos
state: closed
created_at: 2021-08-19T19:41:39Z
updated_at: 2021-08-19T21:11:45Z
closed_at: 2021-08-19T21:09:40Z
author_association: OWNER
type: issue
state_reason: completed
body:
  A useful capability would be if it was super-easy to say "any pushes to branch X should be deployed to `latest-X.datasette.io`".

  I'd like to use this for the column query information work in #1434

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1442/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzQ5OTU1OTI=
title: datasette.databases should be a documented property
state: closed
created_at: 2021-08-19T19:53:04Z
updated_at: 2021-08-19T21:25:07Z
closed_at: 2021-08-19T21:23:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/app.py#L231

  I want to use it in https://github.com/simonw/datasette-block-robots/issues/5

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1443/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzE2MjYyODI5
title: Ability to deploy demos of branches
state: closed
created_at: 2021-08-19T21:08:04Z
updated_at: 2021-08-19T21:09:44Z
closed_at: 2021-08-19T21:09:39Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1444
body: See #1442.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1444/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzUxNTgyNjY=
title: table activity_summary has no column named appleMoveTime
state: closed
created_at: 2021-08-20T00:46:44Z
updated_at: 2021-08-20T00:54:34Z
closed_at: 2021-08-20T00:54:34Z
author_association: MEMBER
type: issue
state_reason: completed
body:
  Got this error today against a fresh export:

      table activity_summary has no column named appleMoveTime

reactions:
  {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzE2MzU3OTgy
title: Add --merged-by flag to pull-requests sub command
state: open
created_at: 2021-08-20T00:57:55Z
updated_at: 2021-09-28T21:50:31Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/github-to-sqlite/pulls/66
type: pull
body:
  ## Description

  Proposing a solution to the API limitation for `merged_by` in pull_requests. Specifically the following called out in the readme:

  ```
  Note that the merged_by column on the pull_requests table will only be populated for pull requests that are loaded using the --pull-request option - the GitHub API does not return this field for pull requests that are loaded in bulk.
  ```

  This approach might cause larger repos to hit rate limits called out in https://github.com/dogsheep/github-to-sqlite/issues/51 but seems to work well in the repos I tested and included below.

  ## Old Behavior
  - Had to list out the pull-requests individually via multiple `--pull-request` flags

  ## New Behavior

  - `--merged-by` flag for getting 'merge_by' information out of pull-requests without having to specify individual PR numbers.

  # Testing

  Picking some repo that has more than one merger (datasette only has 1 😉 )

  ```
  $ github-to-sqlite pull-requests ./github.db opnsense/tools --merged-by
  $ echo "select id, url, merged_by from pull_requests;" | sqlite3 ./github.db 
  83533612|https://github.com/opnsense/tools/pull/39|1915288
  102632885|https://github.com/opnsense/tools/pull/43|1915288
  149114810|https://github.com/opnsense/tools/pull/57|1915288
  160394495|https://github.com/opnsense/tools/pull/64|1915288
  163308408|https://github.com/opnsense/tools/pull/67|1915288
  169723264|https://github.com/opnsense/tools/pull/69|1915288
  171381422|https://github.com/opnsense/tools/pull/72|1915288
  179938195|https://github.com/opnsense/tools/pull/77|1915288
  196233824|https://github.com/opnsense/tools/pull/82|1915288
  215289964|https://github.com/opnsense/tools/pull/93|
  219696100|https://github.com/opnsense/tools/pull/97|1915288
  223664843|https://github.com/opnsense/tools/pull/99|
  228446172|https://github.com/opnsense/tools/pull/103|1915288
  238930434|https://github.com/opnsense/tools/pull/110|1915288
  255507110|https://github.com/opnsense/tools/pull/119|1915288
  255980675|https://github.com/opnsense/tools/pull/120|1915288
  261906770|https://github.com/opnsense/tools/pull/125|
  263800503|https://github.com/opnsense/tools/pull/127|1915288
  264038685|https://github.com/opnsense/tools/pull/128|1915288
  264696704|https://github.com/opnsense/tools/pull/129|1915288
  266660547|https://github.com/opnsense/tools/pull/130|1915288
  273120409|https://github.com/opnsense/tools/pull/133|1915288
  274370803|https://github.com/opnsense/tools/pull/135|
  276600629|https://github.com/opnsense/tools/pull/139|
  277303655|https://github.com/opnsense/tools/pull/141|1915288
  293033714|https://github.com/opnsense/tools/pull/145|
  294827649|https://github.com/opnsense/tools/pull/146|
  295140008|https://github.com/opnsense/tools/pull/147|1915288
  305690829|https://github.com/opnsense/tools/pull/150|9783985
  307077931|https://github.com/opnsense/tools/pull/152|1915288
  321782100|https://github.com/opnsense/tools/pull/155|
  337265672|https://github.com/opnsense/tools/pull/160|
  337267484|https://github.com/opnsense/tools/pull/161|1915288
  368251763|https://github.com/opnsense/tools/pull/169|
  428262505|https://github.com/opnsense/tools/pull/181|
  437557011|https://github.com/opnsense/tools/pull/182|1915288
  447079893|https://github.com/opnsense/tools/pull/185|
  461822092|https://github.com/opnsense/tools/pull/191|
  463290142|https://github.com/opnsense/tools/pull/193|1915288
  470112962|https://github.com/opnsense/tools/pull/194|1915288
  472644649|https://github.com/opnsense/tools/pull/195|1915288
  488696898|https://github.com/opnsense/tools/pull/198|
  513289902|https://github.com/opnsense/tools/pull/201|
  522530265|https://github.com/opnsense/tools/pull/203|
  564443347|https://github.com/opnsense/tools/pull/213|
  597579516|https://github.com/opnsense/tools/pull/220|1915288
  602860357|https://github.com/opnsense/tools/pull/221|1915288
  608744738|https://github.com/opnsense/tools/pull/222|1915288
  623279673|https://github.com/opnsense/tools/pull/228|1915288
  664656182|https://github.com/opnsense/tools/pull/233|
  664781786|https://github.com/opnsense/tools/pull/234|1915288
  670683636|https://github.com/opnsense/tools/pull/235|1915288
  683150764|https://github.com/opnsense/tools/pull/237|
  685016233|https://github.com/opnsense/tools/pull/238|
  687099825|https://github.com/opnsense/tools/pull/239|1915288
  715705652|https://github.com/opnsense/tools/pull/244|1915288
  715721248|https://github.com/opnsense/tools/pull/245|1915288
  ```
  `userid` are now present for those PRs that were merged.

  Without the flag the `merged_by` behavior remains missing as expected when get PRs bulk:

  ```
  $ github-to-sqlite pull-requests ./github.db opnsense/tools
  $ echo "select id, url, merged_by from pull_requests;" | sqlite3 ./github.db 
  83533612|https://github.com/opnsense/tools/pull/39|
  102632885|https://github.com/opnsense/tools/pull/43|
  149114810|https://github.com/opnsense/tools/pull/57|
  160394495|https://github.com/opnsense/tools/pull/64|
  163308408|https://github.com/opnsense/tools/pull/67|
  169723264|https://github.com/opnsense/tools/pull/69|
  171381422|https://github.com/opnsense/tools/pull/72|
  179938195|https://github.com/opnsense/tools/pull/77|
  196233824|https://github.com/opnsense/tools/pull/82|
  215289964|https://github.com/opnsense/tools/pull/93|
  219696100|https://github.com/opnsense/tools/pull/97|
  223664843|https://github.com/opnsense/tools/pull/99|
  228446172|https://github.com/opnsense/tools/pull/103|
  238930434|https://github.com/opnsense/tools/pull/110|
  255507110|https://github.com/opnsense/tools/pull/119|
  255980675|https://github.com/opnsense/tools/pull/120|
  261906770|https://github.com/opnsense/tools/pull/125|
  263800503|https://github.com/opnsense/tools/pull/127|
  264038685|https://github.com/opnsense/tools/pull/128|
  264696704|https://github.com/opnsense/tools/pull/129|
  266660547|https://github.com/opnsense/tools/pull/130|
  273120409|https://github.com/opnsense/tools/pull/133|
  274370803|https://github.com/opnsense/tools/pull/135|
  276600629|https://github.com/opnsense/tools/pull/139|
  277303655|https://github.com/opnsense/tools/pull/141|
  293033714|https://github.com/opnsense/tools/pull/145|
  294827649|https://github.com/opnsense/tools/pull/146|
  295140008|https://github.com/opnsense/tools/pull/147|
  305690829|https://github.com/opnsense/tools/pull/150|
  307077931|https://github.com/opnsense/tools/pull/152|
  321782100|https://github.com/opnsense/tools/pull/155|
  337265672|https://github.com/opnsense/tools/pull/160|
  337267484|https://github.com/opnsense/tools/pull/161|
  368251763|https://github.com/opnsense/tools/pull/169|
  428262505|https://github.com/opnsense/tools/pull/181|
  437557011|https://github.com/opnsense/tools/pull/182|
  447079893|https://github.com/opnsense/tools/pull/185|
  461822092|https://github.com/opnsense/tools/pull/191|
  463290142|https://github.com/opnsense/tools/pull/193|
  470112962|https://github.com/opnsense/tools/pull/194|
  472644649|https://github.com/opnsense/tools/pull/195|
  488696898|https://github.com/opnsense/tools/pull/198|
  513289902|https://github.com/opnsense/tools/pull/201|
  522530265|https://github.com/opnsense/tools/pull/203|
  564443347|https://github.com/opnsense/tools/pull/213|
  597579516|https://github.com/opnsense/tools/pull/220|
  602860357|https://github.com/opnsense/tools/pull/221|
  608744738|https://github.com/opnsense/tools/pull/222|
  623279673|https://github.com/opnsense/tools/pull/228|
  664656182|https://github.com/opnsense/tools/pull/233|
  664781786|https://github.com/opnsense/tools/pull/234|
  670683636|https://github.com/opnsense/tools/pull/235|
  683150764|https://github.com/opnsense/tools/pull/237|
  685016233|https://github.com/opnsense/tools/pull/238|
  687099825|https://github.com/opnsense/tools/pull/239|
  715705652|https://github.com/opnsense/tools/pull/244|
  715721248|https://github.com/opnsense/tools/pull/245|
  ```

  Individual PRs passed via `--pull-request` flag behaves as expected (unchanged):

  ```
  $ github-to-sqlite pull-requests ./github.db opnsense/tools --pull-request 39 --pull-request 237
  $ echo "select id, url, merged_by from pull_requests;" | sqlite3 ./github.db
  83533612|https://github.com/opnsense/tools/pull/39|1915288
  683150764|https://github.com/opnsense/tools/pull/237|
  ```
  > Picking 1 PR that has a merged_by (39) and one that does not (237)

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/66/reactions", "total_count": 3, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzUxNjYyNzE=
title: Add index on workout_points.date
state: open
created_at: 2021-08-20T01:08:04Z
updated_at: 2021-08-20T01:12:48Z
author_association: MEMBER
type: issue
body:
  Sorting that by date makes sense for seeing most recent points, and my DB has 2.5m points in so it's an expensive sort!

reactions:
  {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzYzOTk2Mzg=
title: [Enhancement] Please allow 'insert-files' to insert content as text.
state: closed
created_at: 2021-08-22T15:10:46Z
updated_at: 2021-08-24T23:33:45Z
closed_at: 2021-08-24T23:33:44Z
author_association: NONE
type: issue
state_reason: completed
body:
  'insert-files' creates BLOB columns for file contents. Transforming the column to TEXT still keep the content as binary. Even though I'm sure there is a transform that can be applied decoding the text it would be great to have a argument to make 'insert-files' to do it as text (with optional text encoding).

  The use case is a bunch of htmls (single file) on a directory structure that inserted with this command could be served in Datasette allowing full text search.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/319/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzY0MDUyMjU=
title: sqlite-utils memory --analyze option
state: closed
created_at: 2021-08-22T15:37:10Z
updated_at: 2021-08-22T15:46:56Z
closed_at: 2021-08-22T15:44:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  To provide a way of running [analyze-tables](https://sqlite-utils.datasette.io/en/stable/cli.html#analyzing-tables) directly against JSON or CSV data.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/320/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzcxMjg5MzU=
title: Duplicate Column
state: open
created_at: 2021-08-23T15:00:44Z
updated_at: 2021-08-23T17:00:59Z
author_association: NONE
type: issue
body:
  Hey, thank you for this repo!

  When I try to convert my export, I get a multiple column error. Here is the stack trace:

  ```sh
  (.venv) (base) computer:bodyweight_app user$ healthkit-to-sqlite ./data/Health_export.zip ./data/healthkit.db
  Importing from HealthKit  [###############################-----]   87%  00:00:22
  Traceback (most recent call last):
    File "/MyProject/.venv/bin/healthkit-to-sqlite", line 10, in <module>
      sys.exit(cli())
    File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
      return self.main(*args, **kwargs)
    File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 782, in main
      rv = self.invoke(ctx)
    File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/MyProject/.venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
      return callback(*args, **kwargs)
    File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/cli.py", line 57, in cli
      convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)
    File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 41, in convert_xml_to_sqlite
      write_records(records, db)
    File "/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py", line 146, in write_records
      batch_size=50,
    File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 2579, in insert_all
      extracts=extracts,
    File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 1246, in create
      extracts=extracts,
    File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 767, in create_table
      self.execute(sql)
    File "/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py", line 421, in execute
      return self.conn.execute(sql)
  sqlite3.OperationalError: duplicate column name: metadata_Meal
  ```

reactions:
  {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/21/reactions", "total_count": 5, "+1": 5, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzczMjMxMzM=
title: Ability to search for text across all columns in a table
state: open
created_at: 2021-08-23T18:50:48Z
updated_at: 2021-08-23T19:10:17Z
author_association: OWNER
type: issue
body:
  When I'm working with new data I often find myself wanting to run a search for text embedded in ANY of the columns of a table, without having to even fully understand the schema first.

  I figured out a trick for doing that using a SQL-generated SQL query here: https://til.simonwillison.net/datasette/search-all-columns-trick

  But maybe this should be a core Datasette feature? Or a plugin?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1445/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzE4NzM0MTkx
title: Make sure that case-insensitive column names are unique
state: open
created_at: 2021-08-24T13:13:38Z
updated_at: 2021-08-24T13:26:20Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/healthkit-to-sqlite/pulls/22
type: pull
body:
  This closes #21.

  When there are metadata entries with the same case insensitive string, then there is an error when trying to create a new column for that metadata entry in the database table, because a column with that case insensitive name already exists.

  ```xml
   <Record type="HKQuantityTypeIdentifierDietaryCholesterol" sourceName="MyFitnessPal" sourceVersion="35120" unit="mg" creationDate="2021-07-04 20:55:27 +0200" startDate="2021-07-04 20:55:00 +0200" endDate="2021-07-04 20:55:00 +0200" value="124">
    <MetadataEntry key="meal" value="Dinner"/>
    <MetadataEntry key="Meal" value="Dinner"/>
   </Record>
  ```

  The code added in this PR checks if a key already exists in a record and if so adds a number at its end. The resulting column names look like the example below then. Interestingly, the column names viewed with Datasette are not case insensitive.

  ```text
  startDate, endDate, value, unit, sourceName, sourceVersion, creationDate, metadata_meal, metadata_Meal_2, metadata_Mahlzeit
  ```

reactions:
  {"url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5NzgzNTc5ODQ=
title: Modify base.html template to support optional sticky footer
state: closed
created_at: 2021-08-24T18:11:12Z
updated_at: 2021-08-31T01:54:59Z
closed_at: 2021-08-24T20:32:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The neatest way to have the footer stick to the bottom of the browser window that I've found is to use the flexbox pattern from https://css-tricks.com/couple-takes-sticky-footer/

  ```html
  <body>
    <div class="content">
      content
    </div>
    <footer class="footer"></footer>
  </body>
  ```
  ```css
  html, body {
    height: 100%;
  }
  body {
    display: flex;
    flex-direction: column;
  }
  .content {
    flex: 1 0 auto;
  }
  .footer {
    flex-shrink: 0;
  }
  ```
  I tried this in a custom plugin but it ended up having to duplicate the entire `base.html` template just to get a wrapper around the not-footer content.

  I think Datasette's own `base.html` template should have this wrapper element instead.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1446/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzE5MTA5NzA5
title: Ability to insert file contents as text, in addition to blob
state: closed
created_at: 2021-08-24T22:37:18Z
updated_at: 2021-08-24T23:31:17Z
closed_at: 2021-08-24T23:31:13Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/321
body: Refs #319.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/321/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzE5MTc1NTkz
title: Remove underscore from search mode parameter name
state: closed
created_at: 2021-08-25T01:28:04Z
updated_at: 2021-08-25T01:28:58Z
closed_at: 2021-08-25T01:28:58Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1447
type: pull
body:
  The fulltext search documentation refers to the parameter as `searchmode` but the `metadata.json` example uses `search_mode`. The latter doesn't actually seem to work.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1447/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5Nzg3NDM0MjY=
title: xml.etree.ElementTree.ParseError: not well-formed (invalid token)
state: closed
created_at: 2021-08-25T05:48:21Z
updated_at: 2021-08-26T18:45:13Z
closed_at: 2021-08-26T18:45:13Z
author_association: MEMBER
type: issue
state_reason: completed
body:
  Got this error today:
  ```
  (evernote-to-sqlite) /tmp % evernote-to-sqlite enex evernote.db simonwillison\'s\ notebook.enex 
  Importing from ENEX  [######------------------------------]   17%
  Traceback (most recent call last):
    File "/Users/simon/.local/bin/evernote-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
      return self.main(*args, **kwargs)
    File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1062, in main
      rv = self.invoke(ctx)
    File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1668, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 763, in invoke
      return __callback(*args, **kwargs)
    File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/cli.py", line 31, in enex
      save_note(db, note)
    File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/utils.py", line 36, in save_note
      content = ET.tostring(ET.fromstring(content_xml)).decode("utf-8")
    File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/xml/etree/ElementTree.py", line 1347, in XML
      parser.feed(text)
  xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 2, column 132
  ```

reactions:
  {"url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzE5OTk4MjI1
title: Add dict type to be mapped as TEXT in sqllite
state: closed
created_at: 2021-08-25T20:54:26Z
updated_at: 2021-11-15T00:27:40Z
closed_at: 2021-11-15T00:27:40Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/322
type: pull
body:
  the library deal with Postgres type jsonb as dictionary, add dict type as a TEXT for mapping to sqlite

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/322/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5Nzk2MjcyODU=
title: `table.convert()` method should clean up after itself
state: closed
created_at: 2021-08-25T21:15:39Z
updated_at: 2021-08-25T21:25:26Z
closed_at: 2021-08-25T21:25:18Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It currently works like this: https://github.com/simonw/sqlite-utils/blob/77c240df56068341561e95e4a412cbfa24dc5bc7/sqlite_utils/db.py#L2177-L2195

  It's registering a function called `convert_value()` and then failing to de-register that function once it has finished.

  It might even be possible for two queries running against the same connection to clobber each other's `convert_value()` functions, leading to incorrect behaviour.

  So two fixes: firstly it should register the function with a unique name (maybe add a random suffix). Secondly, it should de-register that function once it has finished.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/323/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzIwNTA2MTM1
title: Update pluggy requirement from ~=0.13.0 to >=0.13,<1.1
state: closed
created_at: 2021-08-26T13:09:52Z
updated_at: 2021-10-13T21:11:01Z
closed_at: 2021-10-13T21:11:00Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1448
type: pull
body:
  Updates the requirements on [pluggy](https://github.com/pytest-dev/pluggy) to permit the latest version.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pluggy/blob/main/CHANGELOG.rst">pluggy's changelog</a>.</em></p>
  <blockquote>
  <h1>pluggy 1.0.0 (2021-08-25)</h1>
  <h2>Deprecations and Removals</h2>
  <ul>
  <li>
  <p><code>[#116](https://github.com/pytest-dev/pluggy/issues/116) &lt;https://github.com/pytest-dev/pluggy/issues/116&gt;</code>_: Remove deprecated <code>implprefix</code> support.
  Decorate hook implementations using an instance of HookimplMarker instead.
  The deprecation was announced in release <code>0.7.0</code>.</p>
  </li>
  <li>
  <p><code>[#120](https://github.com/pytest-dev/pluggy/issues/120) &lt;https://github.com/pytest-dev/pluggy/issues/120&gt;</code>_: Remove the deprecated <code>proc</code> argument to <code>call_historic</code>.
  Use <code>result_callback</code> instead, which has the same behavior.
  The deprecation was announced in release <code>0.7.0</code>.</p>
  </li>
  <li>
  <p><code>[#265](https://github.com/pytest-dev/pluggy/issues/265) &lt;https://github.com/pytest-dev/pluggy/issues/265&gt;</code>_: Remove the <code>_Result.result</code> property. Use <code>_Result.get_result()</code> instead.
  Note that unlike <code>result</code>, <code>get_result()</code> raises the exception if the hook raised.
  The deprecation was announced in release <code>0.6.0</code>.</p>
  </li>
  <li>
  <p><code>[#267](https://github.com/pytest-dev/pluggy/issues/267) &lt;https://github.com/pytest-dev/pluggy/issues/267&gt;</code>_: Remove official support for Python 3.4.</p>
  </li>
  <li>
  <p><code>[#272](https://github.com/pytest-dev/pluggy/issues/272) &lt;https://github.com/pytest-dev/pluggy/issues/272&gt;</code>_: Dropped support for Python 2.
  Continue to use pluggy 0.13.x for Python 2 support.</p>
  </li>
  <li>
  <p><code>[#308](https://github.com/pytest-dev/pluggy/issues/308) &lt;https://github.com/pytest-dev/pluggy/issues/308&gt;</code>_: Remove official support for Python 3.5.</p>
  </li>
  <li>
  <p><code>[#313](https://github.com/pytest-dev/pluggy/issues/313) &lt;https://github.com/pytest-dev/pluggy/issues/313&gt;</code>_: The internal <code>pluggy.callers</code>, <code>pluggy.manager</code> and <code>pluggy.hooks</code> are now explicitly marked private by a <code>_</code> prefix (e.g. <code>pluggy._callers</code>).
  Only API exported by the top-level <code>pluggy</code> module is considered public.</p>
  </li>
  <li>
  <p><code>[#59](https://github.com/pytest-dev/pluggy/issues/59) &lt;https://github.com/pytest-dev/pluggy/issues/59&gt;</code>_: Remove legacy <code>__multicall__</code> recursive hook calling system.
  The deprecation was announced in release <code>0.5.0</code>.</p>
  </li>
  </ul>
  <h2>Features</h2>
  <ul>
  <li>
  <p><code>[#282](https://github.com/pytest-dev/pluggy/issues/282) &lt;https://github.com/pytest-dev/pluggy/issues/282&gt;</code>_: When registering a hookimpl which is declared as <code>hookwrapper=True</code> but whose
  function is not a generator function, a <code>PluggyValidationError</code> exception is
  now raised.</p>
  <p>Previously this problem would cause an error only later, when calling the hook.</p>
  <p>In the unlikely case that you have a hookwrapper that <em>returns</em> a generator</p>
  </li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/4259fdd799d64f953631e47ddcb69c3074a15c13"><code>4259fdd</code></a> Fix CHANGELOG title manually</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/906abca9bb403729f4df44d02142698ddd54d5b5"><code>906abca</code></a> Preparing release 1.0.0</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/56eb23c89aa0a43fa7b139894fbf6142684c2969"><code>56eb23c</code></a> Rename HOWTORELEASE to RELEASING to follow pytest</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/fc6395ca27d5deb008db250da57a23dd40bc93a4"><code>fc6395c</code></a> Fix scripts/release.py to use main instead of master</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/e04816f71b3d40bc959550b93f1d69098fc045fd"><code>e04816f</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pluggy/issues/324">#324</a> from RonnyPfannschmidt/benchmarks</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/1424ab0a6a54a878b171c806a39c79d54817c531"><code>1424ab0</code></a> add micro benchmarks for hook calling playing with a the number of callers, w...</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/5e518648987a3afced3c2b014c73abab78f8306f"><code>5e51864</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pluggy/issues/323">#323</a> from RonnyPfannschmidt/switch-to-main</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/05c3bbd07c7186d3c124f1009aa6c8999ba70c2c"><code>05c3bbd</code></a> switch to main as primary branch</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/6b344fbebc598a623bb8db6b58f6f04b27916515"><code>6b344fb</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pluggy/issues/319">#319</a> from RonnyPfannschmidt/pre-commit-update</li>
  <li><a href="https://github.com/pytest-dev/pluggy/commit/71f2d6b4a21381314f730dd5b5f93519882f79da"><code>71f2d6b</code></a> introduce pyupgrade and update black</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pluggy/compare/0.13.0...1.0.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1448/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODE2NzY4MzI=
title: `register_commands()` plugin hook to register extra CLI commands
state: closed
created_at: 2021-08-28T00:26:21Z
updated_at: 2021-08-28T01:58:35Z
closed_at: 2021-08-28T01:43:11Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The `datasette` CLI tool currently has 7 subcommands: `serve`, `inspect`, `install`, `package`, `plugins`, `publish` and `uninstall`.

  A plugin hook could allow plugins to register extra subcommands.

  I've avoided this for quite a while because I didn't have good use-cases for it - but the existence of the `datasette install xxx` command for installing packages into the correct virtual environment means that actually there's a good reason to do this: it would allow plugins to provide additional command-line mechanisms without the user having to understand how virtual environments work in order to install those commands into the same environment as the rest of Datasette.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1449/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODE2ODExMzg=
title: Datasette --help should show something more useful than "Datasette!"
state: closed
created_at: 2021-08-28T00:44:51Z
updated_at: 2021-08-28T00:49:07Z
closed_at: 2021-08-28T00:49:07Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/simonw/datasette/blob/a1a33bb5822214be1cebd98cd858b2058d91a4aa/datasette/cli.py#L122-L127

  _Originally spotted by @simonw in https://github.com/simonw/datasette/issues/1449#issuecomment-907539668_

  ```
  ~ % datasette --help
  Usage: datasette [OPTIONS] COMMAND [ARGS]...

    Datasette!

  Options:
    --version  Show the version and exit.
    --help     Show this message and exit.
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1450/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzIxNjg2NzIx
title: Replacing step ID key with step_id
state: open
created_at: 2021-08-28T01:26:41Z
updated_at: 2021-08-28T01:27:00Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/github-to-sqlite/pulls/67
type: pull
body:
  Workflows that have an `id` in any step result in the following error when running `workflows`:

  e.g.`github-to-sqlite workflows github.db nixos/nixpkgs`

  ```Traceback (most recent call last):
    File "/usr/local/bin/github-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1137, in __call__
      return self.main(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1062, in main
      rv = self.invoke(ctx)
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1668, in invoke```Traceback (most recent call last):
    File "/usr/local/bin/github-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1137, in __call__
      return self.main(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1062, in main
      rv = self.invoke(ctx)
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1668, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 763, in invoke
      return __callback(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/github_to_sqlite/cli.py", line 601, in workflows
      utils.save_workflow(db, repo_id, filename, content)
    File "/usr/local/lib/python3.8/dist-packages/github_to_sqlite/utils.py", line 865, in save_workflow
      db["steps"].insert_all(
    File "/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py", line 2596, in insert_all
      self.insert_chunk(
    File "/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py", line 2378, in insert_chunk
      result = self.db.execute(query, params)
    File "/usr/local/lib/python3.8/dist-packages/sqlite_utils/db.py", line 419, in execute
      return self.conn.execute(sql, parameters)
  sqlite3.IntegrityError: datatype mismatch
  ```

   - [Information about the ID key in a step for GHA](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsid)
   - [An example workflow from a public repo](https://github.com/NixOS/nixpkgs/blob/b4cc66827745e525ce7bb54659845ac89788a597/.github/workflows/direct-push.yml#L16)

  # Changes
  I'm proposing that the key for `id` in step is replaced with `step_id` so that it no longer interferes with the table `id` for tracking the record.

  Special thanks to @sarcasticadmin @egiffen and @ruebenramirez for helping a bit on this 😄 

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/67/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzIyNDgwNTQy
title: Bump black from 21.7b0 to 21.8b0
state: closed
created_at: 2021-08-30T13:13:39Z
updated_at: 2021-09-14T13:10:40Z
closed_at: 2021-09-14T13:10:38Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1453
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 21.7b0 to 21.8b0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>21.8b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Add support for formatting Jupyter Notebook files (<a href="https://github-redirect.dependabot.com/psf/black/issues/2357">#2357</a>)</li>
  <li>Move from <code>appdirs</code> dependency to <code>platformdirs</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2375">#2375</a>)</li>
  <li>Present a more user-friendly error if .gitignore is invalid (<a href="https://github-redirect.dependabot.com/psf/black/issues/2414">#2414</a>)</li>
  <li>The failsafe for accidentally added backslashes in f-string expressions has been hardened to handle more edge cases during quote normalization (<a href="https://github-redirect.dependabot.com/psf/black/issues/2437">#2437</a>)</li>
  <li>Avoid changing a function return type annotation's type to a tuple by adding a trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2384">#2384</a>)</li>
  <li>Parsing support has been added for unparenthesized walruses in set literals, set comprehensions, and indices (<a href="https://github-redirect.dependabot.com/psf/black/issues/2447">#2447</a>).</li>
  <li>Pin <code>setuptools-scm</code> build-time dependency version (<a href="https://github-redirect.dependabot.com/psf/black/issues/2457">#2457</a>)</li>
  <li>Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2460">#2460</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that scan installed packages (<a href="https://github-redirect.dependabot.com/psf/black/issues/2440">#2440</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>The provided pre-commit hooks no longer specify <code>language_version</code> to avoid overriding <code>default_language_version</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2430">#2430</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>21.8b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Add support for formatting Jupyter Notebook files (<a href="https://github-redirect.dependabot.com/psf/black/issues/2357">#2357</a>)</li>
  <li>Move from <code>appdirs</code> dependency to <code>platformdirs</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2375">#2375</a>)</li>
  <li>Present a more user-friendly error if .gitignore is invalid (<a href="https://github-redirect.dependabot.com/psf/black/issues/2414">#2414</a>)</li>
  <li>The failsafe for accidentally added backslashes in f-string expressions has been
  hardened to handle more edge cases during quote normalization (<a href="https://github-redirect.dependabot.com/psf/black/issues/2437">#2437</a>)</li>
  <li>Avoid changing a function return type annotation's type to a tuple by adding a
  trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2384">#2384</a>)</li>
  <li>Parsing support has been added for unparenthesized walruses in set literals, set
  comprehensions, and indices (<a href="https://github-redirect.dependabot.com/psf/black/issues/2447">#2447</a>).</li>
  <li>Pin <code>setuptools-scm</code> build-time dependency version (<a href="https://github-redirect.dependabot.com/psf/black/issues/2457">#2457</a>)</li>
  <li>Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/2460">#2460</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that
  scan installed packages (<a href="https://github-redirect.dependabot.com/psf/black/issues/2440">#2440</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>The provided pre-commit hooks no longer specify <code>language_version</code> to avoid overriding
  <code>default_language_version</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2430">#2430</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/psf/black/commits">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.7b0&new-version=21.8b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1453/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODI4MDM0MDg=
title: Feature Request: Publish to IPFS
state: open
created_at: 2021-08-30T13:36:18Z
updated_at: 2021-08-30T13:36:18Z
author_association: NONE
type: issue
body:
  Hello,

  I am a huge fan of this being used for exploring data.  I think it has a lot of flexibility not found in other tools.

  I'm not sure if what I'm asking for is possible:  Can this be extended to publish to IPFS?

  IPFS is an attractive hosting option for decentralized journalism.

  Food for thought ~

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1454/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODMyMjE4NTE=
title: Data folder as index command parameter
state: open
created_at: 2021-08-30T21:29:33Z
updated_at: 2021-08-30T21:29:33Z
author_association: NONE
type: issue
body:
  Hi,

  First of all, thank you for this wonderful project :smile:

  I started to use dogsheep to make my personal data searchable, and by using the project I noticed an issue with the index command.

  It always expects you are running it from the root folder from where the data is located, so I got some errors while trying to make it work on my setup.

  I separate all databases inside a `data` folder (I published my setup to be easier to follow: https://github.com/humrochagf/my-dogsheep)

  Before, I configured `dogsheep.yml` to add the data folder to its path like this:

  ```yml
  data/twitter.db:
      tweets:
          sql: |-
  ...
  ```

  And running the index command like this:

  ```
  dogsheep-beta index data/dogsheep.db dogsheep.yml
  ```

  It worked to the normal search feature with no problem this way, but when I started adding `display_sql` rules the app started to crash, because at datasette `get_database` it was looking for `data/twitter` and it only had a db called `twitter` there.

  So my workaround to that was to cd into the data folder and run the indexer. You can check the way I'm doing it at this line of the makefile: https://github.com/humrochagf/my-dogsheep/blob/main/makefile#L3

  It works but it would be nice to have an option to pass the path where the data is located to the index function.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/34/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODQ5MzkzNjY=
title: Error: Use either --since or --since_id, not both - still broken
state: closed
created_at: 2021-09-01T09:45:28Z
updated_at: 2021-09-21T17:37:41Z
closed_at: 2021-09-21T17:37:41Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Hi Simon,

  It appears the fix for #57 doesn't fix things for me:

  ```
  $ twitter-to-sqlite --version
  twitter-to-sqlite, version 0.21.4
  $ python --version
  Python 3.9.6
  ```

  ```
  $ twitter-to-sqlite home-timeline -a twitter-auth.json twitter/timeline.db --since
  Importing tweets
  Error: Use either --since or --since_id, not both
  ```

  Is there any way I can help debug this?

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/58/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI0MzE3NjUw
title: Fix for since_id bug, closes #58
state: closed
created_at: 2021-09-01T09:49:09Z
updated_at: 2021-09-21T17:37:40Z
closed_at: 2021-09-21T17:37:40Z
author_association: CONTRIBUTOR
pull_request: dogsheep/twitter-to-sqlite/pulls/59
body: Fixes remaining instances of this bug
type: pull
reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/59/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODY4MjkxOTQ=
title: xml.etree.ElementTree.Parse Error - mismatched tag
state: open
created_at: 2021-09-02T14:46:36Z
updated_at: 2021-09-02T14:53:11Z
author_association: NONE
type: issue
body:
  This is an error message I get upon parsing the enex file of my Inbox. Please find the full error message below. Any hints welcome.

  ```
  Importing from ENEX  [##################------------------]   50%  00:00:50
  Traceback (most recent call last):
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/bin/evernote-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
      return self.main(*args, **kwargs)
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/lib/python3.9/site-packages/click/core.py", line 1062, in main
      rv = self.invoke(ctx)
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/lib/python3.9/site-packages/click/core.py", line 1668, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/lib/python3.9/site-packages/click/core.py", line 763, in invoke
      return __callback(*args, **kwargs)
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/lib/python3.9/site-packages/evernote_to_sqlite/cli.py", line 30, in enex
      for tag, note in find_all_tags(fp, ["note"], progress_callback=bar.update):
    File "/Users/utopist/.virtualenvs/evernote-to-sqlite-Og2PIW3Y/lib/python3.9/site-packages/evernote_to_sqlite/utils.py", line 17, in find_all_tags
      for event, el in parser.read_events():
    File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/xml/etree/ElementTree.py", line 1329, in read_events
      raise event
    File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/xml/etree/ElementTree.py", line 1301, in feed
      self._parser.feed(data)
  xml.etree.ElementTree.ParseError: mismatched tag: line 6837961, column 2
  ```

reactions:
  {"url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/14/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI2OTkwNjgw
title: Support for Datasette's --base-url setting
state: open
created_at: 2021-09-03T17:47:45Z
updated_at: 2021-09-03T17:47:45Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/dogsheep-beta/pulls/35
type: pull
body:
  This makes it so you can use Dogsheep if you're using Datasette with the `--base-url /some-path/` setting.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/35/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI3MDEyOTk2
title: Use python-dateutil package instead of dateutils
state: closed
created_at: 2021-09-03T18:31:19Z
updated_at: 2021-11-14T23:25:40Z
closed_at: 2021-11-14T23:25:40Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/324
type: pull
body:
  While working on updating `sqlite-utils` for NixOS/Nixpkgs, I came a cross the following:

  In 5ec6686153e29ae10d4921a1ad4c841f192f20e2, a new dependency was added on `dateutils` (https://pypi.org/project/dateutils/). 

  I believe this is unintentional, and instead `python-dateutil` (https://pypi.org/project/python-dateutil/) was intended.

  My reasoning is:
  - `python-dateutil` is imported here in [recipes.py](https://github.com/simonw/sqlite-utils/blob/5ec6686153e29ae10d4921a1ad4c841f192f20e2/sqlite_utils/recipes.py#L1) 
  - The `mypy` `type-python-dateutil` dependency in [setup.py](https://github.com/simonw/sqlite-utils/blob/5ec6686153e29ae10d4921a1ad4c841f192f20e2/setup.py#L36)
  - `python-dateutil` is a dependency of `dateutils` as seen in the output in [docs/tutorial.ipynb](https://github.com/simonw/sqlite-utils/blob/77c240df56068341561e95e4a412cbfa24dc5bc7/docs/tutorial.ipynb#L43)

  Seems like the trailing "s" seems to be the source of confusion 😅

  I've swapped the dependencies out, hope this helps.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/324/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI3MjY1MDI1
title: Add scientists to target groups
state: closed
created_at: 2021-09-04T16:28:58Z
updated_at: 2021-09-04T16:32:21Z
closed_at: 2021-09-04T16:31:38Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1455
type: pull
body:
  Not sure if you want them mentioned explicitly (it's already a long list), but following up on https://twitter.com/simonw/status/1434176989565382656

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1455/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI3MzkwODM1
title: Correct naming of tool in readme
state: open
created_at: 2021-09-05T12:05:40Z
updated_at: 2022-01-06T16:04:46Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/dogsheep-photos/pulls/36
type: pull
reactions:
  {"url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/36/reactions", "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODg1NTI4NTE=
title: conda install results in non-functioning `datasette serve` due to out-of-date asgiref
state: open
created_at: 2021-09-05T16:59:55Z
updated_at: 2021-09-05T16:59:55Z
author_association: CONTRIBUTOR
type: issue
body:
  Over in https://github.com/ctb/2021-sourmash-datasette, I discovered that the following commands fail:
  ```
  conda create -n datasette4 -y datasette=0.58.1
  conda activate datasette4
  datasette gathertax.db
  ```
  with `ImportError: cannot import name 'WebSocketScope' from 'asgiref.typing'`.

  This appears to be because asgiref 3.3.4 doesn't have WebSocketScope, but later versions do - a simple

  ```
  pip install asgiref==3.4.1
  ```

  fixes the problem for me, at least to the point where I can run datasette and poke around as usual.

  I note that over in the conda-forge recipe, https://github.com/conda-forge/datasette-feedstock/blob/master/recipe/meta.yaml pins asgiref to < 3.4.0, but I'm not sure why - so I'm not sure how to best resolve this issue :).

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1456/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODg1NTM4MDY=
title: suggestion: distinguish names in `--static` documentation
state: closed
created_at: 2021-09-05T17:04:27Z
updated_at: 2021-10-14T18:39:55Z
closed_at: 2021-10-14T18:39:55Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Over in https://docs.datasette.io/en/stable/custom_templates.html#serving-static-files, there is the slightly comical example command -

  ```
  datasette -m metadata.json --static static:static/ --memory
  ```

  (now, with MORE STATIC!)

  It took me a while to sort out all the URLs and paths involved because I wasn't being very clever. But in the interests of simplification and distinction, I might suggest something like

  ```
  datasette -m metadata.json --static loc:static-files/ --memory
  ```

  I will submit a PR for your consideration.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1457/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI3NDM2NDA0
title: Rework the `--static` documentation a bit
state: closed
created_at: 2021-09-05T17:08:48Z
updated_at: 2021-10-15T13:24:29Z
closed_at: 2021-10-14T18:39:55Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1458
type: pull
body:
  Per https://github.com/simonw/datasette/issues/1457, I was confused by the current docs and took a few minutes to sort out what the right combination of locations was.

  This is a PR that differentiates the docs to split out  `/static/` in URL from `--static` option and `./static/` path.

  Not wedded to the details in any way, happy to change to suit.

  Fixes #1457.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1458/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODg1NTY0ODg=
title: suggestion: allow `datasette --open` to take a relative URL
state: open
created_at: 2021-09-05T17:17:07Z
updated_at: 2021-09-05T19:59:15Z
author_association: CONTRIBUTOR
type: issue
body:
  (soft suggestion because I'm not sure I'm using datasette right yet)

  Over at https://github.com/ctb/2021-sourmash-datasette, I'm playing around with datasette, and I'm creating some static pages to send people to the right facets. There may well be better ways of achieving this end goal, and I will find out if so, I'm sure!

  But regardless I think it might be neat to support an option to allow `-o/--open` to take a relative URL, that then gets appended to the hostname and port. This would let me improve my documentation. I don't see any downsides, either, but 🤷 there may well be some :)

  Happy to dig in and provide a PR if it's of interest. I'm not sure off the top of my head how to support an optional value to a parameter in argparse - the current `-o` behavior is kinda nice so it'd be suboptimal to require a url for `-o`. Maybe `--open-url=` or something would work?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1459/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODkxMDk4ODg=
title: Override column metadata with metadata from another column
state: open
created_at: 2021-09-06T12:13:33Z
updated_at: 2021-09-06T12:13:33Z
author_association: CONTRIBUTOR
type: issue
body:
  I have a table from the PUDL project (https://github.com/catalyst-cooperative/pudl) that looks like this:

  ```
  CREATE TABLE fuel_ferc1 (
  	id INTEGER NOT NULL, 
  	record_id TEXT, 
  	utility_id_ferc1 INTEGER, 
  	report_year INTEGER, 
  	plant_name_ferc1 TEXT, 
  	fuel_type_code_pudl VARCHAR(7), 
  	fuel_unit VARCHAR(7),
  	fuel_qty_burned FLOAT,
  	fuel_mmbtu_per_unit FLOAT, 
  	fuel_cost_per_unit_burned FLOAT, 
  	fuel_cost_per_unit_delivered FLOAT, 
  	fuel_cost_per_mmbtu FLOAT, 
  	PRIMARY KEY (id), 
  	FOREIGN KEY(plant_name_ferc1, utility_id_ferc1) REFERENCES plants_ferc1 (plant_name_ferc1, utility_id_ferc1), 
  	CONSTRAINT fuel_ferc1_fuel_type_code_pudl_enum CHECK (fuel_type_code_pudl IN ('coal', 'oil', 'gas', 'solar', 'wind', 'hydro', 'nuclear', 'waste', 'unknown')), 
  	CONSTRAINT fuel_ferc1_fuel_unit_enum CHECK (fuel_unit IN ('ton', 'mcf', 'bbl', 'gal', 'kgal', 'gramsU', 'kgU', 'klbs', 'btu', 'mmbtu', 'mwdth', 'mwhth', 'unknown'))
  );
  ```

  Note that `fuel_unit` is a unit that **pint** can understand, and that `fuel_qty_burned` is a column of data that could be expressed in terms of actual units, not merely as a dimensionless number.  Ditto the `fuel_cost_per_unit_...` columns.  Is there a way to give a column a default metadata unit (such as *tons* or *USD/ton*) and then let that be overridden when the metadata in another column says *barrels* or *USD/gramsU*?

  @catalyst-cooperative

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1460/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5ODk5ODY1ODY=
title: Try blacken-docs
state: closed
created_at: 2021-09-07T13:28:50Z
updated_at: 2021-09-07T16:13:59Z
closed_at: 2021-09-07T16:13:59Z
author_association: OWNER
body: https://github.com/asottile/blacken-docs
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1461/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5OTAzNjc2NDY=
title: Separate out "debug" options from "root" options
state: open
created_at: 2021-09-07T21:27:34Z
updated_at: 2021-09-07T21:34:33Z
author_association: OWNER
type: issue
body:
  > I ditched "root" for "admin" because root by default gives you a whole bunch of stuff which I think could be confusing:
  > 
  > <img width="816" alt="Datasette___internal__temporary_and_datasette-app_—_Electron_Helper__Renderer__◂_npm_start_TMPDIR__var_folders_wr_hn3206rs1yzgq3r49bz8nvnh0000gn_T__XPC_FLAGS_0x0_—_180×58_and_datasette-app-support_—_pipenv_shell_▸_zsh_—_147×76" src="https://user-images.githubusercontent.com/9599/132412648-b768495b-ead2-4abe-875a-fd090fe3d71d.png">
  >
  > Maybe the real problem here is that I'm conflating "root" permissions with "debug" options. Perhaps there should be an extra Datasette mode that unlocks debug tools for the root user?

  _Originally posted by @simonw in https://github.com/simonw/datasette-app-support/issues/8#issuecomment-914638998_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1462/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5OTA4NDQwODg=
title: sqlite-utils memory can't deal with multiple files with the same name
state: closed
created_at: 2021-09-08T08:14:42Z
updated_at: 2021-09-22T20:52:56Z
closed_at: 2021-09-22T20:45:45Z
author_association: NONE
type: issue
state_reason: completed
body:
  When I use multiple files with the same name, e.g. in `sqlite-utils memory a/bug.csv b/bug.csv`, sqlite-utils creates invalid views.
  ```
  Traceback (most recent call last):
    File "/home/karl/.local/bin/sqlite-utils", line 8, in <module>
      sys.exit(cli())
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
      return self.main(*args, **kwargs)
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/click/core.py", line 1062, in main
      rv = self.invoke(ctx)
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/click/core.py", line 1668, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/click/core.py", line 763, in invoke
      return __callback(*args, **kwargs)
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/sqlite_utils/cli.py", line 1299, in memory
      db[csv_table].transform(types=tracker.types)
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/sqlite_utils/db.py", line 1287, in transform
      self.db.execute(sql)
    File "/home/karl/.local/pipx/venvs/sqlite-utils/lib/python3.9/site-packages/sqlite_utils/db.py", line 421, in execute
      return self.conn.execute(sql)
  sqlite3.OperationalError: error in view t1: no such table: main.bug
  ```

  This can be reproduced with
  ```sh
  #!/bin/bash
  mkdir foo
  mkdir bar
  echo -e 'col1,col2\nval1,val2' > foo/bug.csv
  echo -e 'col3,col4\nval3,val4' > bar/bug.csv
  sqlite-utils memory */bug.csv 'SELECT 1'
  ```

  Ideally, the tables would get unique names by including the next path segment until the names are unique. But just making the numbered t* aliases work would be good enough.

  This problem can of course be worked around by renaming the files, but it would be nice if this case was handled more gracefully.

  Thanks a lot for this great tool!

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/325/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI5NjMyNjQz
title: Update beautifulsoup4 requirement from <4.10.0,>=4.8.1 to >=4.8.1,<4.11.0
state: closed
created_at: 2021-09-08T13:09:38Z
updated_at: 2021-10-13T22:35:37Z
closed_at: 2021-10-13T22:35:36Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1463
type: pull
body:
  Updates the requirements on [beautifulsoup4](http://www.crummy.com/software/BeautifulSoup/bs4/) to permit the latest version.


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1463/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5OTExOTE5NTE=
title: clean checkout & clean environment has test failures
state: open
created_at: 2021-09-08T14:16:23Z
updated_at: 2021-09-13T22:17:17Z
author_association: CONTRIBUTOR
type: issue
body:
  I followed the instructions [here](https://docs.datasette.io/en/stable/contributing.html#setting-up-a-development-environment), and even after running `python update-docs-help.py` I get the following failed tests -- any thoughts?

  ```
  FAILED tests/test_api.py::test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]
  FAILED tests/test_api.py::test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1]
  FAILED tests/test_api.py::test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2]
  ```

  This is with python 3.9.7 and lots of other packages, as in attached environment listing from `conda list`.
  [conda-installed.txt](https://github.com/simonw/datasette/files/7129487/conda-installed.txt)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1464/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI5NzA0NTM3
title: add support for -o --get /path
state: open
created_at: 2021-09-08T14:30:42Z
updated_at: 2021-09-08T14:31:45Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1465
type: pull
body:
  Fixes https://github.com/simonw/datasette/issues/1459

  Adds support for `--open --get /path` to be used in combination.

  If `--open` is provided alone, datasette will open a web page to a default URL.
  If `--get <url>` is provided alone, datasette will output the result of doing a GET to that URL and then exit.
  If `--open --get <url>` are provided together, datasette will open a web page to that URL.

  TODO items:
  - [ ] update documentation
  - [ ] print out error message when `--root --open --get <url>` is used
  - [ ] adjust code to require that `<url>` start with a `/` when `-o --get <url>` is used
  - [ ] add test(s)

  note, '@CTB' is used in this PR to flag code that needs revisiting.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1465/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzI5NzMxNDQx
title: Test against 3.10-dev
state: closed
created_at: 2021-09-08T15:01:15Z
updated_at: 2021-10-13T21:49:28Z
closed_at: 2021-10-13T21:49:28Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/326
body: This tests against the latest 3.10 RC, https://www.python.org/downloads/release/python-3100rc2/
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/326/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5OTE0Njc1NTg=
title: Add Datasette Desktop to installation documentation
state: closed
created_at: 2021-09-08T19:41:27Z
updated_at: 2022-01-13T22:28:28Z
closed_at: 2022-01-13T21:55:18Z
author_association: OWNER
body: See https://datasette.io/desktop and https://simonwillison.net/2021/Sep/8/datasette-desktop/
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1466/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDExOlB1bGxSZXF1ZXN0NzMwMDIwODY3
title: Add Authorization header when CORS flag is set
state: closed
created_at: 2021-09-08T22:14:41Z
updated_at: 2021-10-17T02:29:07Z
closed_at: 2021-10-14T18:54:18Z
author_association: NONE
pull_request: simonw/datasette/pulls/1467
type: pull
body:
  This PR adds the [`Access-Control-Allow-Headers`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers) flag when CORS mode is enabled.

  This would fix https://github.com/simonw/datasette-auth-tokens/issues/4. When making cross-origin requests, the server must respond with all allowable HTTP headers. A Datasette instance using auth tokens must accept the `Authorization` HTTP header in order for cross-origin authenticated requests to take place.

  Please let me know if there's a better way of doing this! I couldn't figure out a way to change the app's response from the plugin itself, so I'm starting here. If you'd rather this logic live in the plugin, I'd love any guidance you're able to give.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1467/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5OTQzOTA1OTM=
title: Faceting for custom SQL queries
state: closed
created_at: 2021-09-13T02:52:16Z
updated_at: 2021-09-13T04:54:22Z
closed_at: 2021-09-13T04:54:17Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Facets are awesome.  But not when I need to join to tidy tables together.  Or even just running explicitly the default SQL query that simply lists all the rows and columns of a table (up to SIZE).  That is to say, when I browse a table, I see facets:

  https://latest.datasette.io/fixtures/compound_three_primary_keys

  But when I run a custom query, I don't:

  https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101

  Is there an idiom to cause custom SQL to come back with facet suggestions?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1468/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5OTQ0NTA5NjE=
title: Column cog shows "facet by this" when already default faceted
state: closed
created_at: 2021-09-13T04:51:26Z
updated_at: 2021-10-13T21:20:07Z
closed_at: 2021-10-13T21:20:07Z
author_association: OWNER
type: issue
state_reason: completed
body:
  e.g. on https://covid-19.datasettes.com/covid/economist_excess_deaths

  <img width="782" alt="covid__economist_excess_deaths__17_576_rows_where_sorted_by_end_date_descending_and_New_Issue_·_simonw_datasette" src="https://user-images.githubusercontent.com/9599/133025772-beaf055e-374f-4180-aad3-bb0215e8aaf3.png">

  But if you add `?_facet=country` to the URL that goes away: https://covid-19.datasettes.com/covid/economist_excess_deaths?_facet_size=5&_facet=country

  The logic that decides if the "Facet by this" item is shown does not take default `metadata.json` facets into account.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1469/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: MDU6SXNzdWU5OTUwOTgyMzE=
title: ?_sort=rowid with _next= returns error
state: closed
created_at: 2021-09-13T16:36:15Z
updated_at: 2021-10-18T19:30:15Z
closed_at: 2021-10-10T01:15:03Z
author_association: NONE
type: issue
state_reason: completed
body:
  For example:

  - Go to https://cryptics.eigenfoo.xyz/clues/clues?_next=100 (this is the second page of results in a Datasette site)
  - Search anything using the FTS search bar. For example, searching for `hello` will take you to https://cryptics.eigenfoo.xyz/clues/clues?_search=hello&_sort=rowid&_next=100
  - A `500 Error: list index out of range` is raised.

  This is because the search URL includes the `&_next=100` UTM parameter, carried over from where the FTS search was run. However, there isn't a second page in the search results, so a `list index out of range` error is raised. You can confirm that removing this UTM parameter from the URL returns the appropriate search results.

  The FTS search request should strip any `_next` UTM parameter.

  ---

  ```bash
  datasette, version 0.58.1
  sqlite-utils, version 3.17
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1470/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4rutdp
title: Bump black from 21.7b0 to 21.9b0
state: closed
created_at: 2021-09-14T13:10:35Z
updated_at: 2021-10-13T21:47:42Z
closed_at: 2021-10-13T21:47:42Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1471
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 21.7b0 to 21.9b0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>21.9b0</h2>
  <h3>Packaging</h3>
  <ul>
  <li>Fix missing modules in self-contained binaries (<a href="https://github-redirect.dependabot.com/psf/black/issues/2466">#2466</a>)</li>
  <li>Fix missing toml extra used during installation (<a href="https://github-redirect.dependabot.com/psf/black/issues/2475">#2475</a>)</li>
  </ul>
  <h2>21.8b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Add support for formatting Jupyter Notebook files (<a href="https://github-redirect.dependabot.com/psf/black/issues/2357">#2357</a>)</li>
  <li>Move from <code>appdirs</code> dependency to <code>platformdirs</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2375">#2375</a>)</li>
  <li>Present a more user-friendly error if .gitignore is invalid (<a href="https://github-redirect.dependabot.com/psf/black/issues/2414">#2414</a>)</li>
  <li>The failsafe for accidentally added backslashes in f-string expressions has been hardened to handle more edge cases during quote normalization (<a href="https://github-redirect.dependabot.com/psf/black/issues/2437">#2437</a>)</li>
  <li>Avoid changing a function return type annotation's type to a tuple by adding a trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2384">#2384</a>)</li>
  <li>Parsing support has been added for unparenthesized walruses in set literals, set comprehensions, and indices (<a href="https://github-redirect.dependabot.com/psf/black/issues/2447">#2447</a>).</li>
  <li>Pin <code>setuptools-scm</code> build-time dependency version (<a href="https://github-redirect.dependabot.com/psf/black/issues/2457">#2457</a>)</li>
  <li>Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2460">#2460</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that scan installed packages (<a href="https://github-redirect.dependabot.com/psf/black/issues/2440">#2440</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>The provided pre-commit hooks no longer specify <code>language_version</code> to avoid overriding <code>default_language_version</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2430">#2430</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>21.9b0</h2>
  <h3>Packaging</h3>
  <ul>
  <li>Fix missing modules in self-contained binaries (<a href="https://github-redirect.dependabot.com/psf/black/issues/2466">#2466</a>)</li>
  <li>Fix missing toml extra used during installation (<a href="https://github-redirect.dependabot.com/psf/black/issues/2475">#2475</a>)</li>
  </ul>
  <h2>21.8b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Add support for formatting Jupyter Notebook files (<a href="https://github-redirect.dependabot.com/psf/black/issues/2357">#2357</a>)</li>
  <li>Move from <code>appdirs</code> dependency to <code>platformdirs</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2375">#2375</a>)</li>
  <li>Present a more user-friendly error if .gitignore is invalid (<a href="https://github-redirect.dependabot.com/psf/black/issues/2414">#2414</a>)</li>
  <li>The failsafe for accidentally added backslashes in f-string expressions has been
  hardened to handle more edge cases during quote normalization (<a href="https://github-redirect.dependabot.com/psf/black/issues/2437">#2437</a>)</li>
  <li>Avoid changing a function return type annotation's type to a tuple by adding a
  trailing comma (<a href="https://github-redirect.dependabot.com/psf/black/issues/2384">#2384</a>)</li>
  <li>Parsing support has been added for unparenthesized walruses in set literals, set
  comprehensions, and indices (<a href="https://github-redirect.dependabot.com/psf/black/issues/2447">#2447</a>).</li>
  <li>Pin <code>setuptools-scm</code> build-time dependency version (<a href="https://github-redirect.dependabot.com/psf/black/issues/2457">#2457</a>)</li>
  <li>Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10
  (<a href="https://github-redirect.dependabot.com/psf/black/issues/2460">#2460</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that
  scan installed packages (<a href="https://github-redirect.dependabot.com/psf/black/issues/2440">#2440</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>The provided pre-commit hooks no longer specify <code>language_version</code> to avoid overriding
  <code>default_language_version</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2430">#2430</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/psf/black/commits">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.7b0&new-version=21.9b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1471/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c47mU4i
title: base logo link visits `undefined` rather than href url
state: open
created_at: 2021-09-18T04:17:04Z
updated_at: 2021-09-19T00:45:32Z
author_association: CONTRIBUTOR
type: issue
body:
  I have two connected sites:
  http://www.SaferOrToxic.org 
  (a Hugo website)
  and:
  http://disinfectants.SaferOrToxic.org/disinfectants/listN
  (a datasette table page)

  The latter is linked as "The List" in the former's menu.
  (I'd love a prettier URL, but that's what I've got.)

  On:
  http://disinfectants.SaferOrToxic.org/disinfectants/listN
  ... all the other menu links should point back to:
  https://www.SaferOrToxic.org 
  And they do!

  But the logo, for some reason--though it has an href pointing to:
  https://www.SaferOrToxic.org
  Keeps going to this instead:
  https://disinfectants.saferortoxic.org/disinfectants/undefined

  What is causing that? How can I fix it?

  In #1284 back in March, I was doing battle with the index.html template, in a still unresolved issue. (I wanted only a single table page at the root.)

  But I thought, well, if I can't resolve that, at least I could just point the main website to the datasette page ("The List,") and then have the List point back to the home website.

  The menu hrefs to https://www.SaferOrToxic.org work just fine, exactly as they should, from the datasette page. Even the Home link works properly.

  But the logo link keeps rewriting to: https://disinfectants.saferortoxic.org/disinfectants/undefined

  This is the HTML:
  ```
  <a class="text-3xl font-bold leading-none" href="https://www.saferortoxic.org"><img src="https://www.saferortoxic.org/images/logo_hu26e4dce8d5931af1ea33526b28fc8383_9734_c52a4f1635ef88bda858373270551ed2.webp" class="custom-logo" alt="Logo: Safer or Toxic?" width="300px"></a>
  ```


  Is this somehow related to cloudflare?
  Or something in the datasette code?

  I'm starting to think it's a cloudflare issue.
  <img width="1046" alt="Screen Shot 2021-09-18 at 12 09 57 AM" src="https://user-images.githubusercontent.com/192568/133872043-5a008e14-88cb-43fd-b921-e63ff50636fb.png">

  Can I at least rule out it being a datasette issue?

  My repository is here:
  https://github.com/mroswell/list-N

  (BTW, I couldn't figure out how to reference a local image, either, on the datasette side, which is why I'm using the image from the www home page.)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1473/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4r7n-9
title: Extract expand: Support JSON Arrays
state: closed
created_at: 2021-09-19T10:34:30Z
updated_at: 2022-12-29T09:05:36Z
closed_at: 2022-12-29T09:05:36Z
author_association: NONE
pull_request: simonw/sqlite-utils/pulls/327
type: pull
body:
  Hi,

  I needed to extract data in JSON Arrays to normalize data imports. I've quickly hacked the following together based on #241 which refers to #239 where you, @simonw, wrote:

  > Could this handle lists of objects too? That would be pretty amazing - if the column has a [{...}, {...}] list in it could turn that into a many-to-many.

  They way this works in my work is that many-to-many relationships are created for anything that maps to an dictionary in a list, and many-to-one relations for everything else (assumed to be scalar values). Not sure what the best approach here would be? Are many-to-one relationships are at all useful here?

  What do you think about this approach? I could try to add it to the cli interface and documentation if wanted.

  Thanks for this awesome piece of software in any case! :sun_with_face: 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/327/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4r9CTw
title: Update full_text_search.rst
state: closed
created_at: 2021-09-20T09:59:45Z
updated_at: 2021-10-13T21:10:23Z
closed_at: 2021-10-13T21:10:23Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1474
body: Change "above" to "below" to correct correspondence of reference to example.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1474/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4r-EVH
title: feat: allow joins using _through in both directions
state: open
created_at: 2021-09-20T15:28:20Z
updated_at: 2021-09-20T15:28:20Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: simonw/datasette/pulls/1475
type: pull
body:
  Currently the `_through` clause can only work if the FK relationship is defined in a specific direction. I don't think there is any reason for this limitation, as an FK allows joining in both directions.

  This is an admittedly hacky change to implement bidirectional joins using `_through`. It does work for our use-case, but I don't know if there are other implications that I haven't thought of. Also if this change is desirable we probably want to make the code a little nicer.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1475/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4sCfBT
title: Update pytest-xdist requirement from <2.4,>=2.2.1 to >=2.2.1,<2.5
state: closed
created_at: 2021-09-21T13:13:01Z
updated_at: 2021-10-13T21:10:03Z
closed_at: 2021-10-13T21:10:03Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1476
type: pull
body:
  Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p>
  <blockquote>
  <h1>pytest-xdist 2.4.0 (2021-09-20)</h1>
  <h2>Features</h2>
  <ul>
  <li>
  <p><code>[#696](https://github.com/pytest-dev/pytest-xdist/issues/696) &lt;https://github.com/pytest-dev/pytest-xdist/issues/696&gt;</code>_: On Linux, the process title now changes to indicate the current worker state (running/idle).</p>
  <p>Depends on the <code>setproctitle &lt;https://pypi.org/project/setproctitle/&gt;</code>__ package, which can be installed with <code>pip install pytest-xdist[setproctitle]</code>.</p>
  </li>
  <li>
  <p><code>[#704](https://github.com/pytest-dev/pytest-xdist/issues/704) &lt;https://github.com/pytest-dev/pytest-xdist/issues/704&gt;</code>_: Add support for Python 3.10.</p>
  </li>
  </ul>
  <h1>pytest-xdist 2.3.0 (2021-06-16)</h1>
  <h2>Deprecations and Removals</h2>
  <ul>
  <li><code>[#654](https://github.com/pytest-dev/pytest-xdist/issues/654) &lt;https://github.com/pytest-dev/pytest-xdist/issues/654&gt;</code>_: Python 3.5 is no longer supported.</li>
  </ul>
  <h2>Features</h2>
  <ul>
  <li>
  <p><code>[#646](https://github.com/pytest-dev/pytest-xdist/issues/646) &lt;https://github.com/pytest-dev/pytest-xdist/issues/646&gt;</code>_: Add <code>--numprocesses=logical</code> flag, which automatically uses the number of logical CPUs available, instead of physical CPUs with <code>auto</code>.</p>
  <p>This is very useful for test suites which are not CPU-bound.</p>
  </li>
  <li>
  <p><code>[#650](https://github.com/pytest-dev/pytest-xdist/issues/650) &lt;https://github.com/pytest-dev/pytest-xdist/issues/650&gt;</code>_: Added new <code>pytest_handlecrashitem</code> hook to allow handling and rescheduling crashed items.</p>
  </li>
  </ul>
  <h2>Bug Fixes</h2>
  <ul>
  <li>
  <p><code>[#421](https://github.com/pytest-dev/pytest-xdist/issues/421) &lt;https://github.com/pytest-dev/pytest-xdist/issues/421&gt;</code>_: Copy the parent process sys.path into local workers, to work around execnet's python -c adding the current directory to sys.path.</p>
  </li>
  <li>
  <p><code>[#638](https://github.com/pytest-dev/pytest-xdist/issues/638) &lt;https://github.com/pytest-dev/pytest-xdist/issues/638&gt;</code>_: Fix issue caused by changing the branch name of the pytest repository.</p>
  </li>
  </ul>
  <h2>Trivial Changes</h2>
  <ul>
  <li>
  <p><code>[#592](https://github.com/pytest-dev/pytest-xdist/issues/592) &lt;https://github.com/pytest-dev/pytest-xdist/issues/592&gt;</code>_: Replace master with controller where ever possible.</p>
  </li>
  <li>
  <p><code>[#643](https://github.com/pytest-dev/pytest-xdist/issues/643) &lt;https://github.com/pytest-dev/pytest-xdist/issues/643&gt;</code>_: Use 'main' to refer to pytest default branch in tox env names.</p>
  </li>
  </ul>
  <h1>pytest-xdist 2.2.1 (2021-02-09)</h1>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/4b487ed5e1932992bb2fd92cb0d04df5fac4c843"><code>4b487ed</code></a> Manually fix changelog title</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/ecf4d3be31a71365026f0bbb8d67c157f75d695d"><code>ecf4d3b</code></a> Release 2.4.0</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/87d89795e36748e4d0112423570264fcf3c21763"><code>87d8979</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/704">#704</a> from hugovk/add-3.10</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/b4544c8849eb25135113031f6eea277aa7216939"><code>b4544c8</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/706">#706</a> from pytest-dev/pre-commit-ci-update-config</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/66dc3904b4a0bd1ac491af9eb4e59750f92668f5"><code>66dc390</code></a> [pre-commit.ci] pre-commit autoupdate</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/e0ce1b74186043da5e347d63a47a8d34ee4cd36c"><code>e0ce1b7</code></a> Add news file to add support for Python 3.10</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/ed47f0e01a38af95100f806d0cfc98f96723b721"><code>ed47f0e</code></a> Add support for Python 3.10</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/1c8178abc5b0acdb95938d4ea99344fe78383618"><code>1c8178a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/703">#703</a> from pytest-dev/pre-commit-ci-update-config</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/9807064f66d28bbb5aee1bdae0a3f47919c39206"><code>9807064</code></a> [pre-commit.ci] pre-commit autoupdate</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/766e67ce526c88327670e9bcd4804c335df51015"><code>766e67c</code></a> Use setproctitle if available to show state (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/696">#696</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-xdist/compare/v2.2.1...v2.4.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1476/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM474S6T
title: Invalid JSON output when no rows
state: closed
created_at: 2021-09-22T18:37:26Z
updated_at: 2021-09-22T20:21:34Z
closed_at: 2021-09-22T20:20:18Z
author_association: NONE
type: issue
state_reason: completed
body:
  `sqlite-utils query` generates a JSON output with the result from the query:

  ```json
  [{...},{...}]
  ```
  If no rows are returned by the query, I'm expecting an empty JSON array:

  ```json
  []
  ```

  But actually I'm getting an empty string. To be consistent, the output should be `[]` when the request succeeds (return code == `0`).

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/328/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM479K3U
title: Rethink approach to [ and ] in column names (currently throws error)
state: closed
created_at: 2021-09-23T22:14:24Z
updated_at: 2021-11-15T02:57:51Z
closed_at: 2021-11-15T02:57:51Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I think it's best to still keep `[` and `]` out of column names though. Transforming them into `(` and `)` seems reasonable - but should that happen here or in `sqlite-utils`? I think in `sqlite-utils`.

  _Originally posted by @simonw in https://github.com/simonw/datasette-app/issues/121#issuecomment-926200398_

  This is a rethinking of the solution to:

  - https://github.com/simonw/sqlite-utils/issues/86

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/329/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c479pcu
title: Consider adding request to the documented default template context
state: open
created_at: 2021-09-24T02:34:09Z
updated_at: 2021-09-24T02:34:09Z
author_association: OWNER
type: issue
body:
  I made a plugin for this today but I think perhaps it should be a default thing instead: https://datasette.io/plugins/datasette-template-request

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1477/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c48AkX9
title: Documentation Request: Feature alternative ID instead of default ID
state: open
created_at: 2021-09-24T19:56:13Z
updated_at: 2021-09-25T16:18:54Z
author_association: CONTRIBUTOR
type: issue
body:
  My data already has an ID that comes from a federal agency.
  Would love to have documentation on how to modify the template to:
  - Remove the generated ID from the table
  - Link the federal ID to the detail page
  - and to ensure that the JSON file uses that as the ID. I'd be happy to include the database ID in the export, but not as a key.

  I don't want to remove the ID from the database, though, because my experience with the federal agency is that data often has anomalies. I don't want all hell to break loose if they end up applying the same ID to multiple rows (which they haven't done yet). I just don't want it to display in the table or the data exports.

  Perhaps this isn't a template issue, maybe more of a db manipulation...

  Margie

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1478/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c48NRky
title: Win32 "used by another process" error with datasette publish
state: open
created_at: 2021-09-28T19:12:00Z
updated_at: 2023-09-07T02:14:16Z
author_association: NONE
type: issue
body:
  I unfortunately was not successful to deploy to fly.io. Please see the details above of the three scenarios that I took. I am also new to datasette.

  Failed to deploy. Attaching logs:
  1. Tried with an app created via `flyctl apps create frosty-fog-8565` and the ran `datasette publish fly covid.db --app frosty-fog-8565` 
  ``` 
  Deploying frosty-fog-8565
  ==> Validating app configuration
  --> Validating app configuration done
  Services
  TCP 80/443 ⇢ 8080

  Error error connecting to docker: An unknown error occured.

  Traceback (most recent call last):
    File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
      "__main__", mod_spec)
    File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code
      exec(code, run_globals)
    File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
      return self.main(*args, **kwargs)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
      rv = self.invoke(ctx)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
      return callback(*args, **kwargs)
    File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py", line 156, in fly
      "--remote-only",
    File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
      next(self.gen)
    File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
      tmp.cleanup()
    File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
      _shutil.rmtree(self.name)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
      return _rmtree_unsafe(path, onerror)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
      _rmtree_unsafe(fullname, onerror)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
      onerror(os.rmdir, path, sys.exc_info())
    File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
      os.rmdir(path)
  PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpgcm8cz66\\frosty-fog-8565'
  ```

  2. Tried also with an app that gets autogenerate when running `flyctl launch`. This also generates the .toml file. Ran then `datasette publish fly covid.db --app dark-feather-168` **but different error now**
  ```Deploying dark-feather-168
  ==> Validating app configuration

  Error not possible to validate configuration: server returned Post "https://api.fly.io/graphql": unexpected EOF

  Traceback (most recent call last):
    File "c:\users\grott\anaconda3\lib\runpy.py", line 193, in _run_module_as_main        
      "__main__", mod_spec)
      exec(code, run_globals)
    File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
      return self.main(*args, **kwargs)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
      rv = self.invoke(ctx)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
      return callback(*args, **kwargs)
    File "c:\users\grott\anaconda3\lib\site-packages\datasette_publish_fly\__init__.py", line 156, in fly
      "--remote-only",
    File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
      next(self.gen)
    File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
      tmp.cleanup()
    File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
      _shutil.rmtree(self.name)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
      return _rmtree_unsafe(path, onerror)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
      _rmtree_unsafe(fullname, onerror)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
      onerror(os.rmdir, path, sys.exc_info())
    File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
      os.rmdir(path)
  PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpnoyewcre\\dark-feather-168'
  ```

  These are also the contents of the generated **.toml file** in 2 scenario:

  ```
  # fly.toml file generated for dark-feather-168 on 2021-09-28T20:35:44+02:00

  app = "dark-feather-168"

  kill_signal = "SIGINT"
  kill_timeout = 5
  processes = []

  [env]

  [experimental]
    allowed_public_ports = []
    auto_rollback = true

  [[services]]
    http_checks = []
    internal_port = 8080
    processes = ["app"]
    protocol = "tcp"
    script_checks = []

    [services.concurrency]
      hard_limit = 25
      soft_limit = 20
      type = "connections"

    [[services.ports]]
      handlers = ["http"]
      port = 80

    [[services.ports]]
      handlers = ["tls", "http"]
      port = 443

    [[services.tcp_checks]]
      grace_period = "1s"
      interval = "15s"
      restart_limit = 6
      timeout = "2s"
  ```

  3. But also trying `datasette package covid.db` to create a local DOCKERFILE to later try to push it via `flyctl deploy` fails as well.

  ```[+] Building 147.3s (11/11) FINISHED
   => [internal] load build definition from Dockerfile                                                             0.2s 
   => => transferring dockerfile: 396B                                                                             0.0s 
   => [internal] load .dockerignore                                                                                0.1s 
   => => transferring context: 2B                                                                                  0.0s 
   => [internal] load metadata for docker.io/library/python:3.8                                                    4.7s 
   => [auth] library/python:pull token for registry-1.docker.io                                                    0.0s 
   => [internal] load build context                                                                                0.1s 
   => => transferring context: 82.37kB                                                                             0.0s 
   => [1/5] FROM docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3  108.3s 
   => => resolve docker.io/library/python:3.8@sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5  0.0s 
   => => sha256:56182bcdf4d4283aa1f46944b4ef7ac881e28b4d5526720a4e9ba03a4730846a 2.22kB / 2.22kB                   0.0s 
   => => sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366 54.93MB / 54.93MB                21.6s 
   => => sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64 10.87MB / 10.87MB                15.5s 
   => => sha256:530de807b46a11734e2587a784573c12c5034f2f14025f838589e6c0e3b5c5b6 1.86kB / 1.86kB                   0.0s 
   => => sha256:ff08f08727e50193dcf499afc30594c47e70cc96f6fcfd1a01240524624264d0 8.65kB / 8.65kB                   0.0s 
   => => sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8 5.15MB / 5.15MB                   6.4s 
   => => sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587 54.57MB / 54.57MB                37.7s 
   => => sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077 196.45MB / 196.45MB              82.3s 
   => => sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58 6.29MB / 6.29MB                  26.6s 
   => => extracting sha256:955615a668ce169f8a1443fc6b6e6215f43fe0babfb4790712a2d3171f34d366                        5.4s 
   => => sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26 16.81MB / 16.81MB                40.5s 
   => => extracting sha256:2756ef5f69a5190f4308619e0f446d95f5515eef4a814dbad0bcebbbbc7b25a8                        0.6s 
   => => extracting sha256:911ea9f2bd51e53a455297e0631e18a72a86d7e2c8e1807176e80f991bde5d64                        0.6s 
   => => sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b 232B / 232B                      40.1s 
   => => extracting sha256:27b0a22ee906271a6ce9ddd1754fdd7d3b59078e0b57b6cc054c7ed7ac301587                        5.8s 
   => => sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543 2.35MB / 2.35MB                  41.8s 
   => => extracting sha256:8584d51a9262f9a3a436dea09ba40fa50f85802018f9bd299eee1bf538481077                       18.8s 
   => => extracting sha256:524774b7d3638702fe9ae0ea3fcfb81b027dfd75cc2fc14f0119e764b9543d58                        1.2s 
   => => extracting sha256:9460f6b75036e38367e2f27bb15e85777c5d6cd52ad168741c9566186415aa26                        2.9s 
   => => extracting sha256:9bc548096c181514aa1253966a330134d939496027f92f57ab376cd236eb280b                        0.0s 
   => => extracting sha256:1d87379b86b89fd3b8bb1621128f00c8f962756e6aaaed264ec38db733273543                        0.8s 
   => [2/5] COPY . /app                                                                                            2.3s 
   => [3/5] WORKDIR /app                                                                                           0.2s 
   => [4/5] RUN pip install -U datasette                                                                          26.9s 
   => [5/5] RUN datasette inspect covid.db --inspect-file inspect-data.json                                        3.1s
   => exporting to image                                                                                           1.2s 
   => => exporting layers                                                                                          1.2s 
   => => writing image sha256:b5db0c205cd3454c21fbb00ecf6043f261540bcf91c2dfc36d418f1a23a75d7a                     0.0s

  Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
  Traceback (most recent call last):
      "__main__", mod_spec)
    File "c:\users\grott\anaconda3\lib\runpy.py", line 85, in _run_code
      exec(code, run_globals)
    File "C:\Users\grott\Anaconda3\Scripts\datasette.exe\__main__.py", line 7, in <module>
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 829, in __call__
      return self.main(*args, **kwargs)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 782, in main
      rv = self.invoke(ctx)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "c:\users\grott\anaconda3\lib\site-packages\click\core.py", line 610, in invoke
      return callback(*args, **kwargs)
    File "c:\users\grott\anaconda3\lib\site-packages\datasette\cli.py", line 283, in package
      call(args)
    File "c:\users\grott\anaconda3\lib\contextlib.py", line 119, in __exit__
      next(self.gen)
    File "c:\users\grott\anaconda3\lib\site-packages\datasette\utils\__init__.py", line 451, in temporary_docker_directory
      tmp.cleanup()
    File "c:\users\grott\anaconda3\lib\tempfile.py", line 811, in cleanup
      _shutil.rmtree(self.name)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 516, in rmtree
      return _rmtree_unsafe(path, onerror)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 395, in _rmtree_unsafe
      _rmtree_unsafe(fullname, onerror)
    File "c:\users\grott\anaconda3\lib\shutil.py", line 404, in _rmtree_unsafe
      onerror(os.rmdir, path, sys.exc_info())
    File "c:\users\grott\anaconda3\lib\shutil.py", line 402, in _rmtree_unsafe
      os.rmdir(path)
  PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\grott\\AppData\\Local\\Temp\\tmpkb27qid3\\datasette'```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1479/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODFdgUs4skaNS
title: Add support for retrieving teams / members
state: open
created_at: 2021-10-01T15:55:02Z
updated_at: 2021-10-01T15:59:53Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/github-to-sqlite/pulls/68
type: pull
body:
  Adds a method for retrieving all the teams within an organisation and all the members in those teams. The latter is stored as a join table `team_members` beteween `teams` and `users`.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/68/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c48iYih
title: Exceeding Cloud Run memory limits when deploying a 4.8G database
state: open
created_at: 2021-10-04T21:20:24Z
updated_at: 2022-10-07T04:39:10Z
author_association: CONTRIBUTOR
type: issue
body:
  When I try to deploy a 4.8G SQLite database to Google Cloud Run, I get this error message:

  > Memory limit of 8192M exceeded with 8826M used. Consider increasing the memory limit, see https://cloud.google.com/run/docs/configuring/memory-limits

  Unfortunately, the maximum amount of memory that can be allocated to an instance is 8192M.

  Naively profiling the memory usage of running Datasette with this database locally on my MacBook shows the following memory usage (using Activity Monitor) when I just start up Datasette locally:

  - Real Memory Size: 70.6 MB
  - Virtual Memory Size: 4.51 GB
  - Shared Memory Size: 2.5 MB
  - Private Memory Size: 57.4 MB

  I'm trying to understand if there's a query or other operation that gets run during container deployment that causes memory use to be so large and if this can be avoided somehow.

  This is somewhat related to #1082, but on a different platform, so I decided to open a new issue.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1480/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4s6bJm
title: Fix compatibility with Python 3.10
state: closed
created_at: 2021-10-07T20:34:23Z
updated_at: 2021-10-24T22:19:55Z
closed_at: 2021-10-24T22:19:54Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1481
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1481/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4845_O
title: Support Python 3.10
state: closed
created_at: 2021-10-09T00:30:52Z
updated_at: 2021-10-24T22:21:40Z
closed_at: 2021-10-24T22:19:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I started work on this in #1481 where I found a Python 3.10 bug that needs a workaround in Janus, see:

  - https://github.com/aio-libs/janus/issues/358

  This is a tracking issue for anything else that shows up.

  This is also needed for the Homebrew package to upgrade to 3.10:

  - https://github.com/Homebrew/homebrew-core/pull/86932

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1482/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c486DCm
title: Running a search on page 2 of results should not preserve ?_next=
state: closed
created_at: 2021-10-10T01:18:12Z
updated_at: 2021-10-13T21:08:10Z
closed_at: 2021-10-13T21:08:10Z
author_association: OWNER
body: Reported by @eigenfoo in https://github.com/simonw/datasette/issues/1470
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1483/reactions", "total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4s_4Cw
title: GitHub Actions: Add Python 3.10 to the tests
state: closed
created_at: 2021-10-11T06:03:03Z
updated_at: 2021-10-11T06:03:31Z
closed_at: 2021-10-11T06:03:28Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: simonw/datasette/pulls/1484
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1484/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4tBIEE
title: Update pytest-timeout requirement from <1.5,>=1.4.2 to >=1.4.2,<2.1
state: closed
created_at: 2021-10-11T13:10:51Z
updated_at: 2021-10-13T21:09:23Z
closed_at: 2021-10-13T21:09:23Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1485
type: pull
body:
  Updates the requirements on [pytest-timeout](https://github.com/pytest-dev/pytest-timeout) to permit the latest version.
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/pytest-dev/pytest-timeout/commits">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1485/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c48_XNh
title: pipx installation instructions for plugins don't reference pipx inject
state: closed
created_at: 2021-10-12T00:43:42Z
updated_at: 2021-10-13T21:09:11Z
closed_at: 2021-10-13T21:09:11Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  The datasette [installation instructions](https://github.com/simonw/datasette/blob/main/docs/installation.rst) discuss how to install with pipx, how to upgrade with pipx, and how to upgrade plugins with pipx but do not mention how to install a plugin with pipx.  You discussed this on your [blog](https://til.simonwillison.net/python/installing-upgrading-plugins-with-pipx) but looks like this didn't make it in when you updated the docs for pipx (#756). 

  I'll submit a PR shortly to fix this.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1486/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4tC4Lx
title: Added instructions for installing plugins via pipx, #1486
state: closed
created_at: 2021-10-12T00:48:30Z
updated_at: 2021-10-13T21:09:11Z
closed_at: 2021-10-13T21:09:10Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1487
body: Adds missing instructions for installing plugins via pipx
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1487/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4tKxHD
title: Test against Python 3.10
state: closed
created_at: 2021-10-13T21:50:22Z
updated_at: 2021-11-15T02:59:29Z
closed_at: 2021-10-13T22:25:05Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/330
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/330/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c49I8QN
title: Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects')
state: closed
created_at: 2021-10-13T22:37:22Z
updated_at: 2021-10-14T18:03:45Z
closed_at: 2021-10-14T18:03:45Z
author_association: OWNER
body: This is caused by a change made to `httpx` in https://github.com/encode/httpx/releases/tag/0.20.0
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1488/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4tM0JV
title: Update pyyaml requirement from ~=5.3 to >=5.3,<7.0
state: closed
created_at: 2021-10-14T13:09:33Z
updated_at: 2021-10-14T18:10:43Z
closed_at: 2021-10-14T18:10:42Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1489
type: pull
body:
  Updates the requirements on [pyyaml](https://github.com/yaml/pyyaml) to permit the latest version.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/yaml/pyyaml/blob/master/CHANGES">pyyaml's changelog</a>.</em></p>
  <blockquote>
  <p>6.0 (2021-10-13)</p>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/327">yaml/pyyaml#327</a> -- Change README format to Markdown</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/483">yaml/pyyaml#483</a> -- Add a test for YAML 1.1 types</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/497">yaml/pyyaml#497</a> -- fix float resolver to ignore <code>.</code> and <code>._</code></li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/550">yaml/pyyaml#550</a> -- drop Python 2.7</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/553">yaml/pyyaml#553</a> -- Fix spelling of “hexadecimal”</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/556">yaml/pyyaml#556</a> -- fix representation of Enum subclasses</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/557">yaml/pyyaml#557</a> -- fix libyaml extension compiler warnings</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/560">yaml/pyyaml#560</a> -- fix ResourceWarning on leaked file descriptors</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/561">yaml/pyyaml#561</a> -- always require <code>Loader</code> arg to <code>yaml.load()</code></li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/564">yaml/pyyaml#564</a> -- remove remaining direct distutils usage</li>
  </ul>
  <p>5.4.1 (2021-01-20)</p>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/480">yaml/pyyaml#480</a> -- Fix stub compat with older pyyaml versions that may unwittingly load it</li>
  </ul>
  <p>5.4 (2021-01-19)</p>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/407">yaml/pyyaml#407</a> -- Build modernization, remove distutils, fix metadata, build wheels, CI to GHA</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/472">yaml/pyyaml#472</a> -- Fix for CVE-2020-14343, moves arbitrary python tags to UnsafeLoader</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/441">yaml/pyyaml#441</a> -- Fix memory leak in implicit resolver setup</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/392">yaml/pyyaml#392</a> -- Fix py2 copy support for timezone objects</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/378">yaml/pyyaml#378</a> -- Fix compatibility with Jython</li>
  </ul>
  <p>5.3.1 (2020-03-18)</p>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/386">yaml/pyyaml#386</a> -- Prevents arbitrary code execution during python/object/new constructor</li>
  </ul>
  <p>5.3 (2020-01-06)</p>
  <ul>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/290">yaml/pyyaml#290</a> -- Use <code>is</code> instead of equality for comparing with <code>None</code></li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/270">yaml/pyyaml#270</a> -- Fix typos and stylistic nit</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/309">yaml/pyyaml#309</a> -- Fix up small typo</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/161">yaml/pyyaml#161</a> -- Fix handling of <strong>slots</strong></li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/358">yaml/pyyaml#358</a> -- Allow calling add_multi_constructor with None</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/285">yaml/pyyaml#285</a> -- Add use of safe_load() function in README</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/351">yaml/pyyaml#351</a> -- Fix reader for Unicode code points over 0xFFFF</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/360">yaml/pyyaml#360</a> -- Enable certain unicode tests when maxunicode not &gt; 0xffff</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/359">yaml/pyyaml#359</a> -- Use full_load in yaml-highlight example</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/244">yaml/pyyaml#244</a> -- Document that PyYAML is implemented with Cython</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/329">yaml/pyyaml#329</a> -- Fix for Python 3.10</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/310">yaml/pyyaml#310</a> -- Increase size of index, line, and column fields</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/260">yaml/pyyaml#260</a> -- Remove some unused imports</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/163">yaml/pyyaml#163</a> -- Create timezone-aware datetimes when parsed as such</li>
  <li><a href="https://github-redirect.dependabot.com/yaml/pyyaml/pull/363">yaml/pyyaml#363</a> -- Add tests for timezone</li>
  </ul>
  <h2>5.2 (2019-12-02)</h2>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/yaml/pyyaml/commit/8cdff2c80573b8be8e8ad28929264a913a63aa33"><code>8cdff2c</code></a> 6.0 release</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/a4fb55e943abf25055725245b0779cf527069a78"><code>a4fb55e</code></a> Update Python 3.10 versions for Windows build</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/e45b9643de3f7332e874ac67357ffd6dd5200211"><code>e45b964</code></a> Add Python 3.10 to tox.ini</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/4808fba18e067a93a4fcc25ddda1aae9b976ceb8"><code>4808fba</code></a> 6.0b1 release</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/d5aba406edc9d2618fc9d6ffccd257fe593e4283"><code>d5aba40</code></a> Omnibus CI/artifact build update</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/a6d384c52ea54ded80df27032c7b6d3a98b0c2fa"><code>a6d384c</code></a> Various setup fixes</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/8f3f9795b4ccc7087c439aad8342eaf916f273d0"><code>8f3f979</code></a> No longer using appveyor</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/c2743653bc89e42679ba097b4f9888db47c61d63"><code>c274365</code></a> The yaml.load{,_all} functions require Loader= now</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/2f87ac4838840a2e3cb48504dcd74041921ab184"><code>2f87ac4</code></a> Add a basic test file for yaml.load and yaml.dump</li>
  <li><a href="https://github.com/yaml/pyyaml/commit/7bd92df5435030b9a32f8843369ad6673d0d9294"><code>7bd92df</code></a> Makefile tweaks</li>
  <li>Additional commits viewable in <a href="https://github.com/yaml/pyyaml/compare/5.3...6.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1489/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4tNtoe
title: Upgrade to httpx 0.20
state: closed
created_at: 2021-10-14T17:51:05Z
updated_at: 2021-10-14T18:03:45Z
closed_at: 2021-10-14T18:03:44Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1490
body: Refs #1488
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1490/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM49M6JI
title: Mypy error: found module but no type hints or library stubs
state: closed
created_at: 2021-10-14T20:29:50Z
updated_at: 2021-11-14T23:21:08Z
closed_at: 2021-11-14T23:21:08Z
author_association: NONE
type: issue
state_reason: completed
body:
  ```
  Python 3.9.5
  mypy 0.910
  sqlite-utils 3.17.1
  ```

  While using sqlite-utils as a library, when I use mypy for static type checking, it throws an error:

  ```
  mypy .
  src/etl.py:5: error: Skipping analyzing "sqlite_utils": found module but no type hints or library stubs
      import sqlite_utils
      ^
  src/etl.py:5: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
  test/test_etl.py:4: error: Skipping analyzing "sqlite_utils": found module but no type hints or library stubs
      import sqlite_utils
      ^
  Found 2 errors in 2 files (checked 7 source files)
  ```


  When I add a `py.typed` file to the sqlite-utils package to mark it as PEP 561 compatible, the error goes away.

  ```
  al@nbal ..b/python3.9/site-packages/sqlite_utils (git)-[main] % la
  total 200
  drwx------   3 al al   4096 Oct 14 22:00 .
  drwx------ 117 al al   4096 Oct 12 21:12 ..
  -rw-------   1 al al  64409 Oct 12 21:11 cli.py
  -rw-------   1 al al 109092 Oct 12 21:11 db.py
  -rw-------   1 al al      0 Oct 14 22:00 py.typed
  -rw-------   1 al al    684 Oct 12 21:11 recipes.py
  -rw-------   1 al al   7988 Oct 12 21:11 utils.py
  -rw-------   1 al al    113 Oct 12 21:11 __init__.py
  ```

  I would like to suggest adding a `py.typed` file to the repository.

  See also the mypy docs on creating PEP 561 compatible packages:
  https://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/331/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM49RuaJ
title: `sqlite-utils memory --flatten` option to flatten nested JSON
state: closed
created_at: 2021-10-16T14:04:42Z
updated_at: 2021-11-14T23:05:05Z
closed_at: 2021-11-14T23:05:05Z
author_association: NONE
type: issue
state_reason: completed
body:
  currently --flatten option works only for `insert` command, it would be cool if it worked for `memory` as well to query nested json

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/332/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c49R8za
title: `--get '/:memory:.json?sql=select+3*5'` error with datasette 0.59
state: closed
created_at: 2021-10-16T18:22:22Z
updated_at: 2021-10-19T04:39:11Z
closed_at: 2021-10-19T04:39:11Z
author_association: NONE
type: issue
state_reason: completed
body:
  👋  trying to upgrade the formula to use the latest release, but runs into some regression test issue with `--get` command.

  My QQ is does this `datasette --get '/:memory:.json?sql=select+3*5'` supposed to return 15? Thanks!

  relates to https://github.com/Homebrew/homebrew-core/pull/87369

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1493/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4tU5cz
title: Update pytest-asyncio requirement from <0.16,>=0.10 to >=0.10,<0.17
state: closed
created_at: 2021-10-18T13:14:17Z
updated_at: 2021-10-24T22:22:40Z
closed_at: 2021-10-24T22:22:39Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1494
type: pull
body:
  Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/f2fe98e5cbc4a25e31c24f5932d66f9c903f0fe5"><code>f2fe98e</code></a> 0.16.0</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/4e1df3191aee1a902c873901b35d82363e13b08c"><code>4e1df31</code></a> Remove obsolete test, add make test</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/6ec76477061ea14394cadbf2cef673b04971ef4d"><code>6ec7647</code></a> feat: Add support for Python 3.10.</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/42ff5d176b2be5aa77a43f586179d563ae2196b5"><code>42ff5d1</code></a> ci: Include Python 3.10 in the CI test run.</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/be3b32777b32774c288b604f0d95eacb32a4d1a2"><code>be3b327</code></a> build: Include Python 3.10 in Tox test runs.</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/1c283bd821249ee3c695ae59bb022d31a1473ae2"><code>1c283bd</code></a> refactor: test_async_fixtures_with_finalizer no longer trigger a DeprecationW...</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/2751982f3a84d55aad103ea039cc779537e90720"><code>2751982</code></a> refactor: Replaced tests asserting that the event loop is properly closed.</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/70989fd7b626a86db4f75acc40057f982f2c76dd"><code>70989fd</code></a> refactor: Grouped test cases together that are related to the use of the asyn...</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/b27abe8d138998a75cc4eb519be810439d3ed0d9"><code>b27abe8</code></a> refactor: Removed TestUnexistingLoop.remove_loop fixture, because it has no e...</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/e3ec312c1d55b3688e4ce115df7791a0478582d1"><code>e3ec312</code></a> Adjusted Hypothesis integration test to use the same event loop initializatio...</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-asyncio/compare/v0.10.0...v0.16.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1494/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4tjgJ8
title: Allow routes to have extra options
state: open
created_at: 2021-10-22T15:00:45Z
updated_at: 2021-11-19T15:36:27Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1495
type: pull
body:
  Right now, datasette routes can only be a 2-tuple of `(regex, view_fn)`. 

  If it was possible for datasette to handle extra options, like [standard Django does](https://docs.djangoproject.com/en/3.2/topics/http/urls/#passing-extra-options-to-view-functions), it would add flexibility for plugin authors.

  For example, if extra options were enabled, then it would be easy to make a single table the home page (#1284). This plugin would accomplish it.

  ```python
  from datasette import hookimpl
  from datasette.views.table import TableView

  @hookimpl
  def register_routes(datasette):
          return [
          (r"^/$", TableView.as_view(datasette), {'db_name': 'DB_NAME',
                                                  'table': 'TABLE_NAME'})
      ]
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1495/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c49n4Wa
title: Named parameters docs should include an example of a cast
state: closed
created_at: 2021-10-22T18:56:04Z
updated_at: 2021-10-22T19:38:23Z
closed_at: 2021-10-22T19:34:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://docs.datasette.io/en/stable/sql_queries.html#named-parameters

  It's not obvious that the values from parameters are always SQLite strings, which means that you can't do e.g. integer comparisons on them without casting them first. The documentation here should include an example of this.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1496/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c49qcBZ
title: Publish to Docker Hub failing with "libcrypt.so.1: cannot open shared object file"
state: closed
created_at: 2021-10-24T22:57:07Z
updated_at: 2023-01-18T17:13:45Z
closed_at: 2021-10-24T23:36:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This means the Datasette 0.59.1 release has not been published to Docker Hub.

  Here's where that failed: https://github.com/simonw/datasette/runs/3991043374?check_suite_focus=true

  ```
  Preparing to unpack .../libc6_2.32-4_amd64.deb ...
  debconf: unable to initialize frontend: Dialog
  debconf: (TERM is not set, so the dialog frontend is not usable.)
  debconf: falling back to frontend: Readline
  debconf: unable to initialize frontend: Readline
  debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
  debconf: falling back to frontend: Teletype
  Checking for services that may need to be restarted...
  Checking init scripts...
  Unpacking libc6:amd64 (2.32-4) over (2.28-10) ...
  Setting up libc6:amd64 (2.32-4) ...
  /usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
  dpkg: error processing package libc6:amd64 (--configure):
   installed libc6:amd64 package post-installation script subprocess returned error exit status 127
  Errors were encountered while processing:
   libc6:amd64
  E: Sub-process /usr/bin/dpkg returned an error code (1)
  The command '/bin/sh -c apt-get update &&     apt-get -y --no-install-recommends install software-properties-common &&     add-apt-repository "deb http://httpredir.debian.org/debian sid main" &&     apt-get update &&     apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite &&     apt-get remove -y software-properties-common &&     apt clean &&     rm -rf /var/lib/apt &&     rm -rf /var/lib/dpkg/info/*' returned a non-zero code: 100
  ```
  Same problem when I attempted to publish using the "Push specific Docker tag" workflow: https://github.com/simonw/datasette/runs/3991059912?check_suite_focus=true

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1497/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4t0uaI
title: Add functionality to read Parquet files.
state: closed
created_at: 2021-10-28T23:43:19Z
updated_at: 2021-11-25T19:47:35Z
closed_at: 2021-11-25T19:47:35Z
author_association: NONE
pull_request: simonw/sqlite-utils/pulls/333
type: pull
body:
  I needed this for a project of mine, and I thought it'd be useful to have it in sqlite-utils (It's also mentioned in #248 ).
  The current implementation works (data is read & data types are inferred correctly.
  I've added a single straightforward test case, but @simonw please let me know if there are any non-obvious flags/combinations I should test too.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/333/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4t7RKr
title: Bump black from 21.9b0 to 21.10b0
state: closed
created_at: 2021-11-01T13:11:23Z
updated_at: 2021-11-17T13:14:00Z
closed_at: 2021-11-17T13:13:58Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1500
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.10b0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>21.10b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li>
  <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li>
  <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li>
  <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (<a href="https://github-redirect.dependabot.com/psf/black/issues/2519">#2519</a>)</li>
  <li>Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
  even though it has been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2580">#2580</a>)</li>
  <li>Deprecate Python 2 formatting support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2523">#2523</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Remove dependency on aiohttp-cors (<a href="https://github-redirect.dependabot.com/psf/black/issues/2500">#2500</a>)</li>
  <li>Bump required aiohttp version to 3.7.4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2509">#2509</a>)</li>
  </ul>
  <h3><em>Black-Primer</em></h3>
  <ul>
  <li>Add primer support for --projects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2555">#2555</a>)</li>
  <li>Print primer summary after individual failures (<a href="https://github-redirect.dependabot.com/psf/black/issues/2570">#2570</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Allow to pass <code>target_version</code> in the vim plugin (<a href="https://github-redirect.dependabot.com/psf/black/issues/1319">#1319</a>)</li>
  <li>Install build tools in docker file and use multi-stage build to keep the image size
  down (<a href="https://github-redirect.dependabot.com/psf/black/issues/2582">#2582</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>21.10b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li>
  <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li>
  <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li>
  <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (<a href="https://github-redirect.dependabot.com/psf/black/issues/2519">#2519</a>)</li>
  <li>Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
  even though it has been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2580">#2580</a>)</li>
  <li>Deprecate Python 2 formatting support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2523">#2523</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Remove dependency on aiohttp-cors (<a href="https://github-redirect.dependabot.com/psf/black/issues/2500">#2500</a>)</li>
  <li>Bump required aiohttp version to 3.7.4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2509">#2509</a>)</li>
  </ul>
  <h3><em>Black-Primer</em></h3>
  <ul>
  <li>Add primer support for --projects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2555">#2555</a>)</li>
  <li>Print primer summary after individual failures (<a href="https://github-redirect.dependabot.com/psf/black/issues/2570">#2570</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Allow to pass <code>target_version</code> in the vim plugin (<a href="https://github-redirect.dependabot.com/psf/black/issues/1319">#1319</a>)</li>
  <li>Install build tools in docker file and use multi-stage build to keep the image size
  down (<a href="https://github-redirect.dependabot.com/psf/black/issues/2582">#2582</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/psf/black/commits">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.9b0&new-version=21.10b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1500/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4-GEdL
title: Filter by datetime objects using rows_where()
state: closed
created_at: 2021-11-02T00:44:08Z
updated_at: 2021-11-13T19:23:21Z
closed_at: 2021-11-13T19:23:21Z
author_association: NONE
type: issue
state_reason: completed
body:
  Firstly, thanks for this nice utility. 
  It would be nice to have an example in the docs on how to filter by date range using `rows_where()`. 
  This doesn't seem to work:
  ```
  table.rows_where('datetime(created) between datetime("2021-10-31T17:29:59.277428-04:00") AND datetime("2021-11-01T03:44:04.544651+00:00")')
  ```


  I could probably just use `db.query()`, which works for the above, but it would be nice if I could pass in `datetime` objects in `rows_where()`.
  Thanks.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/334/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4-JFnX
title: sqlite-utils index-foreign-keys fails due to pre-existing index
state: closed
created_at: 2021-11-02T16:22:11Z
updated_at: 2021-11-14T22:55:56Z
closed_at: 2021-11-14T22:55:56Z
author_association: NONE
type: issue
state_reason: completed
body:
  While running the command:
  ```sh
  sqlite-utils index-foreign-keys $SQLITE_DIR/pudl.sqlite
  ```

  I got the following error:

  ```
  Traceback (most recent call last):
    File "/home/zane/miniconda3/envs/pudl-dev/bin/sqlite-utils", line 8, in <module>
      sys.exit(cli())
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 829, in __call__
      return self.main(*args, **kwargs)
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 782, in main
      rv = self.invoke(ctx)
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/click/core.py", line 610, in invoke
      return callback(*args, **kwargs)
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/cli.py", line 454, in index_foreign_keys
      db.index_foreign_keys()
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/db.py", line 902, in index_foreign_keys
      table.create_index([fk.column])
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/db.py", line 1563, in create_index
      self.db.execute(sql)
    File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.9/site-packages/sqlite_utils/db.py", line 421, in execute
      return self.conn.execute(sql)
  sqlite3.OperationalError: index idx_generators_eia860_report_date already exists
  ```

  This DB was created with the foreign key constraint `PRAGMA` enabled and a bunch of column-level `CHECK` constraints.  Is this an expected behavior? Should one not try to index foreign keys if FK constraints are already being enforced within the DB?

  I'm also noticing that the size of the DB after FK indexes have been added went from 483MB to 835MB, which seems like a much bigger jump than when I've done this previously.

  Software versions...
  * sqlite-utils 3.17.1
  * sqlite 3.36.0
  * SQLAlchemy 1.4.26 (used to create the DB)

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/335/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOEhK-wc4uAJb9
title: include note tags in the export
state: open
created_at: 2021-11-02T20:04:31Z
updated_at: 2021-11-02T20:04:31Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/evernote-to-sqlite/pulls/15
type: pull
body:
  When parsing the Evernote `<note>` elements, the script will now also parse any nested `<tag>` elements, writing them out into a separate sqlite table.

  Here is an example of how to query the data after the script has run:
  ```
  select notes.*,
  	(select group_concat(tag) from notes_tags where notes_tags.note_id=notes.id) as tags
  from notes;
  ```

  My .enex source file is 3+ years old so I am assuming the structure hasn't changed.  Interestingly, my _notebook names_ show up in the _tags_ list where the tag name is prefixed with `notebook_`, so this could maybe help work around the first limitation mentioned in the [evernote-to-sqlite blog post](https://simonwillison.net/2020/Oct/16/building-evernote-sqlite-exporter/).

reactions:
  {"url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4-PkFE
title: sqlite-util tranform --column-order mangles columns of type "timestamp"
state: closed
created_at: 2021-11-04T01:15:38Z
updated_at: 2023-05-08T21:13:38Z
closed_at: 2023-05-08T21:13:38Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  Reproducible code below:

  ```bash
  > echo 'create table bar (baz text, created_at timestamp default CURRENT_TIMESTAMP)' | sqlite3 foo.db
  > sqlite3 foo.db
  SQLite version 3.36.0 2021-06-18 18:36:39
  Enter ".help" for usage hints.
  sqlite> .schema bar
  CREATE TABLE bar (baz text, created_at timestamp default CURRENT_TIMESTAMP);
  sqlite> .exit
  > sqlite-utils transform foo.db bar --column-order baz
  sqlite3 foo.db
  SQLite version 3.36.0 2021-06-18 18:36:39
  Enter ".help" for usage hints.
  sqlite> .schema bar
  CREATE TABLE IF NOT EXISTS "bar" (
     [baz] TEXT,
     [created_at] FLOAT DEFAULT 'CURRENT_TIMESTAMP'
  );
  sqlite> .exit
  > sqlite-utils transform foo.db bar --column-order baz
  > sqlite3 foo.db
  SQLite version 3.36.0 2021-06-18 18:36:39
  Enter ".help" for usage hints.
  sqlite> .schema bar
  CREATE TABLE IF NOT EXISTS "bar" (
     [baz] TEXT,
     [created_at] FLOAT DEFAULT '''CURRENT_TIMESTAMP'''
  );
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/336/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4uK5z2
title: Default values for `--attach` and `--param` options
state: closed
created_at: 2021-11-05T21:57:53Z
updated_at: 2021-11-05T22:33:03Z
closed_at: 2021-11-05T22:33:02Z
author_association: NONE
pull_request: simonw/sqlite-utils/pulls/337
type: pull
body:
  It seems that `click` 8.x uses `None` as the default value for `multiple=True` options.

  This change makes the code forward-compatible with `click` 8.x.

  See this build failure for more info: https://hydra.nixos.org/build/156926608

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/337/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDODFE5qs4uMsMJ
title: Removed space from filename My Activity.json
state: open
created_at: 2021-11-08T00:04:31Z
updated_at: 2021-11-08T00:04:31Z
author_association: FIRST_TIME_CONTRIBUTOR
pull_request: dogsheep/google-takeout-to-sqlite/pulls/9
type: pull
body:
  File name from google takeout has no space. The code only runs without error if filename is "MyActivity.json" and not "My Activity.json". Is it a new change by Google?

reactions:
  {"url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-lOrH
title: Full-text search: No support to unary "-" operator
state: open
created_at: 2021-11-10T15:11:19Z
updated_at: 2021-11-10T15:11:19Z
author_association: NONE
type: issue
body:
  Reference: https://www.sqlite.org/fts3.html#set_operations_using_the_standard_query_syntax

  Test: https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort+-freedman&_sort=rowid

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1502/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-mDjo
title: `?_nocol=` removes that column from the filter interface
state: closed
created_at: 2021-11-10T18:22:50Z
updated_at: 2021-11-14T05:08:27Z
closed_at: 2021-11-14T04:53:07Z
author_association: OWNER
type: issue
state_reason: completed
body:
  e.g. on https://latest.datasette.io/fixtures/sortable?_nocol=sortable

  <img width="723" alt="fixtures__sortable__201_rows" src="https://user-images.githubusercontent.com/9599/141170777-733b1ab2-c153-4276-8ba2-d975ed84156e.png">

  This causes weird behaviour when you e.g. facet by a hidden column, since selecting facets and then re-submitting the form will clear the selected filter.

  ![nocol-bug](https://user-images.githubusercontent.com/9599/141171135-aded71d1-a4cb-4b7f-a4ea-26828fa98906.gif)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1503/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-qTem
title: Link to ?_size=max at bottom of table page
state: open
created_at: 2021-11-11T19:06:33Z
updated_at: 2021-11-11T19:06:33Z
author_association: OWNER
type: issue
body:
  This can have text such as "Show 1,000 rows per page", based on the max size limit setting. Would make it easier for people to see more data at once without having to know how to hack the URL, similar to the `...` for facet sizes I added in #1337.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1504/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-uAPv
title: Datasette should have an option to output CSV with semicolons
state: open
created_at: 2021-11-12T18:02:21Z
updated_at: 2021-11-16T11:40:52Z
author_association: OWNER
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1505/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-wNm2
title: Columns beginning with an underscore do not facet correctly
state: closed
created_at: 2021-11-14T02:20:32Z
updated_at: 2021-11-14T04:45:21Z
closed_at: 2021-11-14T04:45:21Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Datasette treats columns that start with an underscore as querystring parameters it should ignore!

  <img width="914" alt="bchydro__item_versions__99_918_rows" src="https://user-images.githubusercontent.com/9599/141664864-119990d5-42b2-47f7-91bd-09a74a288af9.png">

  Discovered in https://github.com/simonw/git-history/issues/14#issuecomment-968192464

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1506/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-wTvo
title: ReadTheDocs build failed for 0.59.2 release
state: closed
created_at: 2021-11-14T05:24:34Z
updated_at: 2021-11-14T05:41:55Z
closed_at: 2021-11-14T05:41:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I had to cancel the 0.59.2 release because ReadTheDocs was failing to build the documentation.

  https://readthedocs.org/projects/datasette/builds/15268454/

  ```
   /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m sphinx -T -b html -d _build/doctrees -D language=en . _build/html
  Running Sphinx v1.8.5
  loading translations [en]... done
  making output directory...
  building [mo]: targets for 0 po files that are out of date
  building [html]: targets for 27 source files that are out of date
  updating environment: 27 added, 0 changed, 0 removed
  reading sources... [  3%] authentication

  Traceback (most recent call last):
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/cmd/build.py", line 304, in build_main
      app.build(args.force_all, filenames)
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/application.py", line 341, in build
      self.builder.build_update()
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 347, in build_update
      len(to_build))
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 360, in build
      updated_docnames = set(self.read())
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 468, in read
      self._read_serial(docnames)
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 490, in _read_serial
      self.read_doc(docname)
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 534, in read_doc
      doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/io.py", line 318, in read_doc
      pub.publish()
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py", line 219, in publish
      self.apply_transforms()
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py", line 200, in apply_transforms
      self.document.transformer.apply_transforms()
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py", line 90, in apply_transforms
      Transformer.apply_transforms(self)
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
      transform.apply(**kwargs)
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py", line 245, in apply
      apply_source_workaround(n)
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py", line 94, in apply_source_workaround
      for classifier in reversed(node.parent.traverse(nodes.classifier)):
  TypeError: argument to reversed() must be a sequence

  Exception occurred:
    File "/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py", line 94, in apply_source_workaround
      for classifier in reversed(node.parent.traverse(nodes.classifier)):
  TypeError: argument to reversed() must be a sequence
  The full traceback has been saved in /tmp/sphinx-err-vkl0oE.log, if you want to report the issue to the developers.
  Please also report this if it was a user error, so that a better error message can be provided next time.
  A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks! 
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1507/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4-xNh2
title: dict, list, tuple should all map to TEXT
state: closed
created_at: 2021-11-15T00:28:01Z
updated_at: 2021-11-15T00:36:03Z
closed_at: 2021-11-15T00:36:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947
  >
  > So the `COLUMN_TYPE_MAPPING` should include those  too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/322#issuecomment-968401459_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/338/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4-xV4s
title: `table.lookup()` option to populate additional columns when creating a record
state: closed
created_at: 2021-11-15T01:41:17Z
updated_at: 2021-11-15T02:02:34Z
closed_at: 2021-11-15T02:02:00Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > For the commits table I feel like I want a version of `table.lookup()` that can be passed additional columns to populate only if the record does not exist yet.

  _Originally posted by @simonw in https://github.com/simonw/git-history/issues/12#issuecomment-967455017_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/339/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4-xZZv
title: `hash_id: Optional[Any]` should be `hash_id: Optional[str]`
state: closed
created_at: 2021-11-15T02:12:39Z
updated_at: 2021-11-15T02:19:31Z
closed_at: 2021-11-15T02:19:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  In a few places:

  https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L642

  https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L751

  https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1049

  https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L1230

  But it's correct here:

  https://github.com/simonw/sqlite-utils/blob/54a2269e91ce72b059618662ed133a85f3d42e4a/sqlite_utils/db.py#L2470

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/341/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4uiE0n
title: Update docutils requirement from <0.18 to <0.19
state: closed
created_at: 2021-11-15T13:15:47Z
updated_at: 2021-11-30T02:35:19Z
closed_at: 2021-11-30T02:35:19Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1508
type: pull
body:
  Updates the requirements on [docutils](http://docutils.sourceforge.net/) to permit the latest version.


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1508/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-1nq3
title: Datasette 1.0 JSON API (and documentation)
state: open
created_at: 2021-11-15T23:22:45Z
updated_at: 2022-03-15T20:38:56Z
author_association: OWNER
body: The new JSON API in a stable, documented form.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1509/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-1n9o
title: Datasette 1.0 documented template context (maybe via API docs)
state: open
created_at: 2021-11-15T23:23:58Z
updated_at: 2023-06-28T02:05:21Z
author_association: OWNER
type: issue
body:
  Documented context plus protective unit tests. Goal is that custom templates built for 1.x will not break without a 2.x release.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1510/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-1ogH
title: Review plugin hooks for Datasette 1.0
state: open
created_at: 2021-11-15T23:26:05Z
updated_at: 2021-11-16T01:20:14Z
author_association: OWNER
type: issue
body:
  I need to perform a detailed review of the plugin interface - especially the plugin hooks like [register_facet_classes()](https://docs.datasette.io/en/stable/plugin_hooks.html#register-facet-classes) which I don't yet have complete confidence in.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1511/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4unfnq
title: New pattern for async view classes
state: closed
created_at: 2021-11-16T21:55:44Z
updated_at: 2021-11-17T01:39:54Z
closed_at: 2021-11-17T01:39:44Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1512
type: pull
body:
  Refs #878 - starting out with the new `AsyncBase` class implementing a pytest-inspired `asyncio` parallel execution mechanism.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1512/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-6S4R
title: Research: CTEs and union all to calculate facets AND query at the same time
state: closed
created_at: 2021-11-16T22:26:45Z
updated_at: 2021-11-16T23:41:46Z
closed_at: 2021-11-16T23:41:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Consider this page: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_search=plant&_facet=owner&_facet=country_long&_facet=primary_fuel

  Datasette needs to run the main query for the rows on that page, a count query for the total query, then a separate query for each of those three specified facets.

  This is a `_search=` query, so it needs to execute the FTS code once for the rows, again for the count, and then three more times for each of the facets.

  Could running that query as a CTE and doing the other queries as part of the same large query produce significant speed improvements?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1513/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4up0R0
title: Bump black from 21.9b0 to 21.11b0
state: closed
created_at: 2021-11-17T13:13:55Z
updated_at: 2021-11-18T13:11:17Z
closed_at: 2021-11-18T13:11:15Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1514
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.11b0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>21.11b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
  detection (<a href="https://github-redirect.dependabot.com/psf/black/issues/2592">#2592</a>)</li>
  <li>Add experimental PyPy support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2559">#2559</a>)</li>
  <li>Add partial support for the match statement. As it's experimental, it's only enabled
  when <code>--target-version py310</code> is explicitly specified (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Add support for parenthesized with (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Declare support for Python 3.10 for running Black (<a href="https://github-redirect.dependabot.com/psf/black/issues/2562">#2562</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Fixed vim plugin with Python 3.10 by removing deprecated distutils import (<a href="https://github-redirect.dependabot.com/psf/black/issues/2610">#2610</a>)</li>
  <li>The vim plugin now parses <code>skip_magic_trailing_comma</code> from pyproject.toml (<a href="https://github-redirect.dependabot.com/psf/black/issues/2613">#2613</a>)</li>
  </ul>
  <h2>21.10b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li>
  <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li>
  <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li>
  <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (<a href="https://github-redirect.dependabot.com/psf/black/issues/2519">#2519</a>)</li>
  <li>Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
  even though it has been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2580">#2580</a>)</li>
  <li>Deprecate Python 2 formatting support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2523">#2523</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Remove dependency on aiohttp-cors (<a href="https://github-redirect.dependabot.com/psf/black/issues/2500">#2500</a>)</li>
  <li>Bump required aiohttp version to 3.7.4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2509">#2509</a>)</li>
  </ul>
  <h3><em>Black-Primer</em></h3>
  <ul>
  <li>Add primer support for --projects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2555">#2555</a>)</li>
  <li>Print primer summary after individual failures (<a href="https://github-redirect.dependabot.com/psf/black/issues/2570">#2570</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Allow to pass <code>target_version</code> in the vim plugin (<a href="https://github-redirect.dependabot.com/psf/black/issues/1319">#1319</a>)</li>
  <li>Install build tools in docker file and use multi-stage build to keep the image size
  down (<a href="https://github-redirect.dependabot.com/psf/black/issues/2582">#2582</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>21.11b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
  detection (<a href="https://github-redirect.dependabot.com/psf/black/issues/2592">#2592</a>)</li>
  <li>Add experimental PyPy support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2559">#2559</a>)</li>
  <li>Add partial support for the match statement. As it's experimental, it's only enabled
  when <code>--target-version py310</code> is explicitly specified (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Add support for parenthesized with (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Declare support for Python 3.10 for running Black (<a href="https://github-redirect.dependabot.com/psf/black/issues/2562">#2562</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Fixed vim plugin with Python 3.10 by removing deprecated distutils import (<a href="https://github-redirect.dependabot.com/psf/black/issues/2610">#2610</a>)</li>
  <li>The vim plugin now parses <code>skip_magic_trailing_comma</code> from pyproject.toml (<a href="https://github-redirect.dependabot.com/psf/black/issues/2613">#2613</a>)</li>
  </ul>
  <h2>21.10b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li>
  <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li>
  <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li>
  <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatibility (<a href="https://github-redirect.dependabot.com/psf/black/issues/2519">#2519</a>)</li>
  <li>Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
  even though it has been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2580">#2580</a>)</li>
  <li>Deprecate Python 2 formatting support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2523">#2523</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Remove dependency on aiohttp-cors (<a href="https://github-redirect.dependabot.com/psf/black/issues/2500">#2500</a>)</li>
  <li>Bump required aiohttp version to 3.7.4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2509">#2509</a>)</li>
  </ul>
  <h3><em>Black-Primer</em></h3>
  <ul>
  <li>Add primer support for --projects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2555">#2555</a>)</li>
  <li>Print primer summary after individual failures (<a href="https://github-redirect.dependabot.com/psf/black/issues/2570">#2570</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Allow to pass <code>target_version</code> in the vim plugin (<a href="https://github-redirect.dependabot.com/psf/black/issues/1319">#1319</a>)</li>
  <li>Install build tools in docker file and use multi-stage build to keep the image size
  down (<a href="https://github-redirect.dependabot.com/psf/black/issues/2582">#2582</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/psf/black/commits">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.9b0&new-version=21.11b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1514/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4-_Kpr
title: Handle foreign keys that point to a non-existent table
state: open
created_at: 2021-11-17T23:40:13Z
updated_at: 2021-11-18T01:31:56Z
author_association: OWNER
type: issue
body:
  Spotted in https://github.com/simonw/datasette-graphql/issues/79

  Demo: https://datasette-graphql-demo.datasette.io/fixtures/bad_foreign_key

  The foreign key links to a 404 page.

  ![B87009C7-CFCA-4DF9-8FBA-FA3E6CA28EC2](https://user-images.githubusercontent.com/9599/142334788-4d1a4acd-bc87-4426-b333-d46b221afcec.jpeg)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1515/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4utsKs
title: Bump black from 21.9b0 to 21.11b1
state: closed
created_at: 2021-11-18T13:11:12Z
updated_at: 2021-11-30T02:35:29Z
closed_at: 2021-11-30T02:35:29Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1516
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.11b1.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>21.11b1</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Bumped regex version minimum to 2021.4.4 to fix Pattern class usage (<a href="https://github-redirect.dependabot.com/psf/black/issues/2621">#2621</a>)</li>
  </ul>
  <h2>21.11b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
  detection (<a href="https://github-redirect.dependabot.com/psf/black/issues/2592">#2592</a>)</li>
  <li>Add experimental PyPy support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2559">#2559</a>)</li>
  <li>Add partial support for the match statement. As it's experimental, it's only enabled
  when <code>--target-version py310</code> is explicitly specified (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Add support for parenthesized with (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Declare support for Python 3.10 for running Black (<a href="https://github-redirect.dependabot.com/psf/black/issues/2562">#2562</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Fixed vim plugin with Python 3.10 by removing deprecated distutils import (<a href="https://github-redirect.dependabot.com/psf/black/issues/2610">#2610</a>)</li>
  <li>The vim plugin now parses <code>skip_magic_trailing_comma</code> from pyproject.toml (<a href="https://github-redirect.dependabot.com/psf/black/issues/2613">#2613</a>)</li>
  </ul>
  <h2>21.10b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li>
  <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li>
  <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li>
  <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (<a href="https://github-redirect.dependabot.com/psf/black/issues/2519">#2519</a>)</li>
  <li>Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
  even though it has been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2580">#2580</a>)</li>
  <li>Deprecate Python 2 formatting support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2523">#2523</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Remove dependency on aiohttp-cors (<a href="https://github-redirect.dependabot.com/psf/black/issues/2500">#2500</a>)</li>
  <li>Bump required aiohttp version to 3.7.4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2509">#2509</a>)</li>
  </ul>
  <h3><em>Black-Primer</em></h3>
  <ul>
  <li>Add primer support for --projects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2555">#2555</a>)</li>
  <li>Print primer summary after individual failures (<a href="https://github-redirect.dependabot.com/psf/black/issues/2570">#2570</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Allow to pass <code>target_version</code> in the vim plugin (<a href="https://github-redirect.dependabot.com/psf/black/issues/1319">#1319</a>)</li>
  <li>Install build tools in docker file and use multi-stage build to keep the image size
  down (<a href="https://github-redirect.dependabot.com/psf/black/issues/2582">#2582</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>21.11b1</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Bumped regex version minimum to 2021.4.4 to fix Pattern class usage (<a href="https://github-redirect.dependabot.com/psf/black/issues/2621">#2621</a>)</li>
  </ul>
  <h2>21.11b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
  detection (<a href="https://github-redirect.dependabot.com/psf/black/issues/2592">#2592</a>)</li>
  <li>Add experimental PyPy support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2559">#2559</a>)</li>
  <li>Add partial support for the match statement. As it's experimental, it's only enabled
  when <code>--target-version py310</code> is explicitly specified (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Add support for parenthesized with (<a href="https://github-redirect.dependabot.com/psf/black/issues/2586">#2586</a>)</li>
  <li>Declare support for Python 3.10 for running Black (<a href="https://github-redirect.dependabot.com/psf/black/issues/2562">#2562</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Fixed vim plugin with Python 3.10 by removing deprecated distutils import (<a href="https://github-redirect.dependabot.com/psf/black/issues/2610">#2610</a>)</li>
  <li>The vim plugin now parses <code>skip_magic_trailing_comma</code> from pyproject.toml (<a href="https://github-redirect.dependabot.com/psf/black/issues/2613">#2613</a>)</li>
  </ul>
  <h2>21.10b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Document stability policy, that will apply for non-beta releases (<a href="https://github-redirect.dependabot.com/psf/black/issues/2529">#2529</a>)</li>
  <li>Add new <code>--workers</code> parameter (<a href="https://github-redirect.dependabot.com/psf/black/issues/2514">#2514</a>)</li>
  <li>Fixed feature detection for positional-only arguments in lambdas (<a href="https://github-redirect.dependabot.com/psf/black/issues/2532">#2532</a>)</li>
  <li>Bumped typed-ast version minimum to 1.4.3 for 3.10 compatibility (<a href="https://github-redirect.dependabot.com/psf/black/issues/2519">#2519</a>)</li>
  <li>Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
  even though it has been removed (<a href="https://github-redirect.dependabot.com/psf/black/issues/2580">#2580</a>)</li>
  <li>Deprecate Python 2 formatting support (<a href="https://github-redirect.dependabot.com/psf/black/issues/2523">#2523</a>)</li>
  </ul>
  <h3><em>Blackd</em></h3>
  <ul>
  <li>Remove dependency on aiohttp-cors (<a href="https://github-redirect.dependabot.com/psf/black/issues/2500">#2500</a>)</li>
  <li>Bump required aiohttp version to 3.7.4 (<a href="https://github-redirect.dependabot.com/psf/black/issues/2509">#2509</a>)</li>
  </ul>
  <h3><em>Black-Primer</em></h3>
  <ul>
  <li>Add primer support for --projects (<a href="https://github-redirect.dependabot.com/psf/black/issues/2555">#2555</a>)</li>
  <li>Print primer summary after individual failures (<a href="https://github-redirect.dependabot.com/psf/black/issues/2570">#2570</a>)</li>
  </ul>
  <h3>Integrations</h3>
  <ul>
  <li>Allow to pass <code>target_version</code> in the vim plugin (<a href="https://github-redirect.dependabot.com/psf/black/issues/1319">#1319</a>)</li>
  <li>Install build tools in docker file and use multi-stage build to keep the image size
  down (<a href="https://github-redirect.dependabot.com/psf/black/issues/2582">#2582</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/psf/black/commits">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.9b0&new-version=21.11b1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1516/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_D71P
title: Let `register_routes()` over-ride default routes within Datasette
state: closed
created_at: 2021-11-19T00:22:15Z
updated_at: 2021-11-19T03:20:00Z
closed_at: 2021-11-19T03:07:27Z
author_association: OWNER
type: issue
state_reason: completed
body:
  See https://github.com/simonw/datasette/issues/878#issuecomment-973554024_ - right now `register_routes()` can't replace default Datasette routes.

  It would be neat if plugins could do this - especially if there was a neat documented way for them to then re-dispatch to the original route code after making some kind of modification.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1517/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_EOff
title: Complete refactor of TableView and table.html template
state: open
created_at: 2021-11-19T02:55:16Z
updated_at: 2022-03-15T18:35:49Z
author_association: OWNER
type: issue
body:
  Split from #878. The current `TableView` class is by far the most complex part of Datasette, and the most difficult to work on: https://github.com/simonw/datasette/blob/0.59.2/datasette/views/table.py

  In #878 I started exploring a new pattern for building views. In doing so it became clear that `TableView` is the first beast that I need to slay - if I can refactor that into something neat the pattern for building other views will emerge as a natural consequence.

  I've been trying to build this as a `register_routes()` plugin, as originally suggested in #870 - though unfortunately it looks like those plugins can't replace existing Datasette default views at the moment, see #1517. [UPDATE: I was wrong about this, plugins can over-ride default views just fine]

  I also know that I want to have a fully documented template context for `table.html` as a major step on the way to Datasette 1.0, see #1510.

  All of this adds up to the `TableView` factor being a major project that will unblock a whole flurry of other things - so I'm going to work on that in this separate issue.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1518/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_Esyh
title: Extra options to `lookup()` which get passed to `insert()`
state: closed
created_at: 2021-11-19T06:53:03Z
updated_at: 2021-11-19T07:26:54Z
closed_at: 2021-11-19T07:26:54Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For https://github.com/simonw/git-history/issues/12 I found myself wanting to pass extra options to `lookup()` to set the column order, primary key etc.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/342/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_G9yR
title: base_url is omitted in JSON and CSV views
state: closed
created_at: 2021-11-19T18:10:45Z
updated_at: 2021-12-01T17:50:09Z
closed_at: 2021-11-20T19:11:21Z
author_association: NONE
type: issue
state_reason: completed
body:
  I have a datasette deployment, using Apache2 to reverse proxy:

         ProxyPass /ged http://thor.phfactor.net:8001
         ProxyPreserveHost On

  In settings.json I have
  ```json
  {
      "base_url": "/ged/",
      "trace_debug": 1,
      "template_debug": 1
  }
  ```
  and datasette works correctly. However, if you view a query and then click on the 'This data as json, CSV' both links omit the base_url prefix and are therefore 404.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1519/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_HA4m
title: Pattern for avoiding accidental URL over-rides
state: open
created_at: 2021-11-19T18:28:05Z
updated_at: 2021-11-19T18:29:26Z
author_association: OWNER
type: issue
body:
  Following #1517 I'm experimenting with a plugin that does this:
  ```python
  @hookimpl
  def register_routes():
      return [
          (r"/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?)$", Table().view),
      ]
  ```
  This is supposed to replace the default table page with new code... but there's a problem: `/-/versions` on that instance now returns 404 `Database '-' does not exist`!

  Need to figure out a pattern to avoid that happening. Plugins get to add their routes before Datasette's default routes, which is why this is happening here.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1520/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_HD5F
title: Docker configuration for exercising Datasette behind Apache mod_proxy
state: closed
created_at: 2021-11-19T18:46:18Z
updated_at: 2021-11-19T20:32:29Z
closed_at: 2021-11-19T20:32:29Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Having a live demo running on Cloud Run that proxies through Apache and uses `base_url` would be incredibly useful for replicating and debugging this kind of thing. I wonder how hard it is to run Apache and `mod_proxy` in the same Docker container as Datasette?

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974310208_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1521/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_HXls
title: Deploy a live instance of demos/apache-proxy
state: closed
created_at: 2021-11-19T20:32:55Z
updated_at: 2021-11-23T03:00:34Z
closed_at: 2021-11-20T18:51:56Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I'll get this working on my laptop first, but then I want to get it up and running on Cloud Run - maybe with a GitHub Actions  workflow in this repo that re-deploys it on manual execution.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1521#issuecomment-974322178_

  I started by following https://ahmet.im/blog/cloud-run-multiple-processes-easy-way/ - see example in https://github.com/ahmetb/multi-process-container-lazy-solution

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1522/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_IkDE
title: Come up with a more elegant solution for base_url than ds.urls.path()
state: open
created_at: 2021-11-20T19:05:22Z
updated_at: 2021-11-20T19:05:22Z
author_association: OWNER
type: issue
body:
  While fixing #1519 I added a lot of ugly code that looks like this: https://github.com/simonw/datasette/blob/08947fa76433d18988aa1ee1d929bd8320c75fe2/datasette/facets.py#L228-L230

  See these two commits in particular: fe687fd0207c4c56c4778d3e92e3505fc4b18172  and 08947fa76433d18988aa1ee1d929bd8320c75fe2

  It would be great to come up with a less verbose and error-prone way of handling this problem.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1523/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_Imai
title: Improve Apache proxy documentation, link to demo
state: closed
created_at: 2021-11-20T20:03:14Z
updated_at: 2021-11-20T23:34:03Z
closed_at: 2021-11-20T23:34:03Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > The latest demo is now live at https://datasette-apache-proxy-demo.fly.dev/prefix/fixtures/sortable?_facet=pk2

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974697824_

  I'm going to put out 0.59.3 bugfix release with this, but I'd like to first improve the documentation on https://docs.datasette.io/en/stable/deploying.html#apache-proxy-configuration to highlight the new demo.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1524/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_Jtan
title: "Links from other tables" broken for columns starting with underscore
state: closed
created_at: 2021-11-21T22:55:08Z
updated_at: 2021-11-30T06:39:01Z
closed_at: 2021-11-30T06:34:35Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Same bug as #1506, this time it's this link or the row page:

  <img width="341" alt="image" src="https://user-images.githubusercontent.com/9599/142782167-fe9b6ffa-cd81-4a30-9f3b-401118c498e9.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1525/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_J3FT
title: Add to vercel.json, rather than overwriting it.
state: closed
created_at: 2021-11-22T00:47:12Z
updated_at: 2021-11-22T04:49:45Z
closed_at: 2021-11-22T04:13:47Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  I'd like to be able to add to vercel.json. But Datasette overwrites whatever I put in that file. I originally reported this here:
  https://github.com/simonw/datasette-publish-vercel/issues/51

  In that case, I wanted to do a rewrite... and now I need to do 301 redirects (because we had to rename our site).

  Can this be addressed?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1526/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_J4nP
title: Columns starting with an underscore behave poorly in filters
state: closed
created_at: 2021-11-22T01:01:36Z
updated_at: 2022-01-14T00:57:08Z
closed_at: 2022-01-14T00:57:08Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Similar bug to #1525 (and #1506 before it). Start on https://latest.datasette.io/fixtures/facetable?_facet=_neighborhood - then select a neighborhood - then try to remove that filter using the little "x" and submitting the form again.

  ![filter-bug](https://user-images.githubusercontent.com/9599/142786754-31d265a2-944d-4ea2-af6f-305d445a2ccb.gif)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1527/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_N_LZ
title: Add new `"sql_file"` key to Canned Queries in metadata?
state: open
created_at: 2021-11-22T21:58:01Z
updated_at: 2022-06-10T03:23:08Z
author_association: CONTRIBUTOR
type: issue
body:
  Currently for canned queries, you have to inline SQL in your `metadata.yaml` like so:

  ```yaml
  databases:
    fixtures:
      queries:
        neighborhood_search:
          sql: |-
            select neighborhood, facet_cities.name, state
            from facetable
              join facet_cities on facetable.city_id = facet_cities.id
            where neighborhood like '%' || :text || '%'
            order by neighborhood
          title: Search neighborhoods
  ```

  This works fine, but for a few reasons, I usually have my canned queries already written in separate `.sql` files. I'd like to instead re-use those instead of re-writing it. 

  So, I'd like to see a new `"sql_file"` key that works like so:

  `metadata.yaml`:

  ```yaml
  databases:
    fixtures:
      queries:
        neighborhood_search:
          sql_file: neighborhood_search.sql
          title: Search neighborhoods
  ```
  `neighborhood_search.sql`:
  ```sql
  select neighborhood, facet_cities.name, state
  from facetable
  join facet_cities on facetable.city_id = facet_cities.id
  where neighborhood like '%' || :text || '%'
  order by neighborhood
  ```

  Both of these would work in the exact same way, where Datasette would instead open + include `neighborhood_search.sql` on startup. 


  A few reasons why I'd like to keep my canned queries SQL separate from metadata.yaml:

  - Keeping SQL in standalone SQL files means syntax highlighting and other text editor integrations in my code
  - Multiline strings in yaml, while functional, are a tad cumbersome and are hard to edit
  - Works well with other tools (can pipe `.sql` files into the `sqlite3` CLI, or use with other SQLite clients easier)
  - Typically my canned queries are quite long compared to everything else in my metadata.yaml, so I'd love to separate it where possible

  Let me know if this is a feature you'd like to see, I can try to send up a PR if this sounds right!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1528/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4u9wQq
title: Update janus requirement from <0.7,>=0.6.2 to >=0.6.2,<0.8
state: closed
created_at: 2021-11-24T13:12:42Z
updated_at: 2021-11-30T02:37:13Z
closed_at: 2021-11-30T02:37:13Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1529
type: pull
body:
  Updates the requirements on [janus](https://github.com/aio-libs/janus) to permit the latest version.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/aio-libs/janus/blob/master/CHANGES.rst">janus's changelog</a>.</em></p>
  <blockquote>
  <h2>0.7.0 (2021-11-24)</h2>
  <ul>
  <li>Add SyncQueue and AsyncQueue Protocols to provide type hints for sync and async queues <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/374">#374</a></li>
  </ul>
  <h2>0.6.2 (2021-10-24)</h2>
  <ul>
  <li>Fix Python 3.10 compatibility <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/358">#358</a></li>
  </ul>
  <h2>0.6.1 (2020-10-26)</h2>
  <ul>
  <li>
  <p>Raise RuntimeError on queue.join() after queue closing. <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/295">#295</a></p>
  </li>
  <li>
  <p>Replace <code>timeout</code> type from <code>Optional[int]</code> to <code>Optional[float]</code> <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/267">#267</a></p>
  </li>
  </ul>
  <h2>0.6.0 (2020-10-10)</h2>
  <ul>
  <li>
  <p>Drop Python 3.5, the minimal supported version is Python 3.6</p>
  </li>
  <li>
  <p>Support Python 3.9</p>
  </li>
  <li>
  <p>Refomat with <code>black</code></p>
  </li>
  </ul>
  <h2>0.5.0 (2020-04-23)</h2>
  <ul>
  <li>Remove explicit loop arguments and forbid creating queues outside event loops <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/246">#246</a></li>
  </ul>
  <h2>0.4.0 (2018-07-28)</h2>
  <ul>
  <li>
  <p>Add <code>py.typed</code> macro <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/89">#89</a></p>
  </li>
  <li>
  <p>Drop python 3.4 support and fix minimal version python3.5.3 <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/88">#88</a></p>
  </li>
  <li>
  <p>Add property with that indicates if queue is closed <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/86">#86</a></p>
  </li>
  </ul>
  <h2>0.3.2 (2018-07-06)</h2>
  <ul>
  <li>Fixed python 3.7 support <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/97">#97</a></li>
  </ul>
  <h2>0.3.1 (2018-01-30)</h2>
  <ul>
  <li>Fixed bug with join() in case tasks are added by sync_q.put() <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/75">#75</a></li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/aio-libs/janus/commit/7b72d8577517422dd0719da1c6a0fb33d8a10e23"><code>7b72d85</code></a> Bump to 0.7</li>
  <li><a href="https://github.com/aio-libs/janus/commit/5389dec41aedb1d85243f146d655d123ed348ef0"><code>5389dec</code></a> add SyncQueue and AsyncQueue Protocols (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/374">#374</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/d54ae4230fe1fc78657262771ca8b96172dc91c4"><code>d54ae42</code></a> Bump docutils from 0.18 to 0.18.1 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/381">#381</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/76ccc92890e4fa28966a1b3fe873fc360a2a70e9"><code>76ccc92</code></a> Bump black from 21.11b0 to 21.11b1 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/380">#380</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/967c28426d509671e8f18327848046f7413805f6"><code>967c284</code></a> Bump black from 21.10b0 to 21.11b0 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/379">#379</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/4d625a2138ef85ca964b6a776472d19a92334433"><code>4d625a2</code></a> Bump bandit from 1.7.0 to 1.7.1 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/378">#378</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/e21eec2c04acfba189f7177d2981e24fc3674fd1"><code>e21eec2</code></a> Bump coverage from 6.1.1 to 6.1.2 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/377">#377</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/7ff20afb10149c2d020c388e81a22ed715149914"><code>7ff20af</code></a> Bump isort from 5.10.0 to 5.10.1 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/376">#376</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/67e3782dc540ee13858027c3521691e779fcdc16"><code>67e3782</code></a> Switch from setup.py to declarative setup.cfg (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/375">#375</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/8a0c4983da0b412644d1e84e6e1e2da6c671221e"><code>8a0c498</code></a> Bump isort from 5.9.3 to 5.10.0 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/373">#373</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/aio-libs/janus/compare/v0.6.2...v0.7.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1529/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_YgOF
title: Provide function to generate hash_id from specified columns
state: closed
created_at: 2021-11-25T10:12:12Z
updated_at: 2022-03-02T04:25:25Z
closed_at: 2022-03-02T04:25:25Z
author_association: NONE
type: issue
state_reason: completed
body:
  Hi

  I note that you define `_hash()` to create a `hash_id` from non-id column values in a table [here](https://github.com/simonw/sqlite-utils/blob/8f386a0d300d1b1c76132bb75972b755049fb742/sqlite_utils/db.py#L2996).

  It would be useful to be able to call a complementary function to generate a corresponding `_id` from a subset of specified columns when adding items to another table, eg to support the creation of foreign keys.

  Or is there a better pattern for doing that?

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/343/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODEm0Qs4_axZ4
title: Execution on Windows
state: open
created_at: 2021-11-26T00:24:34Z
updated_at: 2022-10-14T16:58:27Z
author_association: NONE
type: issue
body:
  My installation on Windows using pip has been successful. I have Python 3.6.

  How do I run twitter-to-sqlite? I cannot even figure out how "auth" is a command. I have python on my path:
  C:\prog\python\Python36;C:\prog\python\Python36\Scripts

  Where should the commands be executed, and where are the files created?

  Could some basics please be added to the documentation to get beginners started?

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/60/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_gSuw
title: Use datasette-table Web Component to guide the design of the JSON API for 1.0
state: open
created_at: 2021-11-28T20:37:18Z
updated_at: 2022-03-16T20:13:34Z
author_association: OWNER
type: issue
body:
  I realized that one of the reasons I'm having trouble committing to nailing down the JSON API for 1.0 is that I don't use it much myself - I use the `?_shape=array` one quite often, but I don't have any projects that are using the default, more fully-featured API.

  As an experiment I built a Web Component for embedding Datasette tables on pages - https://github.com/simonw/datasette-table - and I think it's actually going to be a really useful tool for helping me dog food the v1.0 API design.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1532/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_gTFX
title: Add `Link: rel="alternate"` header pointing to JSON for a table/query
state: closed
created_at: 2021-11-28T20:43:25Z
updated_at: 2022-02-02T07:56:51Z
closed_at: 2022-02-02T07:49:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Originally explored in https://github.com/simonw/datasette-notebook/issues/2#issuecomment-980789406 - I wanted an efficient way to scan a list of URLs and figure out which if any of those corresponded to Datasette tables, canned queries or SQL output that could be represented as a table on a page.

  It looks like a neat way to do that is with ` Link:` header like this:

  `Link: http://127.0.0.1:8058/fixtures/compound_three_primary_keys.json; rel="alternate"; type="application/datasette+json"`

  I can put a `<link href=...` in the page header too.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1533/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_gTVE
title: Maybe return JSON from HTML pages if `Accept: application/json` is sent
state: closed
created_at: 2021-11-28T20:48:09Z
updated_at: 2022-04-27T21:59:34Z
closed_at: 2022-02-02T23:39:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Relates to #1533 - and to the work I've been doing on the https://github.com/simonw/datasette-table Web Component.

  It would be useful to support users pasting in a URL to a Datasette table or query without first having to add the `.json` extension themselves - since then other systems could hit that URL with `Accept: application/json` to get back the JSON representation without first needing to read the `Link: ` header from #1533 to figure out what the URL to that JSON is.

  (There is weird logic deep in Datasette that says that you add `.json` to the path UNLESS the table name itself ends with `.json`, in which case you add `?_format=json` - this is super-confusing).

  [Update: I removed that confusing feature here: [https://simonwillison.net/2022/Mar/19/weeknotes/](https://simonwillison.net/2022/Mar/19/weeknotes/)]

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1534/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4vIJqi
title: Update aiofiles requirement from <0.8,>=0.4 to >=0.4,<0.9
state: closed
created_at: 2021-11-29T13:13:52Z
updated_at: 2021-11-30T02:29:55Z
closed_at: 2021-11-30T02:29:54Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1537
type: pull
body:
  Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version.
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/Tinche/aiofiles/commit/d010ff4d789598213334a32ec3d3f55caaab766c"><code>d010ff4</code></a> 0.8.0</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/3e3830daad8f41b1d3548a59c9913e4daf7be327"><code>3e3830d</code></a> Tweak README</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/713984eb46a62f8d504f31ee9b1305a55581675d"><code>713984e</code></a> Update README</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/95709758e635e9155ee5d96c973676cd0815221a"><code>9570975</code></a> Use os.sep</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/2b2500d555769460bc1f226f078d86eeaae06daf"><code>2b2500d</code></a> Add prefix, suffix, dir args to TemporaryDirectory</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/8895eb48b6004c8677da5841e2504a73eb8fad5e"><code>8895eb4</code></a> Fix ospath.exists test</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/83083555512320cf5c4cb10dd4dbda57c4cfbea1"><code>8308355</code></a> Fix typo: aiofiles.ospath.exists</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/b8cf8dd19369c2529181b3a4b8f1c485a2c05dbd"><code>b8cf8dd</code></a> Update Readme with os.makedirs and os.removedirs</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/aea353ebdd2771f2e4ae934077eceb12daaa3481"><code>aea353e</code></a> Merge branch 'patch-2'</li>
  <li><a href="https://github.com/Tinche/aiofiles/commit/761f06ede5925d0cdd22926b0d83564e9c1e1ab5"><code>761f06e</code></a> Fix README</li>
  <li>Additional commits viewable in <a href="https://github.com/Tinche/aiofiles/compare/v0.4.0...v0.8.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1537/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_jkYx
title: Research pattern for re-registering existing Click tools with register_commands
state: closed
created_at: 2021-11-29T17:09:47Z
updated_at: 2021-11-29T17:32:44Z
closed_at: 2021-11-29T17:27:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Building a Datasette plugin that imports an existing Click CLI tool and re-registers it is proving hard - Click doesn't really want you to do that. I tried this:
  ```python
  from datasette import hookimpl
  from git_history.cli import file as git_history_file


  @hookimpl
  def register_commands(cli):
      cli.command(name="git-history")(git_history_file.callback)
  ```
  But when I run this:
  ```
   % datasette git-history --help     
  Usage: datasette git-history [OPTIONS]

    Analyze the history of a specific file and write it to SQLite

  Options:
    --help  Show this message and exit.
  ```
  The options are all missing - which means that the command doesn't actually work. Will need to research this pattern separately.

  _Originally posted by @simonw in https://github.com/simonw/git-history/issues/21#issuecomment-981835305_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1538/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_kRrY
title: Support STRICT tables
state: closed
created_at: 2021-11-29T20:32:23Z
updated_at: 2023-12-08T05:22:39Z
closed_at: 2023-12-08T05:22:39Z
author_association: OWNER
body: New in SQLite 3.37.0, released a few days ago: https://www.sqlite.org/stricttables.html
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/344/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_kYWe
title: `table.strict` introspection boolean for identifying STRICT mode tables
state: closed
created_at: 2021-11-29T21:05:10Z
updated_at: 2021-11-29T22:45:26Z
closed_at: 2021-11-29T22:44:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > From the STRICT docs:

  >> The SQLite parser accepts a comma-separated list of table options after the final close parenthesis in a CREATE TABLE statement. As of this writing (2021-08-23) only two options are recognized:
  >> 
  >> -   STRICT
  >> -   [WITHOUT ROWID](https://www.sqlite.org/withoutrowid.html)
  > 
  > So I think I need to read the `CREATE TABLE` statement from the `sqlite_master` table, split on the last `)`, split those tokens on `,` and see if `create` is in there (case insensitive).

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982020757_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/345/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_knfi
title: Way to test SQLite 3.37 (and potentially other versions) in CI
state: open
created_at: 2021-11-29T22:21:06Z
updated_at: 2021-11-29T23:12:49Z
author_association: OWNER
type: issue
body:
  > Need to figure out a good pattern for testing this in CI too - it will currently skip the new tests if it doesn't have SQLite 3.37 or higher.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982076924_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/346/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4vKAzW
title: Test against pysqlite3 running SQLite 3.37
state: open
created_at: 2021-11-29T23:17:57Z
updated_at: 2021-12-11T01:02:19Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/347
body: Refs #346 and #344.
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/347/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_pOcy
title: Command for creating an empty database
state: closed
created_at: 2021-11-30T23:24:27Z
updated_at: 2022-01-13T07:06:59Z
closed_at: 2022-01-09T20:33:20Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I sometimes find the need to create an empty SQLite database file - for example if I want to enable WAL on it before using it with another script. I currently do that like this:

      sqlite3 my.db vacuum
      sqlite-utils enable-wal my.db

  It would be nice if `sqlite-utils` had a convenience command for doing this.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/348/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_pPRV
title: Research PRAGMA query_only
state: open
created_at: 2021-11-30T23:30:24Z
updated_at: 2021-11-30T23:30:24Z
author_association: OWNER
type: issue
body:
  https://www.sqlite.org/pragma.html#pragma_query_only

  > The query_only pragma prevents data changes on database files when enabled. When this pragma is enabled, any attempt to CREATE, DELETE, DROP, INSERT, or UPDATE will result in an [SQLITE_READONLY](https://www.sqlite.org/rescode.html#readonly) error. However, the database is not truly read-only. You can still run a [checkpoint](https://www.sqlite.org/wal.html#ckpt) or a [COMMIT](https://www.sqlite.org/lang_transaction.html) and the return value of the [sqlite3_db_readonly()](https://www.sqlite.org/c3ref/db_readonly.html) routine is not affected.

  Would it be worth adding this as an extra protection against accidental writes to a DB file over a read-only connection?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1539/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_tHVs
title: Idea: hover to reveal details of linked row
state: open
created_at: 2021-12-01T19:28:07Z
updated_at: 2021-12-09T23:38:39Z
author_association: OWNER
type: issue
body:
  <img width="698" alt="fara__item_version__7_rows_where_where__item___5236" src="https://user-images.githubusercontent.com/9599/144300364-ee621811-8f5a-4c7e-9950-199119092cf3.png">

  Hovering over that could work a little bit like GitHub issue links:

  ![hover](https://user-images.githubusercontent.com/9599/144300537-9cd9e9af-ac16-42db-842f-37661bc94063.gif)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1540/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_xRe8
title: Different default layout for row page
state: open
created_at: 2021-12-02T18:56:36Z
updated_at: 2021-12-02T18:56:54Z
author_association: OWNER
type: issue
body:
  The row page displays as a table even though it only has one table row.

  maybe default to the same display as the narrow page version, even for wide pages?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1541/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODFdgUs4_10Cl
title: View that combines issues and issue comments
state: open
created_at: 2021-12-04T00:34:33Z
updated_at: 2021-12-04T00:34:52Z
author_association: MEMBER
type: issue
body:
  I want to see a reverse chronologically ordered interface onto both issues and comments - essentially a unified log of comments and issues opened across one or multiple projects.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_3kRK
title: A way of creating indexes on newly created tables
state: open
created_at: 2021-12-05T18:56:12Z
updated_at: 2021-12-07T01:04:37Z
author_association: OWNER
type: issue
body:
  I'm writing code for https://github.com/simonw/git-history/issues/33 that creates a table inside a loop:

  ```python
  item_pk = db[item_table].lookup(
      {"_item_id": item_id},
      item_to_insert,
      column_order=("_id", "_item_id"),
      pk="_id",
  )
  ```
  I need to look things up by `_item_id` on this table, which means I need an index on that column (the table can get very big).

  But there's no mechanism in SQLite utils to detect if the table was created for the first time and add an index to it. And I don't want to run `CREATE INDEX IF NOT EXISTS` every time through the loop.

  This should work like the `foreign_keys=` mechanism.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/349/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4_5wp3
title: feature request: order and dependency of plugins (that use js)
state: open
created_at: 2021-12-06T12:40:45Z
updated_at: 2021-12-15T17:47:08Z
author_association: NONE
type: issue
body:
  I have been playing with datasette for the last couple of weeks and it is great! I am a big fan of `datasette-cluster-map` and wanted to enhance it a bit with a what I would call a sub-plugin. I basically want to add more controls to the map that cluster map provides. I have been looking into its code and how the plugin management works, but it seems what I am trying to do is not doable without hacks in js.

  Basically what would like to have is a way to say load my plugin after the plugins I depend on have been loaded and rendered. There seems to be no prior art where plugins have these dependencies on the js level so I was wondering if that could be added or if it exists how to do it.

  Basically what I want to do is:

  my-awesome-plugin has a dependency on datastte-cluster-map. Whenever datasette cluster map has finished rendering on page load, call my plugin, but no earlier. To make that work datasette probably needs some total order in which way plugins are loaded intialized.

  Since I am new to datastte, I may be missing something obvious, so please let me know if the above makes no sense.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1542/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4vb__Y
title: Bump black from 21.11b1 to 21.12b0
state: closed
created_at: 2021-12-06T13:11:16Z
updated_at: 2021-12-13T23:22:29Z
closed_at: 2021-12-13T23:22:29Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1543
type: pull
body:
  Bumps [black](https://github.com/psf/black) from 21.11b1 to 21.12b0.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/releases">black's releases</a>.</em></p>
  <blockquote>
  <h2>21.12b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Fix determination of f-string expression spans (<a href="https://github-redirect.dependabot.com/psf/black/issues/2654">#2654</a>)</li>
  <li>Fix bad formatting of error messages about EOF in multi-line statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2343">#2343</a>)</li>
  <li>Functions and classes in blocks now have more consistent surrounding spacing (<a href="https://github-redirect.dependabot.com/psf/black/issues/2472">#2472</a>)</li>
  </ul>
  <h4>Jupyter Notebook support</h4>
  <ul>
  <li>Cell magics are now only processed if they are known Python cell magics. Earlier, all cell magics were tokenized, leading to possible indentation errors e.g. with <code>%%writefile</code>. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2630">#2630</a>)</li>
  <li>Fix assignment to environment variables in Jupyter Notebooks (<a href="https://github-redirect.dependabot.com/psf/black/issues/2642">#2642</a>)</li>
  </ul>
  <h4>Python 3.10 support</h4>
  <ul>
  <li>Point users to using <code>--target-version py310</code> if we detect 3.10-only syntax (<a href="https://github-redirect.dependabot.com/psf/black/issues/2668">#2668</a>)</li>
  <li>Fix <code>match</code> statements with open sequence subjects, like <code>match a, b:</code> or <code>match a, *b:</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2639">#2639</a>) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2659">#2659</a>)</li>
  <li>Fix <code>match</code>/<code>case</code> statements that contain <code>match</code>/<code>case</code> soft keywords multiple times, like <code>match re.match()</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2661">#2661</a>)</li>
  <li>Fix <code>case</code> statements with an inline body (<a href="https://github-redirect.dependabot.com/psf/black/issues/2665">#2665</a>)</li>
  <li>Fix styling of starred expressions inside <code>match</code> subject (<a href="https://github-redirect.dependabot.com/psf/black/issues/2667">#2667</a>)</li>
  <li>Fix parser error location on invalid syntax in a <code>match</code> statement (<a href="https://github-redirect.dependabot.com/psf/black/issues/2649">#2649</a>)</li>
  <li>Fix Python 3.10 support on platforms without ProcessPoolExecutor (<a href="https://github-redirect.dependabot.com/psf/black/issues/2631">#2631</a>)</li>
  <li>Improve parsing performance on code that uses <code>match</code> under <code>--target-version py310</code> up to ~50% (<a href="https://github-redirect.dependabot.com/psf/black/issues/2670">#2670</a>)</li>
  </ul>
  <h3>Packaging</h3>
  <ul>
  <li>Remove dependency on <code>regex</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2644">#2644</a>) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2663">#2663</a>)</li>
  </ul>
  <hr />
  <h4>Thank you!</h4>
  <ul>
  <li><a href="https://github.com/isidentical"><code>@​isidentical</code></a> for the polishing up 3.10 syntax support (which they contributed in the first place!)</li>
  <li><a href="https://github.com/MarcoGorelli"><code>@​MarcoGorelli</code></a> for their ever-continuing work on Black's jupyter support</li>
  <li><a href="https://github.com/jalaziz"><code>@​jalaziz</code></a> for cleaning up our Pyinstaller CD workflow</li>
  <li><a href="https://github.com/hauntsaninja"><code>@​hauntsaninja</code></a> for helping us drop the <code>regex</code> dependency</li>
  </ul>
  <p>And also congrats to first contributors!</p>
  <ul>
  <li><a href="https://github.com/MatthewScholefield"><code>@​MatthewScholefield</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/psf/black/pull/2631">psf/black#2631</a></li>
  <li><a href="https://github.com/AshIsbitt"><code>@​AshIsbitt</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/psf/black/pull/2632">psf/black#2632</a></li>
  <li><a href="https://github.com/kalbasit"><code>@​kalbasit</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/psf/black/pull/2638">psf/black#2638</a></li>
  <li><a href="https://github.com/danieleades"><code>@​danieleades</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/psf/black/pull/2653">psf/black#2653</a></li>
  <li><a href="https://github.com/danielsparing"><code>@​danielsparing</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/psf/black/pull/2630">psf/black#2630</a></li>
  <li><a href="https://github.com/tanvimoharir"><code>@​tanvimoharir</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/psf/black/pull/2343">psf/black#2343</a></li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/psf/black/blob/main/CHANGES.md">black's changelog</a>.</em></p>
  <blockquote>
  <h2>21.12b0</h2>
  <h3><em>Black</em></h3>
  <ul>
  <li>Fix determination of f-string expression spans (<a href="https://github-redirect.dependabot.com/psf/black/issues/2654">#2654</a>)</li>
  <li>Fix bad formatting of error messages about EOF in multi-line statements (<a href="https://github-redirect.dependabot.com/psf/black/issues/2343">#2343</a>)</li>
  <li>Functions and classes in blocks now have more consistent surrounding spacing (<a href="https://github-redirect.dependabot.com/psf/black/issues/2472">#2472</a>)</li>
  </ul>
  <h4>Jupyter Notebook support</h4>
  <ul>
  <li>Cell magics are now only processed if they are known Python cell magics. Earlier, all
  cell magics were tokenized, leading to possible indentation errors e.g. with
  <code>%%writefile</code>. (<a href="https://github-redirect.dependabot.com/psf/black/issues/2630">#2630</a>)</li>
  <li>Fix assignment to environment variables in Jupyter Notebooks (<a href="https://github-redirect.dependabot.com/psf/black/issues/2642">#2642</a>)</li>
  </ul>
  <h4>Python 3.10 support</h4>
  <ul>
  <li>Point users to using <code>--target-version py310</code> if we detect 3.10-only syntax (<a href="https://github-redirect.dependabot.com/psf/black/issues/2668">#2668</a>)</li>
  <li>Fix <code>match</code> statements with open sequence subjects, like <code>match a, b:</code> or
  <code>match a, *b:</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2639">#2639</a>) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2659">#2659</a>)</li>
  <li>Fix <code>match</code>/<code>case</code> statements that contain <code>match</code>/<code>case</code> soft keywords multiple
  times, like <code>match re.match()</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2661">#2661</a>)</li>
  <li>Fix <code>case</code> statements with an inline body (<a href="https://github-redirect.dependabot.com/psf/black/issues/2665">#2665</a>)</li>
  <li>Fix styling of starred expressions inside <code>match</code> subject (<a href="https://github-redirect.dependabot.com/psf/black/issues/2667">#2667</a>)</li>
  <li>Fix parser error location on invalid syntax in a <code>match</code> statement (<a href="https://github-redirect.dependabot.com/psf/black/issues/2649">#2649</a>)</li>
  <li>Fix Python 3.10 support on platforms without ProcessPoolExecutor (<a href="https://github-redirect.dependabot.com/psf/black/issues/2631">#2631</a>)</li>
  <li>Improve parsing performance on code that uses <code>match</code> under <code>--target-version py310</code>
  up to ~50% (<a href="https://github-redirect.dependabot.com/psf/black/issues/2670">#2670</a>)</li>
  </ul>
  <h3>Packaging</h3>
  <ul>
  <li>Remove dependency on <code>regex</code> (<a href="https://github-redirect.dependabot.com/psf/black/issues/2644">#2644</a>) (<a href="https://github-redirect.dependabot.com/psf/black/issues/2663">#2663</a>)</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li>See full diff in <a href="https://github.com/psf/black/commits">compare view</a></li>
  </ul>
  </details>
  <br />


  [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.11b1&new-version=21.12b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1543/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_7A-0
title: Optional caching mechanism for table.lookup()
state: open
created_at: 2021-12-06T17:54:25Z
updated_at: 2021-12-06T17:56:57Z
author_association: OWNER
type: issue
body:
  Inspired by work on `git-history` where I used this pattern:
  ```python
      column_name_to_id = {}

      def column_id(column):
          if column not in column_name_to_id:
              id = db["columns"].lookup(
                  {"namespace": namespace_id, "name": column},
                  foreign_keys=(("namespace", "namespaces", "id"),),
              )
              column_name_to_id[column] = id
          return column_name_to_id[column]
  ```
  If you're going to be doing a large number of `table.lookup(...)` calls and you know that no other script will be modifying the database at the same time you can presumably get a big speedup using a Python in-memory cache - maybe even a LRU one to avoid memory bloat.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/350/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_8VU_
title: Support `--import xml.etree.ElementTree` in `sqlite-utils convert`
state: closed
created_at: 2021-12-07T00:40:29Z
updated_at: 2021-12-11T00:11:25Z
closed_at: 2021-12-11T00:11:25Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It's not possible to use a module that requires a nested import, such as `xml.etree.ElementTree`, at the moment. I found and fixed this bug in `git-history`, I should replicate that fix (and accompanying documentation) here: https://github.com/simonw/git-history/issues/39

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/351/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM4_8YO7
title: `sqlite-utils insert --extract colname`
state: open
created_at: 2021-12-07T00:55:44Z
updated_at: 2022-02-03T22:59:36Z
author_association: OWNER
type: issue
body:
  Is there a reason I've not added `--extract` as an option for `sqlite-utils insert` next? There's a `extracts=` option for the various `table.insert()` etc methods - last line in this code block:

  https://github.com/simonw/sqlite-utils/blob/213a0ff177f23a35f3b235386366ff132eb879f1/sqlite_utils/db.py#L2483-L2495

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/352/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c4__4z6
title: Code that detects the label column for a table is case-sensitive
state: closed
created_at: 2021-12-07T20:01:25Z
updated_at: 2021-12-07T20:03:43Z
closed_at: 2021-12-07T20:03:43Z
author_association: OWNER
body: I just noticed that a column called `Name` is not being picked up as the label column for a table.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1544/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AINQB
title: Custom pages don't work on windows
state: closed
created_at: 2021-12-09T18:53:05Z
updated_at: 2022-02-03T02:08:31Z
closed_at: 2022-02-03T01:58:35Z
author_association: NONE
type: issue
state_reason: completed
body:
  It seems that custom pages don't work when put in templates/pages

  To reproduce on datasette version 0.59.4  using PowerShell on WIndows 10 with Python 3.10.0

      mkdir -p templates/pages

      echo "hello world" >> templates/pages/about.html

  Start datasette
     
      datasette --template-dir templates/

  Navigate to [http://127.0.0.1:8001/about](url) and receive:
      
     Error 404:
     Database not found: about

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1545/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AI1YK
title: validating the sql
state: closed
created_at: 2021-12-09T21:35:57Z
updated_at: 2021-12-18T02:05:17Z
closed_at: 2021-12-18T02:05:16Z
author_association: NONE
type: issue
state_reason: completed
body:
  Could someone tell me that part of the code is responsible for validating the sql that guarantees that only a table can be read

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1546/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AKGDM
title: Writable canned queries fail to load custom templates
state: closed
created_at: 2021-12-10T03:31:48Z
updated_at: 2022-01-13T22:27:59Z
closed_at: 2021-12-19T21:12:00Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  I've created a canned query with `"write": true` set. I've also created a custom template for it, but the template doesn't seem to be found. If I look in the HTML I see (`stock_exchange` is the db name):

  `<!-- Templates considered: query-stock_exchange.html, *query.html -->`

  My non-writeable canned queries pick up custom templates as expected, and if I look at their HTML I see the canned query name added to the templates considered (the canned query here is `date_search`):

  `<!-- Templates considered: query-stock_exchange-date_search.html, query-stock_exchange.html, *query.html -->`

  So it seems like the writeable canned query is behaving differently for some reason. Is it an authentication thing? I'm using the built in `--root` authentication.

  Thanks!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1547/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4vrZxV
title: Update pytest-xdist requirement from <2.5,>=2.2.1 to >=2.2.1,<2.6
state: closed
created_at: 2021-12-10T13:12:06Z
updated_at: 2021-12-13T23:22:22Z
closed_at: 2021-12-13T23:22:21Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1548
type: pull
body:
  Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p>
  <blockquote>
  <h1>pytest-xdist 2.5.0 (2021-12-10)</h1>
  <h2>Features</h2>
  <ul>
  <li>
  <p><code>[#722](https://github.com/pytest-dev/pytest-xdist/issues/722) &lt;https://github.com/pytest-dev/pytest-xdist/issues/722&gt;</code>_: Full compatibility with pytest 7 - no deprecation warnings or use of legacy features.</p>
  </li>
  <li>
  <p><code>[#733](https://github.com/pytest-dev/pytest-xdist/issues/733) &lt;https://github.com/pytest-dev/pytest-xdist/issues/733&gt;</code>_: New <code>--dist=loadgroup</code> option, which ensures all tests marked with <code>@pytest.mark.xdist_group</code> run in the same session/worker. Other tests run distributed as in <code>--dist=load</code>.</p>
  </li>
  </ul>
  <h2>Trivial Changes</h2>
  <ul>
  <li>
  <p><code>[#708](https://github.com/pytest-dev/pytest-xdist/issues/708) &lt;https://github.com/pytest-dev/pytest-xdist/issues/708&gt;</code>_: Use <code>@pytest.hookspec</code> decorator to declare hook options in <code>newhooks.py</code> to avoid warnings in <code>pytest 7.0</code>.</p>
  </li>
  <li>
  <p><code>[#719](https://github.com/pytest-dev/pytest-xdist/issues/719) &lt;https://github.com/pytest-dev/pytest-xdist/issues/719&gt;</code>_: Use up-to-date <code>setup.cfg</code>/<code>pyproject.toml</code> packaging setup.</p>
  </li>
  <li>
  <p><code>[#720](https://github.com/pytest-dev/pytest-xdist/issues/720) &lt;https://github.com/pytest-dev/pytest-xdist/issues/720&gt;</code>_: Require pytest&gt;=6.2.0.</p>
  </li>
  <li>
  <p><code>[#721](https://github.com/pytest-dev/pytest-xdist/issues/721) &lt;https://github.com/pytest-dev/pytest-xdist/issues/721&gt;</code>_: Started using type annotations and mypy checking internally. The types are incomplete and not published.</p>
  </li>
  </ul>
  <h1>pytest-xdist 2.4.0 (2021-09-20)</h1>
  <h2>Features</h2>
  <ul>
  <li>
  <p><code>[#696](https://github.com/pytest-dev/pytest-xdist/issues/696) &lt;https://github.com/pytest-dev/pytest-xdist/issues/696&gt;</code>_: On Linux, the process title now changes to indicate the current worker state (running/idle).</p>
  <p>Depends on the <code>setproctitle &lt;https://pypi.org/project/setproctitle/&gt;</code>__ package, which can be installed with <code>pip install pytest-xdist[setproctitle]</code>.</p>
  </li>
  <li>
  <p><code>[#704](https://github.com/pytest-dev/pytest-xdist/issues/704) &lt;https://github.com/pytest-dev/pytest-xdist/issues/704&gt;</code>_: Add support for Python 3.10.</p>
  </li>
  </ul>
  <h1>pytest-xdist 2.3.0 (2021-06-16)</h1>
  <h2>Deprecations and Removals</h2>
  <ul>
  <li><code>[#654](https://github.com/pytest-dev/pytest-xdist/issues/654) &lt;https://github.com/pytest-dev/pytest-xdist/issues/654&gt;</code>_: Python 3.5 is no longer supported.</li>
  </ul>
  <h2>Features</h2>
  <ul>
  <li><code>[#646](https://github.com/pytest-dev/pytest-xdist/issues/646) &lt;https://github.com/pytest-dev/pytest-xdist/issues/646&gt;</code>_: Add <code>--numprocesses=logical</code> flag, which automatically uses the number of logical CPUs available, instead of physical CPUs with <code>auto</code>.</li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/13f39349c6950a881c1fe4fcd5984af2e8b7c220"><code>13f3934</code></a> Remove unnecessary skip from test_logfinish_hook as we require pytest&gt;=6.2</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/c76d5622f17135e892965d742377870eb9b07933"><code>c76d562</code></a> Skip test_warning_captured_deprecated_in_pytest_6 in pytest&gt;=7.1</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/5f78c7155e66ab73bdc7631c4ac6bfe684b82500"><code>5f78c71</code></a> Fix CHANGELOG header</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/c8bbc03e49d5a53b5da808c7328e8f3ad6ed2d7e"><code>c8bbc03</code></a> Release 2.5.0</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/8dbf3677dc7cc26ada33cf8a27d7ac51a9be467b"><code>8dbf367</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/738">#738</a> from pytest-dev/pre-commit-ci-update-config</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/a25c14bef59ad728e39cabc64f71190aaad73b0a"><code>a25c14b</code></a> [pre-commit.ci] pre-commit autoupdate</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/110c114025202d11570737be823de158d1bb8d99"><code>110c114</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/734">#734</a> from nicoddemus/revamp-readme</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/83bdbf4b95c914a889d1faa8fba8d506bcc2f8c7"><code>83bdbf4</code></a> Revamp README</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/630c1eb6f2c31dcb4c38c75bb62f868237cdde94"><code>630c1eb</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pytest-dev/pytest-xdist/issues/733">#733</a> from baekdohyeop/feature-loadgroup</li>
  <li><a href="https://github.com/pytest-dev/pytest-xdist/commit/62e50d00977b41e175b5f119381f9db760459ddc"><code>62e50d0</code></a> Address review</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-xdist/compare/v2.2.1...v2.5.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1548/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5AM0lW
title: Allow passing a file of code to "sqlite-utils convert"
state: closed
created_at: 2021-12-10T18:06:14Z
updated_at: 2021-12-11T01:38:29Z
closed_at: 2021-12-11T01:09:39Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  sqlite-utils is so nice, but the ergonomics of the multiline code in kind of tough. It's really hard (maybe impossible) to make the newlines play well with Makefiles.

  it would be great to write your code fragment in a separate file and direct it into the sqlite-utils

  either like

  ```sqlite-utils convert my.db my_table my_column < custom_code.py```

  or

  ```sqlite-utils convert my.db my_table my_column --custom-code=custom_code.py```

  Thanks, as ever, for these great tools!

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5ANW1g
title: Test failure in test_rebuild_fts
state: closed
created_at: 2021-12-10T21:27:55Z
updated_at: 2021-12-11T01:08:46Z
closed_at: 2021-12-11T01:08:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Not sure why this has only just started failing, but I'm getting this: https://github.com/simonw/sqlite-utils/runs/4488687639

  ```
  E           sqlite3.DatabaseError: database disk image is malformed

  sqlite_utils/db.py:425: DatabaseError
  _______________________ test_rebuild_fts[searchable_fts] _______________________

  fresh_db = <Database <sqlite3.Connection object at 0x1084ea9d0>>
  table_to_fix = 'searchable_fts'

      @pytest.mark.parametrize("table_to_fix", ["searchable", "searchable_fts"])
      def test_rebuild_fts(fresh_db, table_to_fix):
          table = fresh_db["searchable"]
          table.insert(search_records[0])
          table.enable_fts(["text", "country"])
          # Run a search
          rows = list(table.search("tanuki"))
          assert len(rows) == 1
          assert {
              "rowid": 1,
              "text": "tanuki are running tricksters",
              "country": "Japan",
              "not_searchable": "foo",
          }.items() <= rows[0].items()
          # Delete from searchable_fts_data
          fresh_db["searchable_fts_data"].delete_where()
          # This should have broken the index
          with pytest.raises(sqlite3.DatabaseError):
              list(table.search("tanuki"))
          # Running rebuild_fts() should fix it
  >       fresh_db[table_to_fix].rebuild_fts()
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5ANqEZ
title: Allow users to pass a full convert() function definition
state: closed
created_at: 2021-12-10T23:59:58Z
updated_at: 2021-12-11T00:51:15Z
closed_at: 2021-12-11T00:49:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I think the fix for this is to change the rules about what code is accepted in both the `-` mode and the literal code string mode: you can pass in a Python expression, OR a fragment that gets turned into a function, OR code that implements its own `def convert(value)` function. So this would work too:
  > ```sh
  > sqlite-utils convert my.db mytable col1 '
  > def convert(value):
  >     return value.upper()
  > '
  > ```
  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991381679_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/355/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5AOE6V
title: `sqlite-utils insert --convert` option
state: closed
created_at: 2021-12-11T07:24:48Z
updated_at: 2022-01-06T06:30:13Z
closed_at: 2022-01-06T06:28:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Idea come to me while re-reading this: https://simonwillison.net/2021/Aug/6/sqlite-utils-convert/

  This is a bit of a hack:
  ```
  cat /tmp/log.txt | \
      jq --raw-input '{line: .}' --compact-output | \
      sqlite-utils insert /tmp/logs.db log - --nl
  ```
  Would be great if you could pipe lines to `insert` and transform them on the way in.

  A `--convert python-code` option, modeled after `sqlite-utils convert`, could do this.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/356/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODEm0Qs5AOkMb
title: Data Pull fails for "Essential" level access to the Twitter API (for Documentation)
state: open
created_at: 2021-12-11T14:59:41Z
updated_at: 2022-10-31T14:47:58Z
author_association: NONE
type: issue
body:
  Per Twitter documentation:
  https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve

  This isn't any fault of twitter-to-sqlite of course, but it should probably be documented as a side-note.

  ![image](https://user-images.githubusercontent.com/57161638/145681272-8c85b3b9-be95-44ff-9760-1bafa4917ce2.png)

  And this is how I'm surfacing the message from utils.py:
  ![image](https://user-images.githubusercontent.com/57161638/145681005-2776c0ad-9822-4461-b43a-450ab2e828eb.png)

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/61/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AOzDb
title: Redesign CSV export to improve usability
state: open
created_at: 2021-12-11T19:02:12Z
updated_at: 2022-04-04T11:17:13Z
author_association: CONTRIBUTOR
type: issue
body:
  *Original title: Set content type for CSV so that browsers will attempt to download instead opening in the browser*

  Right now, if the user clicks on the CSV related to a <s>table or a</s> query, the response header for the content type is 

  "content-type: text/plain; charset=utf-8"

  Most browsers will try to open a file with this content-type in the browser. 

  This is not what most people want to do, and lots of folks don't know that if they want to download the CSV and open it in the a spreadsheet program they next need to save the page through their browser.

  It would be great if the response header could be something like 

  ```
  'Content-type: text/csv');
  'Content-disposition: attachment;filename=MyVerySpecial.csv');
  ```

  which would lead browsers to open a download dialog.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1549/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AO0yp
title: Research option for returning all rows from arbitrary query
state: open
created_at: 2021-12-11T19:31:11Z
updated_at: 2021-12-11T23:43:24Z
author_association: OWNER
type: issue
body:
  Inspired by thinking about #1549 - returning ALL rows from an arbitrary query is a lot easier if you just run that query and keep iterating over the cursor.

  I've avoided doing that in the past because it could tie up a connection for a long time - but in private instances this wouldn't be such a problem.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1550/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AP1eV
title: `keep_blank_values=True` when parsing `request.args`
state: closed
created_at: 2021-12-12T19:53:07Z
updated_at: 2022-01-13T22:26:04Z
closed_at: 2021-12-12T20:02:01Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This code in `TableView` wouldn't be necessary: https://github.com/simonw/datasette/blob/492f9835aa7e90540dd0c6324282b109f73df71b/datasette/views/table.py#L396-L399

  If that happened here instead: https://github.com/simonw/datasette/blob/492f9835aa7e90540dd0c6324282b109f73df71b/datasette/utils/asgi.py#L98-L100

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1518#issuecomment-991827468_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1551/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AS7Mb
title: Allow to set `facets_array` in metadata (like current `facets`)
state: closed
created_at: 2021-12-13T16:00:44Z
updated_at: 2022-01-13T22:26:15Z
closed_at: 2021-12-16T18:47:48Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  For now, you can set a `facets` value (array) in your metadata file but I couldn't find a way to set a `facets_array` in order to provide default facets for arrays (like tags). My use-case is to access to [that kind of view](https://latest.datasette.io/fixtures/facetable?_facet_array=tags) by default without URL's parameters as with other default facets.

  _I'm new to datasette, and I'm willing to help with a PR if that is not already implemented and I missed it!_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1552/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AUe9K
title: if csv export is truncated in non streaming mode set informative response header
state: open
created_at: 2021-12-13T22:50:44Z
updated_at: 2021-12-16T19:17:28Z
author_association: CONTRIBUTOR
type: issue
body:
  streaming mode is currently not enabled for custom queries, so the queries will be truncated to max row limit.

  it would be great if a response is truncated that an header signalling that was set in the header.

  i need to write some pagination code for getting full results back for a custom query and it would make the code much better if i could reliably known when there is nothing more to limit/offset 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1553/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4vynly
title: TableView refactor
state: closed
created_at: 2021-12-13T23:16:04Z
updated_at: 2021-12-20T23:52:11Z
closed_at: 2021-12-20T23:52:04Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1554
type: pull
body:
  I'm starting a PR with almost nothing in it so I can use the GitHub code commenting feature to add a bunch of comments to the code I intend to refactor.

  Related issues:

  - #617 
  - #715 
  - #870
  - #1518

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1554/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AUoRY
title: Optimize all those calls to index_list and foreign_key_list
state: closed
created_at: 2021-12-13T23:50:56Z
updated_at: 2022-01-13T22:27:32Z
closed_at: 2021-12-19T20:55:59Z
author_association: OWNER
type: issue
state_reason: completed
body:
  On the first hit to a restarted index I'm seeing this in the SQL traces: https://latest-with-plugins.datasette.io/github/commits?_trace=1

  <img width="826" alt="image" src="https://user-images.githubusercontent.com/9599/145907011-e4abc93c-8dbf-468d-afef-13d5e7463f92.png">

  I imagine this could be sped up a lot using tricks like this one from the SQLite documentation: https://sqlite.org/pragma.html#pragfunc

  ```sql
  SELECT DISTINCT m.name || '.' || ii.name AS 'indexed-columns'
    FROM sqlite_schema AS m,
         pragma_index_list(m.name) AS il,
         pragma_index_info(il.name) AS ii
   WHERE m.type='table'
   ORDER BY 1;
  ```
  https://latest-with-plugins.datasette.io/fixtures?sql=SELECT+DISTINCT+m.name+%7C%7C+%27.%27+%7C%7C+ii.name+AS+%27indexed-columns%27%0D%0A++FROM+sqlite_schema+AS+m%2C%0D%0A+++++++pragma_index_list%28m.name%29+AS+il%2C%0D%0A+++++++pragma_index_info%28il.name%29+AS+ii%0D%0A+WHERE+m.type%3D%27table%27%0D%0A+ORDER+BY+1%3B

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1555/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5AVq0H
title: pytest-runner is not required
state: closed
created_at: 2021-12-14T07:51:24Z
updated_at: 2021-12-16T20:43:19Z
closed_at: 2021-12-16T20:43:13Z
author_association: NONE
body: Deprecated pytest-runner is not necessary for running the testsuite.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/357/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Ac5tn
title: Show count of facet values always, not just for `?_facet_size=max`
state: closed
created_at: 2021-12-15T17:49:01Z
updated_at: 2022-01-13T22:26:07Z
closed_at: 2021-12-15T17:58:06Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > You've caused me to rethink this feature - I no longer think there's value in only showing these numbers if `?_facet_size=max` as opposed to all of the time.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1423#issuecomment-995023410_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1556/reactions", "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AhqEw
title: `?_nosuggest=1` parameter for disabling facet suggestions on table view
state: closed
created_at: 2021-12-16T19:21:42Z
updated_at: 2022-01-13T22:26:48Z
closed_at: 2021-12-16T19:24:59Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Found I wanted this while I was debugging #625 just to clean up the debug traces, but it makes sense as a partner to `?_nofacet=1` and `?_nocount=1` from #1350 and #1353.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1557/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Ahu2z
title: Redesign `facet_results` JSON structure prior to Datasette 1.0
state: open
created_at: 2021-12-16T19:45:10Z
updated_at: 2023-01-09T15:31:17Z
author_association: OWNER
type: issue
body:
  > Decision: as an initial fix I'm going to de-duplicate those keys by using `tags__array` etc - with a `_2` on the end if that key is already used.
  >
  > I'll open a separate issue to redesign this better for Datasette 1.0.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/625#issuecomment-996130862_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1558/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Ah_Qy
title: Support for CHECK constraints
state: open
created_at: 2021-12-16T21:19:45Z
updated_at: 2022-09-25T07:15:59Z
author_association: NONE
type: issue
body:
  Hi,

  I noticed the `transform.table()` method doesn't have an option to add/change or drop a check constraint (see https://sqlite.org/lang_createtable.html -> 3.7 Check Constraints. would be great to have this as an option! 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/358/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4v-izc
title: filters_from_request plugin hook, now used in TableView
state: closed
created_at: 2021-12-16T23:59:33Z
updated_at: 2021-12-17T23:09:41Z
closed_at: 2021-12-17T19:02:15Z
author_association: OWNER
pull_request: simonw/datasette/pulls/1559
type: pull
body:
  New plugin hook, refs #473

  Used it to extract the logic from TableView that handles _search and
  _through and _where - refs #1518

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1559/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AiWUl
title: Table page title has "where where" in it
state: closed
created_at: 2021-12-17T00:05:48Z
updated_at: 2022-01-13T22:28:35Z
closed_at: 2022-01-13T22:20:15Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Just noticed this while working on #1518.

  ```
  % curl -s 'https://latest.datasette.io/fixtures/facetable?_sort=pk&on_earth__exact=1' | grep -C 1 '<title>'
  <head>
      <title>fixtures: facetable: 14 rows
      where where on_earth = 1 sorted by pk</title>
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1560/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AibFW
title: add hash id to "_memory" url if hashed url mode is turned on and crossdb is also turned on
state: closed
created_at: 2021-12-17T00:45:12Z
updated_at: 2022-03-19T04:45:40Z
closed_at: 2022-03-19T04:45:40Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  If hashed_url mode is turned on and crossdb is also turned on, then queries to _memory should have a hash_id. 

  One way that it could work is to have the _memory hash be a hash of all the individual databases.

  Otherwise, crossdb queries can get quit out of data if using aggressive caching.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1561/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4wAMK8
title: Update janus requirement from <0.8,>=0.6.2 to >=0.6.2,<1.1
state: closed
created_at: 2021-12-17T13:11:10Z
updated_at: 2021-12-17T23:08:29Z
closed_at: 2021-12-17T23:08:28Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1562
type: pull
body:
  Updates the requirements on [janus](https://github.com/aio-libs/janus) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/aio-libs/janus/releases">janus's releases</a>.</em></p>
  <blockquote>
  <h2>janus 1.0.0 release</h2>
  <ul>
  <li>Dropped Python 3.6 support</li>
  <li>Janus is marked as stable, no API changes was made for years</li>
  </ul>
  </blockquote>
  </details>
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/aio-libs/janus/blob/master/CHANGES.rst">janus's changelog</a>.</em></p>
  <blockquote>
  <h2>1.0.0 (2021-12-17)</h2>
  <ul>
  <li>Drop Python 3.6 support</li>
  </ul>
  <h2>0.7.0 (2021-11-24)</h2>
  <ul>
  <li>Add SyncQueue and AsyncQueue Protocols to provide type hints for sync and async queues <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/374">#374</a></li>
  </ul>
  <h2>0.6.2 (2021-10-24)</h2>
  <ul>
  <li>Fix Python 3.10 compatibility <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/358">#358</a></li>
  </ul>
  <h2>0.6.1 (2020-10-26)</h2>
  <ul>
  <li>
  <p>Raise RuntimeError on queue.join() after queue closing. <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/295">#295</a></p>
  </li>
  <li>
  <p>Replace <code>timeout</code> type from <code>Optional[int]</code> to <code>Optional[float]</code> <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/267">#267</a></p>
  </li>
  </ul>
  <h2>0.6.0 (2020-10-10)</h2>
  <ul>
  <li>
  <p>Drop Python 3.5, the minimal supported version is Python 3.6</p>
  </li>
  <li>
  <p>Support Python 3.9</p>
  </li>
  <li>
  <p>Refomat with <code>black</code></p>
  </li>
  </ul>
  <h2>0.5.0 (2020-04-23)</h2>
  <ul>
  <li>Remove explicit loop arguments and forbid creating queues outside event loops <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/246">#246</a></li>
  </ul>
  <h2>0.4.0 (2018-07-28)</h2>
  <ul>
  <li>
  <p>Add <code>py.typed</code> macro <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/89">#89</a></p>
  </li>
  <li>
  <p>Drop python 3.4 support and fix minimal version python3.5.3 <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/88">#88</a></p>
  </li>
  <li>
  <p>Add property with that indicates if queue is closed <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/86">#86</a></p>
  </li>
  </ul>
  <h2>0.3.2 (2018-07-06)</h2>
  <ul>
  <li>Fixed python 3.7 support <a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/97">#97</a></li>
  </ul>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/aio-libs/janus/commit/0783f9b7a9bb7e1c095e93ebb4aad4f1e219f512"><code>0783f9b</code></a> Fix coverage upload</li>
  <li><a href="https://github.com/aio-libs/janus/commit/41c49bafb1b192d2ee25b7394cead2386e452dc2"><code>41c49ba</code></a> Make deployment only if checks are green</li>
  <li><a href="https://github.com/aio-libs/janus/commit/ec94b35b2ae095dcb97827f1369c0cd31b7e8e5e"><code>ec94b35</code></a> Fix CI again</li>
  <li><a href="https://github.com/aio-libs/janus/commit/2303208c2f972e38445e7ecec54fda0f3203f566"><code>2303208</code></a> Fix CI</li>
  <li><a href="https://github.com/aio-libs/janus/commit/dff507895bf8d77efea2c4cc1d8b04a8a2986a0b"><code>dff5078</code></a> Bump to 1.0.0</li>
  <li><a href="https://github.com/aio-libs/janus/commit/3421545f3954b7ef6d90e02b7653a7ab685f3e78"><code>3421545</code></a> Bump mypy from 0.910 to 0.920 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/384">#384</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/56b2d1d8dbd10cce28302a4e1c4224ce219c6246"><code>56b2d1d</code></a> Bump black from 21.11b1 to 21.12b0 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/383">#383</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/883e82bea0af1d12a68e92148b75b3344b31227a"><code>883e82b</code></a> Update README.rst</li>
  <li><a href="https://github.com/aio-libs/janus/commit/2e30d8a0f3c77c383a39da9b5c233a5c93a049fb"><code>2e30d8a</code></a> Bump coverage from 6.1.2 to 6.2 (<a href="https://github-redirect.dependabot.com/aio-libs/janus/issues/382">#382</a>)</li>
  <li><a href="https://github.com/aio-libs/janus/commit/7b72d8577517422dd0719da1c6a0fb33d8a10e23"><code>7b72d85</code></a> Bump to 0.7</li>
  <li>Additional commits viewable in <a href="https://github.com/aio-libs/janus/compare/v0.6.2...v1.0.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1562/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AlgPW
title: Datasette(... files=) should not be a required argument
state: closed
created_at: 2021-12-17T19:54:18Z
updated_at: 2022-01-13T22:27:18Z
closed_at: 2021-12-18T02:19:40Z
author_association: OWNER
type: issue
state_reason: completed
body:
  ```pycon
  >>> ds = Datasette(memory=True)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  TypeError: __init__() missing 1 required positional argument: 'files'
  >>> ds = Datasette(memory=True, files=[])
  ```
  I wanted to create an in-memory Datasette for running some tests, no point in forcing me to pass `files=[]` to do that.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1563/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AliJT
title: _prepare_connection not called on write connections
state: closed
created_at: 2021-12-17T20:06:47Z
updated_at: 2022-01-20T21:29:43Z
closed_at: 2021-12-18T01:58:44Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I was trying to initalize SpatiaLite in a write connection:
  ```pycon
  >>> from datasette.app import Datasette
  >>> ds = Datasette(memory=True, files=[], sqlite_extensions=["spatialite"])
  >>> db = ds.add_memory_database('geo')
  >>> await db.execute_write("select InitSpatialMetadata(1)")
  UUID('3f143baa-4e3d-5842-a36f-4fa2f683b72f')
  no such function: InitSpatialMetadata
  ```
  It looks like the code that loads additional modules only works on read-only connections, not on write connections:

  https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L146-L153

  Compared to:

  https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L124-L132

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1564/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Al06M
title: Documented JavaScript variables on different templates made available for plugins
state: open
created_at: 2021-12-17T22:30:51Z
updated_at: 2021-12-19T22:37:29Z
author_association: OWNER
type: issue
body:
  While working on https://github.com/simonw/datasette-leaflet-freedraw/issues/10 I found myself writing this atrocity to figure out the SQL query used for a specific table page:

  ```javascript
  let innerSql = Array.from(document.getElementsByTagName("span")).filter(
      el => el.innerText == "View and edit SQL"
  )[0].parentElement.getAttribute("title")
  ```
  This is obviously bad - it's very brittle, and will break if I ever change the text on that link (like localizing it for example).

  Instead, I think pages like that one should have a block of script at the bottom something like this:
  ```javascript
  window.datasette = window.datasette || {};
  datasette.view_name = 'table';
  datasette.table_sql = 'select * from ...';
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1565/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Al3ui
title: Release Datasette 0.60
state: closed
created_at: 2021-12-17T22:58:12Z
updated_at: 2022-01-14T01:59:55Z
closed_at: 2022-01-14T01:59:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Using this as a tracking issue. I'm hoping to get the bulk of the JSON redesign work from the refactor in #1554 in for this release.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1566/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AmD1W
title: Remove undocumented sqlite_functions mechanism
state: closed
created_at: 2021-12-18T01:51:10Z
updated_at: 2022-01-13T22:27:04Z
closed_at: 2021-12-18T01:54:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I added this in 0b8c1b0a6da9cb8ac0d28cc90dd783de87554036 but it's never been documented and the same thing can now be achieved using the `prepare_connection` plugin hook.

  https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L262

  https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L551-L552

  It's used here in the tests:

  https://github.com/simonw/datasette/blob/69244a617b1118dcbd04a8f102173f04680cf08c/tests/fixtures.py#L156

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1567/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AmFrW
title: Trace should show queries on the write connection too
state: closed
created_at: 2021-12-18T02:34:12Z
updated_at: 2022-01-13T22:27:23Z
closed_at: 2021-12-18T02:42:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Here's why - `trace` only applies to read, not write SQL operations: https://github.com/simonw/datasette/blob/7c8f8aa209e4ba7bf83976f8495d67c28fbfca24/datasette/database.py#L209-L211

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997128508_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1568/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Amu5j
title: db.execute_write(..., executescript=True) parameter
state: closed
created_at: 2021-12-18T18:20:47Z
updated_at: 2022-01-13T22:27:27Z
closed_at: 2021-12-18T18:34:18Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Idea: teach `execute_write` to accept an optional `executescript=True` parameter, like this:
  ```diff
  diff --git a/datasette/database.py b/datasette/database.py
  index 468e936..1a424f5 100644
  --- a/datasette/database.py
  +++ b/datasette/database.py
  @@ -94,10 +94,14 @@ class Database:
               f"file:{self.path}{qs}", uri=True, check_same_thread=False
           )
   
  -    async def execute_write(self, sql, params=None, block=False):
  +    async def execute_write(self, sql, params=None, executescript=False, block=False):
  +        assert not executescript and params, "Cannot use params with executescript=True"
           def _inner(conn):
               with conn:
  -                return conn.execute(sql, params or [])
  +                if executescript:
  +                    return conn.executescript(sql)
  +                else:
  +                    return conn.execute(sql, params or [])
   
           with trace("sql", database=self.name, sql=sql.strip(), params=params):
               results = await self.execute_write_fn(_inner, block=block)
  ```

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997248364_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1569/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Am1Pb
title: Separate db.execute_write() into three methods
state: closed
created_at: 2021-12-18T18:45:54Z
updated_at: 2022-01-13T22:27:38Z
closed_at: 2021-12-18T18:57:25Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Rather than adding a `executemany=True` parameter, I'm now thinking a better design might be to have three methods:
  >
  > - `db.execute_write(sql, params=None, block=False)`
  > - `db.execute_write_script(sql, block=False)`
  > - `db.execute_write_many(sql, params_seq, block=False)`

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997267416_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1570/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Am2pr
title: Track number of executions for execute_write_many() in traces
state: closed
created_at: 2021-12-18T19:16:17Z
updated_at: 2022-01-13T22:27:49Z
closed_at: 2021-12-19T20:30:40Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Spotted while working on #1555

  <img width="1171" alt="image" src="https://user-images.githubusercontent.com/9599/146653166-ce5358bc-b822-4961-a2dd-ed3efaa8dd13.png">

  There's no indication there of how many times `execute_write_many()` executed the SQL.

  Solving this is a tiny bit tricky because `params_seq` is an iterator that we don't want to exhaust before passing it to `conn.executemany()` - so we need to instead wrap it in something that counts how many times it was called.

  But then we need a way to attach that to the trace here: https://github.com/simonw/datasette/blob/d637ed46762fdbbd8e32b86f258cd9a53c1cfdc7/datasette/database.py#L115-L122

  So probably need to redesign the `trace()` decorator to allow extra pairs to be attached to it within the `with` statement.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1571/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AnKVl
title: "Query took" should be "Queries took"
state: closed
created_at: 2021-12-19T04:03:00Z
updated_at: 2022-01-13T22:27:43Z
closed_at: 2021-12-19T04:03:24Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This is misleading, since usually there have been more than one query executed:

  ![CleanShot 2021-12-18 at 20 02 35@2x](https://user-images.githubusercontent.com/9599/146663457-9c4c2900-5cc0-4650-a565-bb1ff0b8a725.png)

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1572/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5An1pk
title: Make trace() a documented internal API
state: open
created_at: 2021-12-19T20:32:56Z
updated_at: 2021-12-19T21:13:13Z
author_association: OWNER
type: issue
body:
  This should be documented so plugin authors can use it to add their own custom traces: https://github.com/simonw/datasette/blob/8f311d6c1d9f73f4ec643009767749c17b5ca5dd/datasette/tracer.py#L28-L52

  Including the new `kwargs` pattern I added in #1571: https://github.com/simonw/datasette/blob/f65817000fdf87ce8a0c23edc40784ebe33b5842/datasette/database.py#L128-L132

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1573/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4wDKmb
title: introduce new option for datasette package to use a slim base image
state: closed
created_at: 2021-12-19T21:18:19Z
updated_at: 2022-08-15T08:49:31Z
closed_at: 2022-08-15T08:49:31Z
author_association: NONE
pull_request: simonw/datasette/pulls/1574
type: pull
body:
  The official python images on docker hub come with a slim variant that is significantly smaller than the default. The diff does not change the default, but allows to switch to the `slim` variant with commandline switch (`--slim-base-image`)

  Size comparison:

  ```
  $ datasette package some.db -t fat --install "datasette-basemap datasette-cluster-map"

  $ datasette  package some.db -t slim --slim-base-image --install "datasette-basemap datasette-cluster-map"

  $ docker images
  REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
  fat          latest    807b393ace0d   9 seconds ago    978MB
  slim         latest    31bc5e63505c   8 minutes ago    191MB

  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1574/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AoHYy
title: __call__() got an unexpected keyword argument 'specname'
state: closed
created_at: 2021-12-20T01:24:04Z
updated_at: 2021-12-20T01:48:03Z
closed_at: 2021-12-20T01:47:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > I've installed the alpha version but get an error when starting up Datasette:

  ```
  Traceback (most recent call last):
    File "/Users/tim/.pyenv/versions/stock-exchange/bin/datasette", line 5, in <module>
      from datasette.cli import cli
    File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py", line 15, in <module>
      from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm
    File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py", line 31, in <module>
      from .views.database import DatabaseDownload, DatabaseView
    File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py", line 25, in <module>
      from datasette.plugins import pm
    File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py", line 29, in <module>
      mod = importlib.import_module(plugin)
    File "/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py", line 9, in <module>
      @hookimpl(specname="filters_from_request")
  TypeError: __call__() got an unexpected keyword argument 'specname'
  ```

  _Originally posted by @wragge in https://github.com/simonw/datasette/issues/1547#issuecomment-997511968_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1575/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5AzRR_
title: Traces should include SQL executed by subtasks created with `asyncio.gather`
state: closed
created_at: 2021-12-22T20:52:02Z
updated_at: 2022-02-05T05:21:35Z
closed_at: 2022-02-05T05:19:53Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I tried running some parallel SQL queries using `asyncio.gather()` but the SQL that was executed didn't show up in the trace rendered by https://datasette.io/plugins/datasette-pretty-traces

  I realized that was because traces are keyed against the current task ID, which changes when a sub-task is run using `asyncio.gather` or similar.

  The faceting and suggest faceting queries are missing from this trace:

  ![image](https://user-images.githubusercontent.com/9599/147153855-2d611f07-922a-4d18-9e6e-4be89e010dc4.png)

  > The reason they aren't showing up in the traces is that traces are stored just for the currently executing `asyncio` task ID: https://github.com/simonw/datasette/blob/ace86566b28280091b3844cf5fbecd20158e9004/datasette/tracer.py#L13-L25
  >
  > This is so traces for other incoming requests don't end up mixed together. But there's no current mechanism to track async tasks that are effectively "child tasks" of the current request, and hence should be tracked the same.
  >
  > https://stackoverflow.com/a/69349501/6083 suggests that you pass the task ID as an argument to the child tasks that are executed using `asyncio.gather()` to work around this kind of problem.

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1518#issuecomment-999870993_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1576/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5A2D78
title: Drop support for Python 3.6
state: closed
created_at: 2021-12-23T18:17:03Z
updated_at: 2022-01-25T23:30:03Z
closed_at: 2022-01-20T04:31:41Z
author_association: OWNER
type: issue
state_reason: completed
body:
  *Original title: Decide when to drop support for Python 3.6*

  > `context_vars` can solve this but they were introduced in Python 3.7: https://www.python.org/dev/peps/pep-0567/
  >
  > Python 3.6 support ends in a few days time, and it looks like Glitch has updated to 3.7 now - so maybe I can get away with Datasette needing 3.7 these days?
  >
  > Tweeted about that here: https://twitter.com/simonw/status/1473761478155010048

  _Originally posted by @simonw in https://github.com/simonw/datasette/issues/1576#issuecomment-999878907_

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1577/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5A2FUM
state: open
created_at: 2021-12-23T18:27:59Z
updated_at: 2021-12-24T21:33:19Z
author_association: OWNER
type: issue
title:
  Confirm if documented nginx proxy config works for row pages with escaped characters in their primary key

body:
  Found this while working on https://github.com/simonw/datasette-tiddlywiki

  <img width="1254" alt="image" src="https://user-images.githubusercontent.com/9599/147279097-e02f80f3-cc88-4bdd-a26f-03f924c13b5e.png">

  Then clicking on `/tiddlywiki/tiddlers/%24%3A%2FDefaultTiddlers` returns a 404.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1578/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5A2IYO
title: `.execute_write(... block=True)` should be the default behaviour
state: closed
created_at: 2021-12-23T18:54:28Z
updated_at: 2022-01-13T22:28:08Z
closed_at: 2021-12-23T19:18:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Every single piece of code I've written against the write APIs has used the `block=True` option to wait for the result.

  Without that, it instead fires the write into the queue but then continues even before it has finished executing.

  `block=True` should clearly be the default behaviour here!

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1579/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODEm0Qs5A5gdB
title: KeyError: 'created_at' for private accounts?
state: closed
created_at: 2021-12-26T17:51:51Z
updated_at: 2022-03-12T02:36:32Z
closed_at: 2022-02-24T18:10:18Z
author_association: NONE
type: issue
state_reason: completed
body:
  hey Simon!

  i was running `twitter-to-sqlite user-timeline twitter.db` for [my private alt](https://twitter.com/swyxio) and ran into this error:

  <details>
  <summary>

  ![image](https://user-images.githubusercontent.com/6764957/147416165-46b69c30-100a-406f-8534-8612b75547ae.png)

  </summary>



  ```bash
  Traceback (most recent call last):
    File "/Users/swyx/Work/datasette/env/bin/twitter-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/cli.py", line 291, in user_timeline
      profile = utils.get_profile(db, session, **kwargs)
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py", line 133, in get_profile
      save_users(db, [profile])
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py", line 453, in save_users
      transform_user(user)
    File "/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py", line 285, in transform_user
      user["created_at"] = parser.parse(user["created_at"])
  KeyError: 'created_at'
  ```

  </details>


  this looks awfully like #37  but it can't be, because i'm authed into my account and obviously i have perms to read my own account. wonder if there's any diagnostic methods i should apply here? just filing an issue for others to find while i investigate.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/62/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5A8ObT
title: when hashed urls are turned on, the _memory db has improperly long-lived cache expiry
state: closed
created_at: 2021-12-28T00:05:48Z
updated_at: 2022-03-24T04:08:18Z
closed_at: 2022-03-24T04:08:18Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  if hashed_urls are on, then a -000 suffix is added to the `_memory` database, and the cache settings are set just as if it was a normal hashed database.

  in particular, this header is set:

  `cache-control: max-age=31536000`

  this is not appropriate because the `_memory-000` database isn't really hashed based on the contents of the databases (see #1561).

  Either the cache-control header should be changed, or the _memory db should have a hash suffix that does depend on the contents of the databases.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1581/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4wWDxH
title: don't set far expiry if hash is '000'
state: closed
created_at: 2021-12-28T18:16:13Z
updated_at: 2022-03-24T04:07:58Z
closed_at: 2022-03-24T04:07:58Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1582
type: pull
body:
  This will close #1581.

  I couldn't find any unit tests related to the testing hashed urls, and I know that you want to break that code out of the core application (#1561), so I'm not quite sure what you would like me to for testing.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1582/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BBEKd
title: Use RETURNING if available to populate last_pk
state: open
created_at: 2021-12-29T23:43:23Z
updated_at: 2021-12-29T23:43:23Z
author_association: OWNER
type: issue
body:
  Inspired by this: https://news.ycombinator.com/item?id=29729283

  > Because SQLite is effectively serializing all the writes for us, we have zero locking in our code. We used to have to lock when inserting new items (to get the LastInsertRowId), but the newer version of SQLite supports the RETURNING keyword, so we don't even have to lock on inserts now.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/359/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BBHFU
title: consider adding deletion step of cloudbuild artifacts to gcloud publish
state: open
created_at: 2021-12-30T00:33:23Z
updated_at: 2021-12-30T00:34:16Z
author_association: CONTRIBUTOR
type: issue
body:
  right now, as part of the the publish process images and other artifacts are stored to gcloud's cloud storage before being deployed to cloudrun.

  after successfully deploying, it would be nice if the the script deleted these artifacts. otherwise, if you have regularly scheduled build process, you can end up paying to store lots of out of date artifacts.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1583/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BC0XE
title: give error with recursive sql
state: open
created_at: 2021-12-30T18:53:16Z
updated_at: 2021-12-30T18:53:16Z
author_association: NONE
type: issue
body:
  I got an error "near "WITH": syntax error" after I upgraded to version 0.59 from 0.52.4.    This error is related to recursive sql.  It works great on the previous version but it failed after upgraded.  Below is an example of sql:

  WITH RECURSIVE manager_of(position, super_position) AS (SELECT position, case ifnull(INDIRECT_SUPER_POSITION,'') when '' then super_position else INDIRECT_SUPER_POSITION end as SUPER_POSITION FROM position where super_position<>'SGV000000001' and super_position!=''  and position <> super_position),chain_manager_of_position(position, level) AS (SELECT super_position, 1 as level FROM manager_of WHERE super_position!='' and (position=:pos or position in (Select position from employee where employee=:ein)) UNION ALL SELECT super_position, level+1 as level FROM manager_of JOIN chain_manager_of_position USING(position)) SELECT * FROM chain_manager_of_position left join employee using(position)  where  employee is not NULL order by level limit 1

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1584/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BE9ZR
title: MemoryError
state: closed
created_at: 2022-01-01T13:39:17Z
updated_at: 2022-03-21T04:22:46Z
closed_at: 2022-03-21T04:22:46Z
author_association: NONE
type: issue
state_reason: completed
body:
  HI, when dealing with large json file (~170GB) i got the following error 
  ```
  Traceback (most recent call last):
    File "/usr/local/bin/sqlite-utils", line 8, in <module>
      sys.exit(cli())
    File "/usr/lib/python3/dist-packages/click/core.py", line 1126, in __call__
      return self.main(*args, **kwargs)
    File "/usr/lib/python3/dist-packages/click/core.py", line 1051, in main
      rv = self.invoke(ctx)
    File "/usr/lib/python3/dist-packages/click/core.py", line 1657, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/usr/lib/python3/dist-packages/click/core.py", line 1393, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/usr/lib/python3/dist-packages/click/core.py", line 752, in invoke
      return __callback(*args, **kwargs)
    File "/usr/local/lib/python3.9/dist-packages/sqlite_utils/cli.py", line 1300, in memory
      rows, format_used = rows_from_file(csv_fp, format=format, encoding=encoding)
    File "/usr/local/lib/python3.9/dist-packages/sqlite_utils/utils.py", line 185, in rows_from_file
      return rows_from_file(buffered, format=Format.JSON)
    File "/usr/local/lib/python3.9/dist-packages/sqlite_utils/utils.py", line 156, in rows_from_file
      decoded = json.load(fp)
    File "/usr/lib/python3.9/json/__init__.py", line 293, in load
      return loads(fp.read(),
  MemoryError
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/360/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BFCMW
title: Fire base caching for `publish cloudrun`
state: open
created_at: 2022-01-01T15:38:15Z
updated_at: 2022-01-01T15:40:38Z
author_association: OWNER
type: issue
body:
  https://gist.github.com/steren/03d3e58c58c9a53fd49bb78f58541872 has a recipe for this, via https://twitter.com/steren/status/1477038411114446848

  Could this enable easier vanity URLs of the format `https://$project_id.web.app/`? How about CDN caching?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1585/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODEm0Qs5BFFEi
title: Import archive error 'withheld_in_countries'
state: open
created_at: 2022-01-01T16:58:59Z
updated_at: 2022-01-01T16:58:59Z
author_association: NONE
type: issue
body:
  Importing the twitter archive  I received this error:
  ```bash
  $ twitter-to-sqlite import archive.db twitter-2021-12-31-<hash>.zip 
  birdwatch-note-rating: not yet implemented
  birdwatch-note: not yet implemented
  branch-links: not yet implemented
  community-tweet: not yet implemented
  contact: not yet implemented
  device-token: not yet implemented
  direct-message-mute: not yet implemented
  mute: not yet implemented
  periscope-account-information: not yet implemented
  periscope-ban-information: not yet implemented
  periscope-broadcast-metadata: not yet implemented
  periscope-comments-made-by-user: not yet implemented
  periscope-expired-broadcasts: not yet implemented
  periscope-followers: not yet implemented
  periscope-profile-description: not yet implemented
  professional-data: not yet implemented
  protected-history: not yet implemented
  reply-prompt: not yet implemented
  screen-name-change: not yet implemented
  smartblock: not yet implemented
  spaces-metadata: not yet implemented
  sso: not yet implemented
  Traceback (most recent call last):
    File "/home/paulox/.virtualenvs/dogsheep/bin/twitter-to-sqlite", line 8, in <module>
      sys.exit(cli())
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
      return self.main(*args, **kwargs)
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1053, in main
      rv = self.invoke(ctx)
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 754, in invoke
      return __callback(*args, **kwargs)
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/cli.py", line 759, in import_
      archive.import_from_file(db, filename, content)
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/archive.py", line 246, in import_from_file
      db[table_name].insert_all(rows, pk=pk, replace=True)
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2625, in insert_all
      self.insert_chunk(
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2406, in insert_chunk
      result = self.db.execute(query, params)
    File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 422, in execute
      return self.conn.execute(sql, parameters)
  sqlite3.OperationalError: table archive_tweet has no column named withheld_in_countries
  ```

  I found only a single tweet with the key `withheld_in_countries` in `tweet.js` that seems the problems:
  ```JSON
  [
  {
      "tweet" : {
        "retweeted" : false,
        "source" : "<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a>",
        "entities" : {
          "hashtags" : [
            {
              "text" : "NowOnAndroid",
              "indices" : [
                "64",
                "77"
              ]
            }
          ],
          "symbols" : [ ],
          "user_mentions" : [
            {
              "name" : "Periscope",
              "screen_name" : "PeriscopeCo",
              "indices" : [
                "3",
                "15"
              ],
              "id_str" : "1111111111",
              "id" : "222222222"
            }
          ],
          "urls" : [
            {
              "url" : "https://t.co/xxxxxxxxx",
              "expanded_url" : "https://vine.co/v/xxxxxxxxx",
              "display_url" : "vine.co/v/xxxxxxxxxx",
              "indices" : [
                "78",
                "101"
              ]
            }
          ]
        },
        "display_text_range" : [
          "0",
          "101"
        ],
        "favorite_count" : "0",
        "id_str" : "1111111111111111111111",
        "truncated" : false,
        "retweet_count" : "0",
        "withheld_in_countries" : [
          "TR"
        ],
        "id" : "000000000000000000",
        "possibly_sensitive" : false,
        "created_at" : "Fri Aug 14 06:04:03 +0000 2015",
        "favorited" : false,
        "full_text" : "RT @periscopeco: Travel the world. LIVE. The Global Map is here #NowOnAndroid https://t.co/NZXdsPWROk",
        "lang" : "en"
      }
    }
    ]
  ```

  I solved the error removing the key from the `tweet.js` but I'm reporting this error to improve the project.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/63/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4wlm3B
title: --lines and --text and --convert and --import
state: closed
created_at: 2022-01-06T01:49:44Z
updated_at: 2022-01-06T06:37:03Z
closed_at: 2022-01-06T06:24:54Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/361
type: pull
body:
  Refs #356

  Still TODO:

  - [x] Get `--lines` working, with tests
  - [x] Get `--text` working, with tests
  - [x] Get regular JSON import working with `--convert` with tests
  - [x] Get `--lines` working with `--convert` with tests
  - [x] Get `--text` working with `--convert` with tests
  - [x] Get `--csv` and `--tsv` import working with `--convert` with tests
  - [x] Get `--nl` working with `--convert` with tests
  - [x] Documentation for all of the above

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/361/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BQ_z5
title: upsert --detect-types is broken
state: closed
created_at: 2022-01-06T05:12:10Z
updated_at: 2022-01-06T06:54:45Z
closed_at: 2022-01-06T06:28:34Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Noticed this thanks to syntax highlighting in VS Code showing an unused variable - need to fix it and add a test.

  <img src="https://user-images.githubusercontent.com/9599/148331804-6295bcf5-01fd-4159-b8ad-60c82acb97b4.png" width="200">

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/362/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BRBbb
title: Better error message if `--convert` code fails to return a dict
state: closed
created_at: 2022-01-06T05:26:28Z
updated_at: 2022-02-03T22:52:30Z
closed_at: 2022-02-03T22:51:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Here's the traceback if your `--convert` function doesn't return a dict right now:
  ```
  % sqlite-utils insert /tmp/all.db blah /tmp/log.log --convert 'all.upper()' --all         

  Traceback (most recent call last):
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils", line 33, in <module>
      sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')())
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1137, in __call__
      return self.main(*args, **kwargs)
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1062, in main
      rv = self.invoke(ctx)
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1668, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 763, in invoke
      return __callback(*args, **kwargs)
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 949, in insert
      insert_upsert_implementation(
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 834, in insert_upsert_implementation
      db[table].insert_all(
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 2602, in insert_all
      first_record = next(records)
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 3044, in fix_square_braces
      for record in records:
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 831, in <genexpr>
      docs = (decode_base64_values(doc) for doc in docs)
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/utils.py", line 86, in decode_base64_values
      to_fix = [
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/utils.py", line 89, in <listcomp>
      if isinstance(doc[k], dict)
  TypeError: string indices must be integers
  ```
  It would be nicer if that returned a more useful error message.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/361#issuecomment-1006295276_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/363/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BTRKa
title: `--batch-size 1` doesn't seem to commit for every item
state: closed
created_at: 2022-01-06T18:18:50Z
updated_at: 2022-01-10T19:27:17Z
closed_at: 2022-01-10T05:36:19Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I'm trying this, but it doesn't seem to write anything to the database file until I hit `CTRL+C`:

  ```
  heroku logs --app=simonwillisonblog --tail | grep 'measure#nginx.service' | \
    sqlite-utils insert /tmp/herokutail.db log - --import re --convert "$(cat <<EOD
      r = re.compile(r'([^\s=]+)=(?:"(.*?)"|(\S+))')
      pairs = {}
      for key, value1, value2 in r.findall(line):
          pairs[key] = value1 or value2
      return pairs
  EOD
  )" --lines --batch-size 1
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/364/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BW9Cw
title: run analyze on all databases as part of start up or publishing
state: open
created_at: 2022-01-07T17:52:34Z
updated_at: 2022-02-02T07:13:37Z
author_association: CONTRIBUTOR
type: issue
body:
  Running `analyze;` lets sqlite's query planner make *much* better use of any indices.

  It might be nice if the analyze was run as part of the start up of "serve" or "publish".

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1586/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BXCbH
title: create-index should run analyze after creating index
state: closed
created_at: 2022-01-07T18:21:25Z
updated_at: 2022-01-11T02:43:34Z
closed_at: 2022-01-11T01:36:48Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  sqlite's query planner depends upon analyze to make good use of indices. It would be nice if analyze was run as part of the create-index command.

  If data is inserted later, things can get out date, but it would still probably be a net win. 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/365/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BXDpB
title: Python library methods for calling ANALYZE
state: closed
created_at: 2022-01-07T18:28:01Z
updated_at: 2022-01-11T01:09:33Z
closed_at: 2022-01-11T01:09:33Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Relevant documentation: https://www.sqlite.org/lang_analyze.html

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1007633376_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/366/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BY4Ir
title: Add `sqlite_stat1`(-4) tables to hidden table list
state: closed
created_at: 2022-01-08T21:28:20Z
updated_at: 2022-01-20T04:12:59Z
closed_at: 2022-01-20T04:12:59Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Running `ANALYZE` creates a new visible table called `sqlite_stat1`: https://www.sqlite.org/fileformat.html#the_sqlite_stat1_table
  >
  > This should be added to the default list of hidden tables in Datasette.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1587/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4wsVM6
title: Initial prototype of .analyze() methods
state: closed
created_at: 2022-01-08T21:35:12Z
updated_at: 2022-01-10T19:31:08Z
closed_at: 2022-01-10T19:31:08Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/367
body: Refs #366
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/367/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BZDkw
title: Offer `python -m sqlite_utils` as an alternative to `sqlite-utils`
state: closed
created_at: 2022-01-09T02:29:30Z
updated_at: 2022-01-10T19:27:20Z
closed_at: 2022-01-09T02:40:50Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Add this to `sqlite_utils/cli.py`:
  >
  > ```python
  > if __name__ == "__main__":
  >     cli()
  > ```
  > Now the tool can be run using `python -m sqlite_utils.cli --help`

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008214998_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/368/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BZEnH
title: Research how much of a difference analyze / sqlite_stat1 makes
state: closed
created_at: 2022-01-09T03:03:36Z
updated_at: 2022-02-03T21:07:41Z
closed_at: 2022-02-03T21:07:35Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > Is there a downside to having a `sqlite_stat1` table if it has wildly incorrect statistics in it?

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/365#issuecomment-1008163050_

  More generally: how much of a difference does the `sqlite_stat1` table created by `ANALYZE` make to queries?

  I'm particularly interested in `group by` / `count *` queries since Datasette uses those for faceting.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/369/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BZHJd
title: `explain query plan select` is too strict about whitespace
state: closed
created_at: 2022-01-09T04:22:42Z
updated_at: 2022-01-13T22:28:19Z
closed_at: 2022-01-13T20:35:05Z
author_association: OWNER
type: issue
state_reason: completed
body:
  `explain query plan select * from facetable` is allowed: https://latest.datasette.io/fixtures?sql=explain+query+plan+select+*+from+facetable

  But... `explain query plan  select * from facetable` (with two spaces before the `select`) returns a "Statement must be a SELECT" error: https://latest.datasette.io/fixtures?sql=explain+query+plan++select+*+from+facetable

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1588/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BZNmO
title: Support mutating row in `--convert` without returning it
state: closed
created_at: 2022-01-09T07:38:44Z
updated_at: 2022-01-10T19:27:30Z
closed_at: 2022-01-09T20:06:15Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Currently you have to do this:
  ```
  $ sqlite-utils insert dogs.db dogs dogs.json --convert '
  row["is_good"] = 1
  return row'
  ```
  Would be neat if this worked too:
  ```
  $ sqlite-utils insert dogs.db dogs dogs.json \
    --convert 'row["is_good"] = 1'
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/371/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BZN7u
title: Idea: `suffix` and `stem` file columns
state: closed
created_at: 2022-01-09T07:48:53Z
updated_at: 2022-01-10T19:27:34Z
closed_at: 2022-01-09T20:17:00Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-data-from-files

  Given a file called `dogs.jpg` stem would be `dogs` and ext would be `jpg`.

  Need to decide what happens for `dogs.and.cats.jpg.gz`.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/372/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BZPZ0
title: List `--fmt` options in the docs
state: closed
created_at: 2022-01-09T08:22:11Z
updated_at: 2022-01-10T19:27:24Z
closed_at: 2022-01-09T17:49:00Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://sqlite-utils.datasette.io/en/stable/cli.html#table-formatted-output currently cheats and tells the user to run `--help` - can fix this using `cog`. 

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/373/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BZPb0
title: `--fmt` should imply `-t`
state: closed
created_at: 2022-01-09T08:23:07Z
updated_at: 2022-01-10T19:27:26Z
closed_at: 2022-01-09T18:07:59Z
author_association: OWNER
body: Not sure why I didn't implement this.
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/374/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BZrjG
title: `sqlite-utils bulk` command
state: closed
created_at: 2022-01-09T17:12:38Z
updated_at: 2022-01-11T02:12:58Z
closed_at: 2022-01-11T02:10:55Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The `.executemany()` method is a very efficient way to execute the same SQL query against a huge list of parameters.

  `sqlite-utils insert` supports a bunch of ways of loading a list of dictionaries - from CSV, TSV, JSON, newline JSON and more thanks to:
  - #361

  What if you could load a list of dictionaries and provide a SQL query with `:named` parameters that correspond to keys in those dictionaries instead?

  This would need to be a new command - I thought about adding a `--sql` option to `insert` but that doesn't make sense as that command already requires a table name.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/375/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDODEm0Qs5BZ_WC
title: Include all entities for tweets
state: open
created_at: 2022-01-09T23:35:28Z
updated_at: 2022-01-09T23:35:28Z
author_association: NONE
type: issue
body:
  Per our conversation [on Twitter](https://twitter.com/mschoening/status/1480312477246054401):

  It would be neat if all entities (including URLs) were captured. This way you can ensure, that URLs are parsed out exactly the same way Twitter parses URLs – we all know parsing URLs with a regex ain't fun.

  Right now, I believe the tool filters out all entities that are not of type `media`.

reactions:
  {"url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/64/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BaY8f
title: `--nl` mode should ignore blank lines
state: closed
created_at: 2022-01-10T04:10:54Z
updated_at: 2022-01-10T19:27:41Z
closed_at: 2022-01-10T04:12:46Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Spotted this while manually testing #364 - there's no reason `--nl` should crash if you feed it an empty line in between JSON objects.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/376/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4wtl17
title: `sqlite-utils bulk` command
state: closed
created_at: 2022-01-10T05:34:24Z
updated_at: 2022-01-11T02:10:57Z
closed_at: 2022-01-11T02:10:54Z
author_association: OWNER
pull_request: simonw/sqlite-utils/pulls/377
type: pull
body:
  Refs #375

  Still needs:

  - [x] Refactor `@insert_upsert_options` so that it doesn't duplicate `@import_options`
  - [x] Tests
  - [x] Documentation
  - [x] Try it against a really big file

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/377/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4wwNCl
title: Typo in docs about default redirect status code
state: closed
created_at: 2022-01-10T19:14:36Z
updated_at: 2022-03-06T02:27:49Z
closed_at: 2022-03-06T01:58:32Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1589
type: pull
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1589/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BduEJ
title: analyze=True parameter for some methods
state: closed
created_at: 2022-01-10T19:54:52Z
updated_at: 2022-01-11T01:08:11Z
closed_at: 2022-01-11T01:08:09Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This would cause `ANALYZE` to be run against the relevant table at the end of executing the method.

  > Having browsed the API reference I think the methods that would benefit from an `analyze=True` parameter are:

  - [x] `table.create_index`
  - [x] `table.insert_all`
  - [x] `table.upsert_all`
  - [x] `table.delete_where`

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009288898_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/378/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BenX0
title: CLI options for running ANALYZE
state: closed
created_at: 2022-01-11T01:09:16Z
updated_at: 2022-01-11T01:38:01Z
closed_at: 2022-01-11T01:36:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  > The Python methods are all done now, next step is the CLI options. I'll do those in a separate issue.

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009508865_

  - [x] `sqlite-utils analyze` command
  - [x] `sqlite-utils create-index --analyze` option (see #365)
  - [x] `sqlite-utils insert --analyze` option
  - [x] `sqlite-utils upsert --analyze` option

  In #378 I also added `.delete_where(..., analyze=True)` but there isn't currently a `sqlite-utils delete-where` CLI command - deletions via CLI are expected to be handled using SQL queries.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/379/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Beurs
title: Release notes for 3.21
state: closed
created_at: 2022-01-11T02:12:30Z
updated_at: 2022-01-11T02:34:26Z
closed_at: 2022-01-11T02:34:26Z
author_association: OWNER
type: issue
state_reason: completed
body:
  For these commits: https://github.com/simonw/sqlite-utils/compare/3.20...129141572f249ea290e2a075437e2ebaad215859

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/380/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BilSt
title: `sqlite-utils rows` options `--limit` and `--offset`
state: closed
created_at: 2022-01-11T20:23:12Z
updated_at: 2022-01-11T23:33:37Z
closed_at: 2022-01-11T23:19:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Because I often want to use it just to preview a few rows from the database. Piping through `| head -n 20` works for JSON and CSV (they stream) but not for `--table`.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/381/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5BilhL
title: `--where` option for `sqlite-rows`
state: closed
created_at: 2022-01-11T20:24:23Z
updated_at: 2022-01-11T23:33:14Z
closed_at: 2022-01-11T23:32:47Z
author_association: OWNER
type: issue
state_reason: completed
body:
  CLI equivalent of `table.rows_where()` - should accept parameters too. Work on this at the same time as #381.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/382/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Bilzi
title: Add documentation page with the output of `--help`
state: closed
created_at: 2022-01-11T20:25:58Z
updated_at: 2022-01-11T22:55:05Z
closed_at: 2022-01-11T21:44:05Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Can be maintained using `cog` from #373. Similar in purpose to the API reference page, but this is for the CLI.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/383/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BjHSM
title: Table+query JSON and CSV links broken when using `base_url` setting
state: closed
created_at: 2022-01-11T23:46:39Z
updated_at: 2022-01-14T01:16:34Z
closed_at: 2022-01-14T01:16:08Z
author_association: NONE
type: issue
state_reason: completed
body:
  Datasette appends the prefix found in the `base_url` setting twice if a `base_url` is set.

  In the follow asgi example, I'm hosting a custom Datasette instance:

  ```python
  # asgi.py
  import pathlib
  from asgi_cors import asgi_cors
  from channels.routing import URLRouter
  from django.urls import re_path
  from datasette.app import Datasette

  datasette_ = Datasette(
      files=[],
      settings={
          "base_url": "/datasettes/",
          "plugins": {}
      },
      config_dir=pathlib.Path('.'),
  )
  application = URLRouter([
    re_path(r"^datasettes/.*", asgi_cors(datasette_.app(), allow_all=True)),
  ])
  ```

  Running it with:
  ```shell
  $ daphne -p 8002 asgi:application
  ```

  Using a simple query on the `_memory` table: 
  ```sql
  select sqlite_version()
  ```

  http://localhost:8002/datasettes/_memory?sql=select+sqlite_version%28%29

  It renders the following upon inspection:
  ![image](https://user-images.githubusercontent.com/1001306/149038851-aa842950-126a-467c-9a86-fae13bce6221.png)

  I am using datasette version `0.59.4`

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1590/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Bjxsw
title: Add examples to every `--help`
state: closed
created_at: 2022-01-12T05:31:25Z
updated_at: 2022-01-26T03:15:02Z
closed_at: 2022-01-26T03:15:02Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Everything on https://sqlite-utils.datasette.io/en/stable/cli-reference.html would benefit from an example.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/384/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BkOcm
title: Maybe let plugins define custom serve options?
state: open
created_at: 2022-01-12T08:18:47Z
updated_at: 2022-01-15T11:56:59Z
author_association: OWNER
type: issue
body:
  https://twitter.com/psychemedia/status/1481171650934714370

  > can extensions be passed their own cli args? eg `--ext-tiddlywiki-dbname tiddlywiki2.sqlite` ?

  I've thought something like this might be useful for other plugins in the past, too.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1591/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BmEqj
title: Row pages should show links to foreign keys
state: open
created_at: 2022-01-12T15:50:20Z
updated_at: 2022-01-12T15:52:17Z
author_association: OWNER
body: Refs #1518 refactor.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1592/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4w7eqc
title: Update pytest-asyncio requirement from <0.17,>=0.10 to >=0.10,<0.18
state: closed
created_at: 2022-01-13T13:11:50Z
updated_at: 2022-02-07T13:13:24Z
closed_at: 2022-02-07T13:13:23Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1593
type: pull
body:
  Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
  <details>
  <summary>Release notes</summary>
  <p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-asyncio/releases">pytest-asyncio's releases</a>.</em></p>
  <blockquote>
  <h2>pytest-asyncio 0.17.0</h2>
  <hr />
  <h2>title: 'pytest-asyncio: pytest support for asyncio'</h2>
  <p><a href="https://pypi.python.org/pypi/pytest-asyncio"><img src="https://img.shields.io/pypi/v/pytest-asyncio.svg" alt="image" /></a></p>
  <p><a href="https://github.com/pytest-dev/pytest-asyncio/actions?workflow=CI"><img src="https://github.com/pytest-dev/pytest-asyncio/workflows/CI/badge.svg" alt="image" /></a></p>
  <p><a href="https://codecov.io/gh/pytest-dev/pytest-asyncio"><img src="https://codecov.io/gh/pytest-dev/pytest-asyncio/branch/master/graph/badge.svg" alt="image" /></a></p>
  <p><a href="https://github.com/pytest-dev/pytest-asyncio"><img src="https://img.shields.io/pypi/pyversions/pytest-asyncio.svg" alt="Supported Python versions" /></a></p>
  <p><a href="https://github.com/ambv/black"><img src="https://img.shields.io/badge/code%20style-black-000000.svg" alt="image" /></a></p>
  <p>pytest-asyncio is an Apache2 licensed library, written in Python, for
  testing asyncio code with pytest.</p>
  <p>asyncio code is usually written in the form of coroutines, which makes
  it slightly more difficult to test using normal testing tools.
  pytest-asyncio provides useful fixtures and markers to make testing
  easier.</p>
  <pre lang="{.sourceCode" data-meta=".python}"><code>@pytest.mark.asyncio
  async def test_some_asyncio_code():
      res = await library.do_something()
      assert b&quot;expected result&quot; == res
  </code></pre>
  <p>pytest-asyncio has been strongly influenced by
  <a href="https://github.com/eugeniy/pytest-tornado">pytest-tornado</a>.</p>
  <h1>Features</h1>
  <ul>
  <li>fixtures for creating and injecting versions of the asyncio event
  loop</li>
  <li>fixtures for injecting unused tcp/udp ports</li>
  <li>pytest markers for treating tests as asyncio coroutines</li>
  <li>easy testing with non-default event loops</li>
  <li>support for [async def]{.title-ref} fixtures and async generator
  fixtures</li>
  <li>support <em>auto</em> mode to handle all async fixtures and tests
  automatically by asyncio; provide <em>strict</em> mode if a test suite
  should work with different async frameworks simultaneously, e.g.
  <code>asyncio</code> and <code>trio</code>.</li>
  </ul>
  <h1>Installation</h1>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/2e2d5d202ecfea6fd403d92a9dd6ab6734aa0f85"><code>2e2d5d2</code></a> Bump to 0.17 release</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/90436c98dd27e670f0c721b48bb28001ccdcbbda"><code>90436c9</code></a> Fix pandoc installation procedure</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/d291c666870d2d903fc99543dccd77dca8496d5b"><code>d291c66</code></a> Convert README.rst to Markdown for making githun release</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/141937b89aa6ced9856ed3f997818ef8e3fbea57"><code>141937b</code></a> Fix release artifacts</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/696cf7d5e0a458825c27ae8c7d621fb538c70827"><code>696cf7d</code></a> Fix trove classifier for asyncio</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/8ccb30650068f58d6f3e7314d2c1f2d59cdba1e9"><code>8ccb306</code></a> Build on tag</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/cd8498709d5cf763c7a73cac9b43a0bd2a2d4fb7"><code>cd84987</code></a> Release process automation (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/252">#252</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/2eb12a7b0591dfb8578303235d87bb25ddeedf77"><code>2eb12a7</code></a> Setup GitHub Workflows linter and yaml-reformatter (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/253">#253</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/d28b826b8acb329401cceed286ef3b42dc82df05"><code>d28b826</code></a> Bump codecov/codecov-action from 1 to 2.1.0 (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/251">#251</a>)</li>
  <li><a href="https://github.com/pytest-dev/pytest-asyncio/commit/2f523bad98ee1256c26ffe94c78ebcd8a1d03688"><code>2f523ba</code></a> Configure dependabot version updater (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-asyncio/issues/250">#250</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-asyncio/compare/v0.10.0...v0.17.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1593/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BtKyu
title: Add a CLI reference page to the docs, inspired by sqlite-utils
state: closed
created_at: 2022-01-13T20:55:08Z
updated_at: 2022-01-13T22:28:22Z
closed_at: 2022-01-13T21:38:48Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Thought of this while posting this comment: https://github.com/simonw/datasette/issues/1591#issuecomment-1012506595

  I added https://sqlite-utils.datasette.io/en/stable/cli-reference.html to `sqlite-utils` in https://github.com/simonw/sqlite-utils/issues/383 and I _really_ like it - it's a page showing the `--help` output of every CLI command for that tool.

  It's maintained using `cog`. One of the benefits is that I get a free commit history of changes to `--help` at https://github.com/simonw/sqlite-utils/commits/main/docs/cli-reference.rst

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1594/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BtpKe
title: Release notes for 0.60
state: closed
created_at: 2022-01-13T22:23:14Z
updated_at: 2022-01-14T01:37:39Z
closed_at: 2022-01-14T01:37:39Z
author_association: OWNER
type: issue
state_reason: completed
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1595/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Bt9pv
title: Documentation page warning of changes coming in 1.0
state: open
created_at: 2022-01-13T23:26:04Z
updated_at: 2022-01-13T23:26:04Z
author_association: OWNER
body: I should start this relatively soon.
type: issue
reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1596/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BuIm6
title: "datasette inspect" has no help summary
state: closed
created_at: 2022-01-14T00:02:16Z
updated_at: 2022-01-14T00:07:36Z
closed_at: 2022-01-14T00:07:36Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Made obvious by the new CLI reference page added in #1594. https://docs.datasette.io/en/latest/cli-reference.html#datasette-inspect-help
  ```
  Commands:
    serve*     Serve up specified SQLite database files with a web UI
    inspect
    install    Install Python packages - e.g.
  ```
  ```
  Usage: datasette inspect [OPTIONS] [FILES]...

  Options:
    --inspect-file TEXT
    --load-extension TEXT  Path to a SQLite extension to load
    --help                 Show this message and exit.
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1597/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5BuOkn
title: Replace update-docs-help.py script with cog
state: closed
created_at: 2022-01-14T00:33:27Z
updated_at: 2022-01-14T00:47:57Z
closed_at: 2022-01-14T00:47:57Z
author_association: OWNER
type: issue
state_reason: completed
body:
  I introduced `cog` in #1594 - I can use this to replace the older `update-docs-help.py` mechanism:

  https://github.com/simonw/datasette/blob/76d66d5b2bf10249c0beaac0999b93ac8d757f48/tests/test_docs.py#L36-L53

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1598/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOCGYnMM4w_p22
title: Add new spatialite helper methods
state: closed
created_at: 2022-01-14T03:57:30Z
updated_at: 2022-02-05T00:04:26Z
closed_at: 2022-02-04T05:55:10Z
author_association: CONTRIBUTOR
pull_request: simonw/sqlite-utils/pulls/385
type: pull
body:
  Refs #79 

  This PR adds three new Spatialite-related methods to Database and Table:

  - `Database.init_spatialite` loads the Spatialite extension and initializes it
  - `Table.add_geometry_column` adds a geometry column
  - `Table.create_spatial_index` creates a spatial index

  Has tests and documentation. Feedback very welcome.

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/385/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5Bve5q
title: Add architecture documentation
state: open
created_at: 2022-01-14T04:55:38Z
updated_at: 2022-01-14T04:56:03Z
author_association: OWNER
type: issue
body:
  Inspired by https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html

  Good example: https://github.com/rust-analyzer/rust-analyzer/blob/d7c99931d05e3723d878bea5dc26766791fa4e69/docs/dev/architecture.md

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1599/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5B2EHO
title: plugins --all example should use cog
state: closed
created_at: 2022-01-15T11:47:49Z
updated_at: 2022-01-20T05:06:21Z
closed_at: 2022-01-20T05:04:16Z
author_association: OWNER
type: issue
state_reason: completed
body:
  The example output for `datasette plugins --all`on this page has got out of date: https://docs.datasette.io/en/stable/plugins.html#seeing-what-plugins-are-installed

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1600/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5B6vCd
title: Add KNN and data_licenses to hidden tables list
state: closed
created_at: 2022-01-17T14:19:57Z
updated_at: 2022-01-20T21:29:44Z
closed_at: 2022-01-20T04:38:54Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  They're generated by Spatialite and not very interesting in most cases.

  <img width="626" alt="Screen Shot 2022-01-17 at 9 17 31 AM" src="https://user-images.githubusercontent.com/25778/149786443-7832363c-2a2c-4533-9edc-048cd00e4a08.png">

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1601/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5CA_3H
title: Better "contributing" documentation
state: closed
created_at: 2022-01-19T02:11:48Z
updated_at: 2022-01-19T02:15:21Z
closed_at: 2022-01-19T02:15:21Z
author_association: OWNER
type: issue
state_reason: completed
body:
  This page jumps straight into running the tests: https://sqlite-utils.datasette.io/en/latest/contributing.html

  It should add a little more about expected collaboration styles - opening an issue before filing a pull request - and probably link to https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/386/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4xQ0uZ
title: Update pytest-timeout requirement from <2.1,>=1.4.2 to >=1.4.2,<2.2
state: closed
created_at: 2022-01-19T13:11:50Z
updated_at: 2022-03-06T01:41:50Z
closed_at: 2022-03-06T01:41:49Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1602
type: pull
body:
  Updates the requirements on [pytest-timeout](https://github.com/pytest-dev/pytest-timeout) to permit the latest version.
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/8e4800ecbcf149790bb8035ec37dc30f3d48a127"><code>8e4800e</code></a> Fixup readme</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/dc1efca167ef1d466ab805595528e6c64996cdeb"><code>dc1efca</code></a> 2.1.0 release</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/dd9d6080763c559de1ff5b15deb9c27559dd3431"><code>dd9d608</code></a> Add custom hooks specifications for overriding setup_timeout and teardown_tim...</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/ed8ecd6cbb8b8acccd57f6782dab94dcde0f83ea"><code>ed8ecd6</code></a> module names, they're difficult</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/3ab4319c5170bc9b9d8052019103c93c2ed74c86"><code>3ab4319</code></a> Add changelog</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/4f7ebae5789902b3097dbe618b109b8cafa3fd64"><code>4f7ebae</code></a> Replace deprecated py.io.get_terminal_width() with shutil.get_terminal_size()...</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/b8a2fa6e56330ca6510c50b59ba8ee75ec962822"><code>b8a2fa6</code></a> Prep release</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/951972da1c11ea5ca8ac76691f7c78e5deb1faaf"><code>951972d</code></a> Update changelog</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/748a9c37466b7381626a5f4152ec5dc9d2cc3907"><code>748a9c3</code></a> Making detection of whether a debugger is currently attached more flexible. (...</li>
  <li><a href="https://github.com/pytest-dev/pytest-timeout/commit/f8a46a194c89f1459fd97d5a8fa618e8326a2bfa"><code>f8a46a1</code></a> Github removed the git protocol (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-timeout/issues/112">#112</a>)</li>
  <li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-timeout/compare/1.4.2...2.1.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1602/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CDlWu
title: A proper favicon
state: closed
created_at: 2022-01-19T15:24:55Z
updated_at: 2022-03-19T04:04:49Z
closed_at: 2022-01-20T06:07:31Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Tips here: https://adamj.eu/tech/2022/01/18/how-to-add-a-favicon-to-your-django-site/ - I think a PNG served at `/favicon.ico` is the best option, since safari doesn't support SVG yet.

  Relevant code: https://github.com/simonw/datasette/blob/cb29119db9115b1f40de2fb45263ed77e3bfbb3e/datasette/app.py#L182-L183

  I can reuse the icon for https://datasette.io/desktop

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1603/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CD1ON
title: Option to assign a domain/subdomain using `datasette publish cloudrun`
state: open
created_at: 2022-01-19T16:21:17Z
updated_at: 2022-01-19T16:23:54Z
author_association: OWNER
type: issue
body:
  Looks like this API should be able to do that: https://twitter.com/steren/status/1483835859191304192 - https://cloud.google.com/run/docs/reference/rest/v1/namespaces.domainmappings/create

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1604/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CFP3Q
title: Scripted exports
state: open
created_at: 2022-01-19T23:45:55Z
updated_at: 2022-11-30T15:06:38Z
author_association: CONTRIBUTOR
type: issue
body:
  Posting this while I'm thinking about it: I mentioned at the end of [this thread](https://twitter.com/eyeseast/status/1483893011658551299) that I'm usually doing `datasette --get` to export canned queries.

  I used to use a tool called [datafreeze](https://github.com/pudo/datafreeze) to do scripted exports, but that project looks dead now. The ergonomics of it are pretty nice, though, and the `Freezefile.yml` structure is actually not too far from Datasette's canned queries.

  This is related to the idea for `datasette query` (#1356) but I think it's a distinct feature. It's most likely a plugin, but I want to raise it here because it's probably something other people have thought about.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1605/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CF6Xz
title: Tests failing against Python 3.6
state: closed
created_at: 2022-01-20T04:22:44Z
updated_at: 2022-01-20T04:36:42Z
closed_at: 2022-01-20T04:36:42Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://github.com/simonw/datasette/runs/4877484366

  ```
  E           File "/opt/hostedtoolcache/Python/3.6.15/x64/lib/python3.6/site-packages/uvicorn/server.py", line 67, in run
  E             return asyncio.run(self.serve(sockets=sockets))
  E         AttributeError: module 'asyncio' has no attribute 'run'
  ```
  I think this may mean `uvicorn` has dropped support for Python 3.6.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1606/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CJfH2
title: More detailed information about installed SpatiaLite version
state: closed
created_at: 2022-01-20T21:28:03Z
updated_at: 2022-02-09T06:42:02Z
closed_at: 2022-02-09T06:32:28Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.0.0.html#version has a whole bunch of interesting functions for things like `freexl_version()` and `geos_version()` and `HasMathSQL()` and suchlike.

  These could be shown on the `/-/versions` page.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1607/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CJlQa
title: Documentation should clarify /stable/ vs /latest/
state: closed
created_at: 2022-01-20T22:02:59Z
updated_at: 2023-03-26T23:41:12Z
closed_at: 2022-01-20T22:53:17Z
author_association: OWNER
type: issue
state_reason: completed
body:
  It's not currently clear what the difference between https://docs.datasette.io/en/latest/ and https://docs.datasette.io/en/stable/ is - I should fix that.

  On Twitter: https://twitter.com/simonw/status/1484285006243528705

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1608/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CJ38w
title: Ensure "pip install datasette" still works with Python 3.6
state: closed
created_at: 2022-01-21T00:08:10Z
updated_at: 2022-01-24T19:20:09Z
closed_at: 2022-01-21T02:24:13Z
author_association: OWNER
type: issue
state_reason: completed
body:
  ## Original title: Can I keep "pip install datasette" working on Python 3.6?

  I dropped support for 3.6 in:
  - #1577

  I'm getting reports that `pip3 install datasette` throws an error on that Python, even though I haven't made that new release yet - presumably due to lack of pinning of Uvicorn: https://twitter.com/ldodds/status/1484289475195080706

  Is it possible to get `pip` on that version of Python to install the highest possible version of the packages that are still known to support Python 3.6?

  If so, how?

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1609/reactions", "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5CPPx6
title: Python library docs should start with a self contained example
state: closed
created_at: 2022-01-22T06:23:56Z
updated_at: 2022-01-26T01:37:17Z
closed_at: 2022-01-26T01:35:30Z
author_association: OWNER
type: issue
state_reason: completed
body:
  You have to read a lot of stuff in a lot of different places to get started with the Python library. Add a getting started introduction to https://sqlite-utils.datasette.io/en/stable/python-api.html

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/387/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: PR_kwDOBm6k_c4xfryi
title: Update asgiref requirement from <3.5.0,>=3.2.10 to >=3.2.10,<3.6.0
state: closed
created_at: 2022-01-24T13:14:18Z
updated_at: 2022-03-06T01:30:27Z
closed_at: 2022-03-06T01:30:27Z
author_association: CONTRIBUTOR
pull_request: simonw/datasette/pulls/1610
type: pull
body:
  Updates the requirements on [asgiref](https://github.com/django/asgiref) to permit the latest version.
  <details>
  <summary>Changelog</summary>
  <p><em>Sourced from <a href="https://github.com/django/asgiref/blob/main/CHANGELOG.txt">asgiref's changelog</a>.</em></p>
  <blockquote>
  <h2>3.5.0 (2022-01-22)</h2>
  <ul>
  <li>
  <p>Python 3.6 is no longer supported, and asyncio calls have been changed to
  use only the modern versions of the APIs as a result</p>
  </li>
  <li>
  <p>Several causes of RuntimeErrors in cases where an event loop was assigned
  to a thread but not running</p>
  </li>
  <li>
  <p>Speed improvements in the Local class</p>
  </li>
  </ul>
  <h2>3.4.1 (2021-07-01)</h2>
  <ul>
  <li>Fixed an issue with the deadlock detection where it had false positives
  during exception handling.</li>
  </ul>
  <h2>3.4.0 (2021-06-27)</h2>
  <ul>
  <li>
  <p>Calling sync_to_async directly from inside itself (which causes a deadlock
  when in the default, thread-sensitive mode) now has deadlock detection.</p>
  </li>
  <li>
  <p>asyncio usage has been updated to use the new versions of get_event_loop,
  ensure_future, wait and gather, avoiding deprecation warnings in Python 3.10.
  Python 3.6 installs continue to use the old versions; this is only for 3.7+</p>
  </li>
  <li>
  <p>sync_to_async and async_to_sync now have improved type hints that pass
  through the underlying function type correctly.</p>
  </li>
  <li>
  <p>All Websocket* types are now spelled WebSocket, to match our specs and the
  official spelling. The old names will work until release 3.5.0, but will
  raise deprecation warnings.</p>
  </li>
  <li>
  <p>The typing for WebSocketScope and HTTPScope's <code>extensions</code> key has been
  fixed.</p>
  </li>
  </ul>
  <h2>3.3.4 (2021-04-06)</h2>
  <ul>
  <li>The async_to_sync type error is now a warning due the high false negative
  rate when trying to detect coroutine-returning callables in Python.</li>
  </ul>
  <h2>3.3.3 (2021-04-06)</h2>
  <!-- raw HTML omitted -->
  </blockquote>
  <p>... (truncated)</p>
  </details>
  <details>
  <summary>Commits</summary>
  <ul>
  <li><a href="https://github.com/django/asgiref/commit/8b615134bd481c4262356ad3194fbeff9240d0d3"><code>8b61513</code></a> Releasing 3.5.0</li>
  <li><a href="https://github.com/django/asgiref/commit/b2e1c9dfdef212e69fc00adc8c83ffbc6fbac73c"><code>b2e1c9d</code></a> Fixed pytest_asyncio deprecation warning.</li>
  <li><a href="https://github.com/django/asgiref/commit/2eda55127a82618f83bb54788fecd239babb86ce"><code>2eda551</code></a> Added testing for Python 3.10.</li>
  <li><a href="https://github.com/django/asgiref/commit/02fecb6046bb5ec0dbbad00ffcd2043e893fcea5"><code>02fecb6</code></a> Drop Python 3.6 (<a href="https://github-redirect.dependabot.com/django/asgiref/issues/307">#307</a>)</li>
  <li><a href="https://github.com/django/asgiref/commit/6689c0a1e820a5c32532f069e945da8230d808ad"><code>6689c0a</code></a> Added stacklevel to warning in AsyncToSync.</li>
  <li><a href="https://github.com/django/asgiref/commit/4364f9b4bb052d9ec54b806867fd859b07c39186"><code>4364f9b</code></a> Changed how StatelessServer handles event loops</li>
  <li><a href="https://github.com/django/asgiref/commit/7bc055c0883db13ca8622e11ab116d0bd180acaf"><code>7bc055c</code></a> Update implementations.rst (<a href="https://github-redirect.dependabot.com/django/asgiref/issues/295">#295</a>)</li>
  <li><a href="https://github.com/django/asgiref/commit/c758984f4ab4864ac5787ef90d03b3589b2bcb90"><code>c758984</code></a> Move current_task import choice to module definition time</li>
  <li><a href="https://github.com/django/asgiref/commit/dfe87b2633d111f4f67c2cdb29a8c1ebe2eb8dae"><code>dfe87b2</code></a> Fixed <a href="https://github-redirect.dependabot.com/django/asgiref/issues/292">#292</a>: Use get_event_loop in class-level code</li>
  <li><a href="https://github.com/django/asgiref/commit/b3a65e34e9f445a1ccc234282deedafe4b8b5de2"><code>b3a65e3</code></a> Removed class variable which has been unused since a0bbe90</li>
  <li>Additional commits viewable in <a href="https://github.com/django/asgiref/compare/3.2.10...3.5.0">compare view</a></li>
  </ul>
  </details>
  <br />


  Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

  [//]: # (dependabot-automerge-start)
  [//]: # (dependabot-automerge-end)

  ---

  <details>
  <summary>Dependabot commands and options</summary>
  <br />

  You can trigger Dependabot actions by commenting on this PR:
  - `@dependabot rebase` will rebase this PR
  - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
  - `@dependabot merge` will merge this PR after your CI passes on it
  - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
  - `@dependabot cancel merge` will cancel a previously requested merge and block automerging
  - `@dependabot reopen` will reopen this PR if it is closed
  - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


  </details>

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1610/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CXOW_
title: Avoid ever running count(*) against SpatiaLite KNN table
state: open
created_at: 2022-01-25T03:32:54Z
updated_at: 2022-02-02T06:45:47Z
author_association: OWNER
type: issue
body:
  Got this in a trace:

  <img width="941" alt="image" src="https://user-images.githubusercontent.com/9599/150906011-5f09dc6d-4def-433a-8546-e4fd94e0edf0.png">

  Looks like running `count(*)` against KNN took 83s! It ignored the time limit. And still only returned a count of 0.

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1611/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOBm6k_c5CaIxB
title: Move canned queries closer to the SQL input area
state: closed
created_at: 2022-01-25T17:06:39Z
updated_at: 2022-03-19T04:04:49Z
closed_at: 2022-01-25T18:34:21Z
author_association: CONTRIBUTOR
type: issue
state_reason: completed
body:
  *Original title:  Consider placing example queries above the sql input?*

  Hi! Have been enjoying deploying ad hoc datasettes for collaborators to pick over! 

  I keep finding myself manually "fixing" the database.html template so that the "example queries" (canned queries) appear directly *over* the sql box? So they are sorta more a suggestion for collaborators who aren't inclined to write their own queries? 

  My sense is any time I go to the trouble of writing canned queries my users should see 'em? 

  (( I have also considered a client-side reactive-ish option where selecting a query just places the raw SQL in the box and doesn't execute it, but this seems to end up being an inconvenience, rather than a teaching tool. )) 

reactions:
  {"url": "https://api.github.com/repos/simonw/datasette/issues/1612/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5CbpVz
title: Link to stable docs from older versions
state: closed
created_at: 2022-01-26T01:55:46Z
updated_at: 2023-03-26T23:43:12Z
closed_at: 2022-01-26T02:00:22Z
author_association: OWNER
type: issue
state_reason: completed
body:
  https://sqlite-utils.datasette.io/en/2.14.1/ isn't showing a link to the stable release right now.

  I should also apply the same fix I used for Datasette in:
  - https://github.com/simonw/datasette/issues/1608

  TIL: https://til.simonwillison.net/readthedocs/link-from-latest-to-stable

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/388/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5CbppX
title: Plausible analytics for documentation
state: closed
created_at: 2022-01-26T01:58:35Z
updated_at: 2022-01-26T02:07:41Z
closed_at: 2022-01-26T02:07:41Z
author_association: OWNER
type: issue
state_reason: completed
body:
  ```html
  <script defer data-domain="sqlite-utils.datasette.io" src="https://plausible.io/js/plausible.js"></script>
  ```

  _Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/388#issuecomment-1021785268_

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/389/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


node_id: I_kwDOCGYnMM5Cbstk
title: `sqlite-utils upsert` should require `--pk` more elegantly
state: closed
created_at: 2022-01-26T02:20:31Z
updated_at: 2022-01-26T03:20:25Z
closed_at: 2022-01-26T03:19:43Z
author_association: OWNER
type: issue
state_reason: completed
body:
  Currently throws an ugly traceback:
  ```
  % echo '[
              {"id": 1, "name": "Lila"},
              {"id": 1, "name": "Lila"}
          ]' | sqlite-utils upsert data.db chickens -       
  Traceback (most recent call last):
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/bin/sqlite-utils", line 33, in <module>
      sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')())
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1137, in __call__
      return self.main(*args, **kwargs)
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1062, in main
      rv = self.invoke(ctx)
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1668, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/Users/simon/.local/share/virtualenvs/sqlite-utils-C4Ilevlm/lib/python3.8/site-packages/click/core.py", line 763, in invoke
      return __callback(*args, **kwargs)
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 1104, in upsert
      insert_upsert_implementation(
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py", line 906, in insert_upsert_implementation
      db[table].insert_all(
    File "/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/db.py", line 2615, in insert_all
      raise PrimaryKeyRequired("upsert() requires a pk")
  sqlite_utils.db.PrimaryKeyRequired: upsert() requires a pk
  ```

reactions:
  {"url": "https://api.github.com/repos/simonw/sqlite-utils/issues/390/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0}


