issues

Table actions

320 rows where comments = 0 and state = "closed" sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: milestone, author_association, created_at (date), updated_at (date), closed_at (date)

type 2

state 1

  • closed · 320 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1553425465 I_kwDOCGYnMM5cl2Q5 522 Add COLUMN_TYPE_MAPPING for timedelta maport 81377 closed 0     0 2023-01-23T16:49:54Z 2023-11-04T00:49:51Z 2023-11-04T00:49:51Z NONE  

Currently trying to create a column with Python type datetime.timedelta results in an error:

```

from sqlite_utils import Database db = Database("test.db") test_tbl = db['test'] test_tbl.insert({'col1': datetime.timedelta()}) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 2979, in insert return self.insert_all( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 3082, in insert_all self.create( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 1574, in create self.db.create_table( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 961, in create_table sql = self.create_table_sql( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 852, in create_table_sql column_type=COLUMN_TYPE_MAPPING[column_type], KeyError: <class 'datetime.timedelta'> ```

The reason this would be useful is that MySQLdb uses timedelta for MySQL TIME columns:

```

import MySQLdb conn = MySQLdb.connect(host='database', user='user', passwd='pw') csr = conn.cursor() csr.execute("SELECT CAST('11:20' AS TIME)") tuple(csr) ((datetime.timedelta(seconds=40800),),) ```

So currently any attempt to convert a MySQL DB with a TIME column using db-to-sqlite will result in the above error.

I was rather surprised that MySQLdb uses timedelta for TIME columns but I see that this column type is intended for time intervals as well as the time of day so it makes sense.

sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/522/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1886649402 I_kwDOBm6k_c5wc_w6 2179 Flaky test: test_hidden_sqlite_stat1_table simonw 9599 closed 0     0 2023-09-07T22:48:43Z 2023-09-07T22:51:19Z 2023-09-07T22:51:19Z OWNER  

This test here: https://github.com/simonw/datasette/blob/fbcb103c0cb6668018ace539a01a6a1f156e8d6a/tests/test_api.py#L1011-L1020

It failed for me like this:

E AssertionError: assert [('normal', False), ('sqlite_stat1', True), ('sqlite_stat4', True)] in ([('normal', False), ('sqlite_stat1', True)],)

Looks like some builds of SQLite include a sqlite_stat4 table.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2179/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1884333600 PR_kwDOBm6k_c5Zszqk 2175 Test against Python 3.12 preview simonw 9599 closed 0     0 2023-09-06T16:09:05Z 2023-09-06T16:16:28Z 2023-09-06T16:16:27Z OWNER simonw/datasette/pulls/2175

https://dev.to/hugovk/help-test-python-312-beta-1508/

:books: Documentation preview :books:: https://datasette--2175.org.readthedocs.build/en/2175/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2175/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1876407598 I_kwDOBm6k_c5v17Uu 2169 execute-sql on a database should imply view-database/view-permission simonw 9599 closed 0     0 2023-08-31T22:45:56Z 2023-08-31T22:46:28Z 2023-08-31T22:46:28Z OWNER  

I noticed that a token with execute-sql permission alone did not work, because it was not allowed to view the instance of the database.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2169/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1871935751 I_kwDOD079W85vk3kH 40 ImportError: cannot import name 'formatargspec' from 'inspect' hosslikw 36752421 closed 0     0 2023-08-29T15:36:31Z 2023-08-31T03:18:07Z 2023-08-31T03:18:06Z NONE  

I get the following error when running "pip3 install dogsheep-photos" " from inspect import ismethod, isclass, formatargspec ImportError: cannot import name 'formatargspec' from 'inspect' (/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/inspect.py). Did you mean: 'formatargvalues'?"

Python 3.12.0rc1 sqlite 3.43.0 datasette, version 0.64.3

 dogsheep-photos 256834907 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/40/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1795051447 I_kwDOBm6k_c5q_k-3 2097 Drop Python 3.7 simonw 9599 closed 0     0 2023-07-08T18:39:44Z 2023-08-23T18:18:00Z 2023-08-23T18:18:00Z OWNER  

I'm going to drop Python 3.7.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1153#issuecomment-1627455892

It's not supported any more: https://devguide.python.org/versions/

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2097/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1822940964 I_kwDOBm6k_c5sp98k 2115 Ensure all tests pass against new query view JSON simonw 9599 closed 0   Datasette 1.0a3 9700784 0 2023-07-26T18:25:20Z 2023-08-08T02:01:39Z 2023-08-08T02:01:38Z OWNER  

  • 2109

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2115/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1786243905 I_kwDOCGYnMM5qd-tB 564 Document that running `db.transform()` tidies up the schema indentation simonw 9599 closed 0     0 2023-07-03T13:59:28Z 2023-07-22T22:15:34Z 2023-07-22T22:15:34Z OWNER  

... and it turns out running .transform() with no arguments still fixes the format of the schema!

```pycon

db["log"].add_column("foo", str)

<Table log (id, name2, age, weight, foo)> >>> db["log"].add_column("bar", str) <Table log (id, name2, age, weight, foo, bar)> >>> db["log"].add_column("baz", str) <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT , [foo] TEXT, [bar] TEXT, [baz] TEXT) >>> db["log"].transform() <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT, [foo] TEXT, [bar] TEXT, [baz] TEXT ) ``` _Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618347727_
 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/564/reactions",
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 1
}
   completed
1816857105 I_kwDOCGYnMM5sSwoR 570 `sqlite-utils install -e` option simonw 9599 closed 0     0 2023-07-22T18:32:23Z 2023-07-22T18:55:59Z 2023-07-22T18:32:56Z OWNER  

As seen in LLM.

Needed while working on: - #567

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/570/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1798901709 PR_kwDOBm6k_c5VM2MK 2099 Bump black from 23.3.0 to 23.7.0 dependabot[bot] 49699333 closed 0     0 2023-07-11T13:05:53Z 2023-07-21T21:19:25Z 2023-07-21T21:19:24Z CONTRIBUTOR simonw/datasette/pulls/2099

Bumps black from 23.3.0 to 23.7.0.

Release notes

Sourced from black's releases.

23.7.0

Highlights

  • Runtime support for Python 3.7 has been removed. Formatting 3.7 code will still be supported until further notice (#3765)

Stable style

  • Fix a bug where an illegal trailing comma was added to return type annotations using PEP 604 unions (#3735)
  • Fix several bugs and crashes where comments in stub files were removed or mishandled under some circumstances (#3745)
  • Fix a crash with multi-line magic comments like type: ignore within parentheses (#3740)
  • Fix error in AST validation when Black removes trailing whitespace in a type comment (#3773)

Preview style

  • Implicitly concatenated strings used as function args are no longer wrapped inside parentheses (#3640)
  • Remove blank lines between a class definition and its docstring (#3692)

Configuration

  • The --workers argument to Black can now be specified via the BLACK_NUM_WORKERS environment variable (#3743)
  • .pytest_cache, .ruff_cache and .vscode are now excluded by default (#3691)
  • Fix Black not honouring pyproject.toml settings when running --stdin-filename and the pyproject.toml found isn't in the current working directory (#3719)
  • Black will now error if exclude and extend-exclude have invalid data types in pyproject.toml, instead of silently doing the wrong thing (#3764)

Packaging

  • Upgrade mypyc from 0.991 to 1.3 (#3697)
  • Remove patching of Click that mitigated errors on Python 3.6 with LANG=C (#3768)

Parser

  • Add support for the new PEP 695 syntax in Python 3.12 (#3703)

Performance

  • Speed up Black significantly when the cache is full (#3751)
  • Avoid importing IPython in a case where we wouldn't need it (#3748)

Output

... (truncated)

Changelog

Sourced from black's changelog.

23.7.0

Highlights

  • Runtime support for Python 3.7 has been removed. Formatting 3.7 code will still be supported until further notice (#3765)

Stable style

  • Fix a bug where an illegal trailing comma was added to return type annotations using PEP 604 unions (#3735)
  • Fix several bugs and crashes where comments in stub files were removed or mishandled under some circumstances (#3745)
  • Fix a crash with multi-line magic comments like type: ignore within parentheses (#3740)
  • Fix error in AST validation when Black removes trailing whitespace in a type comment (#3773)

Preview style

  • Implicitly concatenated strings used as function args are no longer wrapped inside parentheses (#3640)
  • Remove blank lines between a class definition and its docstring (#3692)

Configuration

  • The --workers argument to Black can now be specified via the BLACK_NUM_WORKERS environment variable (#3743)
  • .pytest_cache, .ruff_cache and .vscode are now excluded by default (#3691)
  • Fix Black not honouring pyproject.toml settings when running --stdin-filename and the pyproject.toml found isn't in the current working directory (#3719)
  • Black will now error if exclude and extend-exclude have invalid data types in pyproject.toml, instead of silently doing the wrong thing (#3764)

Packaging

  • Upgrade mypyc from 0.991 to 1.3 (#3697)
  • Remove patching of Click that mitigated errors on Python 3.6 with LANG=C (#3768)

Parser

  • Add support for the new PEP 695 syntax in Python 3.12 (#3703)

Performance

  • Speed up Black significantly when the cache is full (#3751)
  • Avoid importing IPython in a case where we wouldn't need it (#3748)

Output

... (truncated)

Commits


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--2099.org.readthedocs.build/en/2099/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2099/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1756975532 PR_kwDOBm6k_c5S_5Jl 2083 Bump blacken-docs from 1.13.0 to 1.14.0 dependabot[bot] 49699333 closed 0     0 2023-06-14T13:57:52Z 2023-06-29T14:31:55Z 2023-06-29T14:31:54Z CONTRIBUTOR simonw/datasette/pulls/2083

Bumps blacken-docs from 1.13.0 to 1.14.0.

Changelog

Sourced from blacken-docs's changelog.

1.14.0 (2023-06-13)

  • Support Python 3.12.
Commits


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--2083.org.readthedocs.build/en/2083/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2083/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1773450152 I_kwDOCGYnMM5ptLOo 559 sqlean support simonw 9599 closed 0     0 2023-06-25T19:27:26Z 2023-06-25T23:25:53Z 2023-06-25T23:25:53Z OWNER  

If sqlean is available, use that.

Refs: - https://github.com/nalgeon/sqlean.py/issues/1#issuecomment-1605707788

This will provide a good workaround for: - #235

sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/559/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1701018909 I_kwDOCGYnMM5lY30d 543 Tests broken on Windows due to new convert() lambda names simonw 9599 closed 0     0 2023-05-08T22:11:29Z 2023-05-08T22:19:04Z 2023-05-08T22:19:04Z OWNER  

https://github.com/simonw/sqlite-utils/actions/runs/4920084038/jobs/8788501314 python sql = 'update [example] set [dt] = lambda_-9223371942137158589([dt]);' From: - #526

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/543/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1576990618 PR_kwDOCGYnMM5JkkED 526 Fix repeated calls to `Table.convert()` mcarpenter 167893 closed 0     0 2023-02-09T00:14:49Z 2023-05-08T21:56:05Z 2023-05-08T21:53:58Z CONTRIBUTOR simonw/sqlite-utils/pulls/526

Fixes #525. All tests pass.

There's perhaps a better way to name lambdas? There could be a collision if a caller passes a function with name like lambda_123456.

SQLite documentation is a little, ah, lite on function name specs. If there is a character that can be used in place of underscore in a SQLite function name that is not permitted in a Python function identifier then that could be a good way to prevent accidental collisions. (I tried dash, colon, dot, no joy).

Otherwise, there is little chance of this happening and if it should happen the risk is mitigated by now throwing an exception in the case of a (name, arity) collision without replace=True.

:books: Documentation preview :books:: https://sqlite-utils--526.org.readthedocs.build/en/526/

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/526/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1432377191 I_kwDOCGYnMM5VYFdn 509 `sqlite-utils transform` breaks DEFAULT string values and STRFTIME() kennysong 2199875 closed 0     0 2022-11-02T02:32:23Z 2023-05-08T21:13:38Z 2023-05-08T21:13:38Z NONE  

Very nice library! Our team found sqlite-utils through @simonw's comment on the "Simple declarative schema migration for SQLite" article, and we were excited to use it, but unfortunately sqlite-utils transform seems to break our DB.

Running sqlite-utils transform to modify a column mangles their DEFAULT values:

  • Default string values are wrapped in extra single quotes
  • Function expressions such as STRFTIME() are turned into strings!

Here are steps to reproduce:

Original database

``` $ sqlite3 test.db << EOF CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) EOF

$ sqlite3 test.db "SELECT sql FROM sqlite_master WHERE name = 'mytable';" CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) ```

Modified database after sqlite-utils

``` $ sqlite3 test.db "INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;" foo|2022-11-02 02:26:58.038

$ sqlite-utils transform test.db mytable --rename col1 renamedcol1

$ sqlite3 test.db "SELECT sql FROM sqlite_master WHERE name = 'mytable';" CREATE TABLE "mytable" ( [renamedcol1] TEXT DEFAULT '''foo''', [col2] TEXT DEFAULT 'STRFTIME(''%Y-%m-%d %H:%M:%f'', ''NOW'')' )

$ sqlite3 test.db "INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;" foo|2022-11-02 02:26:58.038 'foo'|STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW') ```

(Related: #336)

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/509/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1661617056 I_kwDODD6af85jCkOg 15 ambiguous column name: createdAt - on checkin_details view simonw 9599 closed 0     0 2023-04-11T01:07:47Z 2023-04-11T03:16:37Z 2023-04-11T03:16:37Z MEMBER  

It looks like Swarm changed their schema and now both venues and checkins have createdAt fields.

Which breaks this view: https://github.com/dogsheep/swarm-to-sqlite/blob/719b6e96a016d0ca8b316d3bed9c2a7a0cb499ee/swarm_to_sqlite/utils.py#L171-L188

 swarm-to-sqlite 205429375 issue     
{
    "url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/15/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1645098678 PR_kwDOBm6k_c5NIQri 2047 Bump black from 22.12.0 to 23.3.0 dependabot[bot] 49699333 closed 0     0 2023-03-29T06:09:06Z 2023-03-29T06:12:21Z 2023-03-29T06:12:05Z CONTRIBUTOR simonw/datasette/pulls/2047

Bumps black from 22.12.0 to 23.3.0.

Release notes

Sourced from black's releases.

23.3.0

Highlights

This release fixes a longstanding confusing behavior in Black's GitHub action, where the version of the action did not determine the version of Black being run (issue #3382). In addition, there is a small bug fix around imports and a number of improvements to the preview style.

Please try out the preview style with black --preview and tell us your feedback. All changes in the preview style are expected to become part of Black's stable style in January 2024.

Stable style

  • Import lines with # fmt: skip and # fmt: off no longer have an extra blank line added when they are right after another import line (#3610)

Preview style

  • Add trailing commas to collection literals even if there's a comment after the last entry (#3393)
  • async def, async for, and async with statements are now formatted consistently compared to their non-async version. (#3609)
  • with statements that contain two context managers will be consistently wrapped in parentheses (#3589)
  • Let string splitters respect East Asian Width (#3445)
  • Now long string literals can be split after East Asian commas and periods (、 U+3001 IDEOGRAPHIC COMMA, 。 U+3002 IDEOGRAPHIC FULL STOP, & ， U+FF0C FULLWIDTH COMMA) besides before spaces (#3445)
  • For stubs, enforce one blank line after a nested class with a body other than just ... (#3564)
  • Improve handling of multiline strings by changing line split behavior (#1879)

Parser

  • Added support for formatting files with invalid type comments (#3594)

Integrations

  • Update GitHub Action to use the version of Black equivalent to action's version if version input is not specified (#3543)
  • Fix missing Python binary path in autoload script for vim (#3508)

Documentation

  • Document that only the most recent release is supported for security issues; vulnerabilities should be reported through Tidelift (#3612)

... (truncated)

Changelog

Sourced from black's changelog.

23.3.0

Highlights

This release fixes a longstanding confusing behavior in Black's GitHub action, where the version of the action did not determine the version of Black being run (issue #3382). In addition, there is a small bug fix around imports and a number of improvements to the preview style.

Please try out the preview style with black --preview and tell us your feedback. All changes in the preview style are expected to become part of Black's stable style in January 2024.

Stable style

  • Import lines with # fmt: skip and # fmt: off no longer have an extra blank line added when they are right after another import line (#3610)

Preview style

  • Add trailing commas to collection literals even if there's a comment after the last entry (#3393)
  • async def, async for, and async with statements are now formatted consistently compared to their non-async version. (#3609)
  • with statements that contain two context managers will be consistently wrapped in parentheses (#3589)
  • Let string splitters respect East Asian Width (#3445)
  • Now long string literals can be split after East Asian commas and periods (、 U+3001 IDEOGRAPHIC COMMA, 。 U+3002 IDEOGRAPHIC FULL STOP, & ， U+FF0C FULLWIDTH COMMA) besides before spaces (#3445)
  • For stubs, enforce one blank line after a nested class with a body other than just ... (#3564)
  • Improve handling of multiline strings by changing line split behavior (#1879)

Parser

  • Added support for formatting files with invalid type comments (#3594)

Integrations

  • Update GitHub Action to use the version of Black equivalent to action's version if version input is not specified (#3543)
  • Fix missing Python binary path in autoload script for vim (#3508)

Documentation

  • Document that only the most recent release is supported for security issues; vulnerabilities should be reported through Tidelift (#3612)

... (truncated)

Commits


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--2047.org.readthedocs.build/en/2047/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2047/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1528995601 PR_kwDOBm6k_c5HJ55o 1986 Bump sphinx from 6.1.2 to 6.1.3 dependabot[bot] 49699333 closed 0     0 2023-01-11T13:02:36Z 2023-03-29T06:09:50Z 2023-03-29T06:09:49Z CONTRIBUTOR simonw/datasette/pulls/1986

Bumps sphinx from 6.1.2 to 6.1.3.

Release notes

Sourced from sphinx's releases.

v6.1.3

Changelog: https://www.sphinx-doc.org/en/master/changes.html

Changelog

Sourced from sphinx's changelog.

Release 6.1.3 (released Jan 10, 2023)

Bugs fixed

  • #11116: Reverted to previous Sphinx 5 node copying method
  • #11117: Reverted changes to parallel image processing from Sphinx 6.1.0
  • #11119: Supress ValueError in the linkcheck builder
Commits
  • 776d01e Bump to 6.1.3 final
  • a2e922a CHANGES for Sphinx 6.1.3
  • 31162a9 Handle exceptions for get_node_source and get_node_line
  • dcb4429 Restore Sphinx 5 nodes.Element copying behaviour
  • 2a7c40d Undo parallel image changes
  • 7841d3d Ignore more checks in Ruff 0.0.214
  • ddbc5b5 Bump version
  • See full diff in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--1986.org.readthedocs.build/en/1986/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1986/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1644018605 PR_kwDOBm6k_c5NEqBO 2046 Bump furo from 2022.12.7 to 2023.3.27 dependabot[bot] 49699333 closed 0     0 2023-03-28T13:58:14Z 2023-03-29T06:08:02Z 2023-03-29T06:08:01Z CONTRIBUTOR simonw/datasette/pulls/2046

Bumps furo from 2022.12.7 to 2023.3.27.

Changelog

Sourced from furo's changelog.

Changelog

2023.03.27 -- Tasty Tangerine

  • Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
  • Add missing class to Font Awesome examples

2023.03.23 -- Sassy Saffron

  • Update Python version classifiers.
  • Increase the icon size in mobile header.
  • Increase admonition title bg opacity.
  • Change the default API background to transparent.
  • Transition the API background change.
  • Remove the "indent" of API entries which have a background.
  • Break long inline code literals.

2022.12.07 -- Reverent Raspberry

  • ✨ Add support for Sphinx 6.
  • ✨ Improve footnote presentation with docutils 0.18+.
  • Drop support for Sphinx 4.
  • Improve documentation about what the edit button does.
  • Improve handling of empty-flexboxes for better print experience on Chrome.
  • Improve styling for inline signatures.
  • Replace the meta generator tag with a comment.
  • Tweak labels with icons to prevent users selecting icons as text on touch.

2022.09.29 -- Quaint Quartz

  • Add ability to set arbitrary URLs for edit button.
  • Add support for aligning text in MyST-parser generated tables.

2022.09.15 -- Pragmatic Pistachio

  • Add a minimum version constraint on pygments.
  • Add an explicit dependency on sass.
  • Change right sidebar title from "Contents" to "On this page".
  • Correctly position sidebars on small screens.

... (truncated)

Commits


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--2046.org.readthedocs.build/en/2046/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2046/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1617962395 I_kwDOJHON9s5gcCWb 10 Include schema in README simonw 9599 closed 0     0 2023-03-09T20:38:59Z 2023-03-09T20:48:18Z 2023-03-09T20:48:18Z MEMBER  

As seen in other tools like https://github.com/simonw/git-history

 apple-notes-to-sqlite 611552758 issue     
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/10/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1616422013 I_kwDOJHON9s5gWKR9 3 `apple-notes-to-sqlite --dump` option simonw 9599 closed 0     0 2023-03-09T05:05:49Z 2023-03-09T05:06:14Z 2023-03-09T05:06:14Z MEMBER  

Option that doesn't write to the database at all, it just outputs all the notes to stdout as newline-delimited JSON.

 apple-notes-to-sqlite 611552758 issue     
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/3/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1515186569 I_kwDOBm6k_c5aT-mJ 1972 Fix Sphinx warning about extlink extension simonw 9599 closed 0     0 2022-12-31T19:12:04Z 2022-12-31T19:13:26Z 2022-12-31T19:13:26Z OWNER  

[sphinx-autobuild] > sphinx-build -b html /Users/simon/Dropbox/Development/datasette/docs /Users/simon/Dropbox/Development/datasette/docs/_build Running Sphinx v5.3.0 loading pickled environment... done WARNING: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1971#issuecomment-1368266904

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1972/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1000275035 PR_kwDOCGYnMM4r7n-9 327 Extract expand: Support JSON Arrays phaer 101753 closed 0     0 2021-09-19T10:34:30Z 2022-12-29T09:05:36Z 2022-12-29T09:05:36Z NONE simonw/sqlite-utils/pulls/327

Hi,

I needed to extract data in JSON Arrays to normalize data imports. I've quickly hacked the following together based on #241 which refers to #239 where you, @simonw, wrote:

Could this handle lists of objects too? That would be pretty amazing - if the column has a [{...}, {...}] list in it could turn that into a many-to-many.

They way this works in my work is that many-to-many relationships are created for anything that maps to an dictionary in a list, and many-to-one relations for everything else (assumed to be scalar values). Not sure what the best approach here would be? Are many-to-one relationships are at all useful here?

What do you think about this approach? I could try to add it to the cli interface and documentation if wanted.

Thanks for this awesome piece of software in any case! :sun_with_face:

sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/327/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1499387139 PR_kwDOBm6k_c5FmNXs 1961 Use click.echo() instead of print() for --root option simonw 9599 closed 0     0 2022-12-16T00:54:56Z 2022-12-16T00:55:19Z 2022-12-16T00:55:18Z OWNER simonw/datasette/pulls/1961

This ensures the URL is output correctly when running under Docker.

Closes #1958

:books: Documentation preview :books:: https://datasette--1961.org.readthedocs.build/en/1961/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1961/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1491840863 PR_kwDOBm6k_c5FMKSG 1944 Bump black from 22.10.0 to 22.12.0 dependabot[bot] 49699333 closed 0     0 2022-12-12T13:05:11Z 2022-12-13T05:23:31Z 2022-12-13T05:23:30Z CONTRIBUTOR simonw/datasette/pulls/1944

Bumps black from 22.10.0 to 22.12.0.

Release notes

Sourced from black's releases.

22.12.0

Preview style

  • Enforce empty lines before classes and functions with sticky leading comments (#3302)
  • Reformat empty and whitespace-only files as either an empty file (if no newline is present) or as a single newline character (if a newline is present) (#3348)
  • Implicitly concatenated strings used as function args are now wrapped inside parentheses (#3307)
  • Correctly handle trailing commas that are inside a line's leading non-nested parens (#3370)

Configuration

  • Fix incorrectly applied .gitignore rules by considering the .gitignore location and the relative path to the target file (#3338)
  • Fix incorrectly ignoring .gitignore presence when more than one source directory is specified (#3336)

Parser

  • Parsing support has been added for walruses inside generator expression that are passed as function args (for example, any(match := my_re.match(text) for text in texts)) (#3327).

Integrations

  • Vim plugin: Optionally allow using the system installation of Black via let g:black_use_virtualenv = 0(#3309)
Changelog

Sourced from black's changelog.

22.12.0

Preview style

  • Enforce empty lines before classes and functions with sticky leading comments (#3302)
  • Reformat empty and whitespace-only files as either an empty file (if no newline is present) or as a single newline character (if a newline is present) (#3348)
  • Implicitly concatenated strings used as function args are now wrapped inside parentheses (#3307)
  • Correctly handle trailing commas that are inside a line's leading non-nested parens (#3370)

Configuration

  • Fix incorrectly applied .gitignore rules by considering the .gitignore location and the relative path to the target file (#3338)
  • Fix incorrectly ignoring .gitignore presence when more than one source directory is specified (#3336)

Parser

  • Parsing support has been added for walruses inside generator expression that are passed as function args (for example, any(match := my_re.match(text) for text in texts)) (#3327).

Integrations

  • Vim plugin: Optionally allow using the system installation of Black via let g:black_use_virtualenv = 0(#3309)
Commits


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1944/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1493306655 I_kwDOBm6k_c5ZAg0f 1945 `view-instance` should not be checked for /-/actor.json simonw 9599 closed 0   Datasette 1.0a2 8711695 0 2022-12-13T04:01:46Z 2022-12-13T04:11:56Z 2022-12-13T04:11:56Z OWNER  

Spotted this while testing:

  • 1855

    export TOKEN=$(datasette create-token root --secret s -a foo) curl -H "Authorization: Bearer $TOKEN" http://localhost:8002/-/actor.json Returned a Forbidden error (and not in JSON either).
 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1945/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1487764628 I_kwDOCGYnMM5YrXyU 518 flake8 ValueError: Error code '#' supplied to 'extend-ignore' option... simonw 9599 closed 0     0 2022-12-10T01:30:24Z 2022-12-10T01:36:46Z 2022-12-10T01:36:46Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/518/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1455932972 I_kwDOBm6k_c5Wx8Ys 1904 Datasette Lite tests failing due to httpx upgrade simonw 9599 closed 0   Datasette 1.0a0 8658075 0 2022-11-18T22:49:31Z 2022-11-18T22:57:48Z 2022-11-18T22:52:22Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1904/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1450796965 I_kwDOBm6k_c5WeWel 1894 Initialize CodeMirror during DOMContentLoaded instead of onload bgrins 95570 closed 0     0 2022-11-16T03:52:19Z 2022-11-18T07:29:02Z 2022-11-18T07:29:02Z CONTRIBUTOR  

As per https://github.com/simonw/datasette/pull/1893/files#r1023248927 this should prevent a flash between the textarea being replaced by CodeMirror.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1894/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1402608214 PR_kwDOBm6k_c5AdyZ4 1840 test commit 7lingyuan 102635518 closed 0     0 2022-10-10T05:15:26Z 2022-10-10T09:11:50Z 2022-10-10T09:11:50Z FIRST_TIMER simonw/datasette/pulls/1840

lalalalalalala

:books: Documentation preview :books:: https://datasette--1840.org.readthedocs.build/en/1840/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1840/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1400083043 I_kwDOBm6k_c5Tc5Jj 1834 inspect data is not used for caching database hash fgregg 536941 closed 0     0 2022-10-06T17:52:01Z 2022-10-06T20:06:21Z 2022-10-06T20:06:08Z CONTRIBUTOR  

When databases are loaded,

https://github.com/simonw/datasette/blob/cb1e093fd361b758120aefc1a444df02462389a3/datasette/app.py#L257-L260

there is nothing preventing the rehashing of the database for immutable databases.

https://github.com/simonw/datasette/blob/cb1e093fd361b758120aefc1a444df02462389a3/datasette/database.py#L50-L53

what i might expect is that relevant values of inspect_data get passed to the Database class to prevent re-hashing?

With data that is many gigs large, this is a significant start up time.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1834/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1358848933 PR_kwDOBm6k_c4-NhzQ 1797 Bump black from 22.6.0 to 22.8.0 dependabot[bot] 49699333 closed 0     0 2022-09-01T13:25:14Z 2022-09-05T18:51:52Z 2022-09-05T18:51:52Z CONTRIBUTOR simonw/datasette/pulls/1797

Bumps black from 22.6.0 to 22.8.0.

Release notes

Sourced from black's releases.

22.8.0

Highlights

  • Python 3.11 is now supported, except for blackd as aiohttp does not support 3.11 as of publishing (#3234)
  • This is the last release that supports running Black on Python 3.6 (formatting 3.6 code will continue to be supported until further notice)
  • Reword the stability policy to say that we may, in rare cases, make changes that affect code that was not previously formatted by Black (#3155)

Stable style

  • Fix an infinite loop when using # fmt: on/off in the middle of an expression or code block (#3158)
  • Fix incorrect handling of # fmt: skip on colon (:) lines (#3148)
  • Comments are no longer deleted when a line had spaces removed around power operators (#2874)

Preview style

  • Single-character closing docstring quotes are no longer moved to their own line as this is invalid. This was a bug introduced in version 22.6.0. (#3166)
  • --skip-string-normalization / -S now prevents docstring prefixes from being normalized as expected (#3168)
  • When using --skip-magic-trailing-comma or -C, trailing commas are stripped from subscript expressions with more than 1 element (#3209)
  • Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside parentheses (#3162)
  • Fix a string merging/split issue when a comment is present in the middle of implicitly concatenated strings on its own line (#3227)

Blackd

  • blackd now supports enabling the preview style via the X-Preview header (#3217)

Configuration

  • Black now uses the presence of debug f-strings to detect target version (#3215)
  • Fix misdetection of project root and verbose logging of sources in cases involving --stdin-filename (#3216)
  • Immediate .gitignore files in source directories given on the command line are now also respected, previously only .gitignore files in the project root and automatically discovered directories were respected (#3237)

Documentation

  • Recommend using BlackConnect in IntelliJ IDEs (#3150)

Integrations

  • Vim plugin: prefix messages with Black: so it's clear they come from Black (#3194)
  • Docker: changed to a /opt/venv installation + added to PATH to be available to non-root users (#3202)

Output

  • Change from deprecated asyncio.get_event_loop() to create our event loop which removes DeprecationWarning (#3164)
  • Remove logging from internal blib2to3 library since it regularly emits error logs about failed caching that can and should be ignored (#3193)

Parser

  • Type comments are now included in the AST equivalence check consistently so accidental deletion raises an error. Though type comments can't be tracked when running on PyPy 3.7 due to standard library limitations. (#2874)

Performance

... (truncated)

Changelog

Sourced from black's changelog.

22.8.0

Highlights

  • Python 3.11 is now supported, except for blackd as aiohttp does not support 3.11 as of publishing (#3234)
  • This is the last release that supports running Black on Python 3.6 (formatting 3.6 code will continue to be supported until further notice)
  • Reword the stability policy to say that we may, in rare cases, make changes that affect code that was not previously formatted by Black (#3155)

Stable style

  • Fix an infinite loop when using # fmt: on/off in the middle of an expression or code block (#3158)
  • Fix incorrect handling of # fmt: skip on colon (:) lines (#3148)
  • Comments are no longer deleted when a line had spaces removed around power operators (#2874)

Preview style

  • Single-character closing docstring quotes are no longer moved to their own line as this is invalid. This was a bug introduced in version 22.6.0. (#3166)
  • --skip-string-normalization / -S now prevents docstring prefixes from being normalized as expected (#3168)
  • When using --skip-magic-trailing-comma or -C, trailing commas are stripped from subscript expressions with more than 1 element (#3209)
  • Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside parentheses (#3162)
  • Fix a string merging/split issue when a comment is present in the middle of implicitly concatenated strings on its own line (#3227)

Blackd

  • blackd now supports enabling the preview style via the X-Preview header (#3217)

Configuration

  • Black now uses the presence of debug f-strings to detect target version (#3215)
  • Fix misdetection of project root and verbose logging of sources in cases involving --stdin-filename (#3216)
  • Immediate .gitignore files in source directories given on the command line are now also respected, previously only .gitignore files in the project root and automatically discovered directories were respected (#3237)

Documentation

  • Recommend using BlackConnect in IntelliJ IDEs (#3150)

Integrations

... (truncated)

Commits
  • 2018e66 Prepare docs for release 22.8.0 (#3248)
  • 0019261 Update stable branch after publishing to PyPI (#3223)
  • 7757078 Improve & update release process to reflect recent changes (#3242)
  • 767604e Use .gitignore files in the initial source directories (#3237)
  • 2c90480 Use strict mypy checking (#3222)
  • ba618a3 Add parens around implicit string concatenations where it increases readabili...
  • c0cc19b Delay worker count determination
  • afed2c0 Load .gitignore and exclude regex at time of use
  • e269f44 Lazily import parallelized format modules
  • c47b91f Fix misdetection of project root with --stdin-filename (#3216)
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

:books: Documentation preview :books:: https://datasette--1797.org.readthedocs.build/en/1797/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1797/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1334415381 PR_kwDOBm6k_c488nq6 1778 Use Read the Docs action v1 humitos 244656 closed 0     0 2022-08-10T10:30:50Z 2022-08-20T00:04:17Z 2022-08-20T00:04:17Z CONTRIBUTOR simonw/datasette/pulls/1778

Read the Docs repository was renamed from readthedocs/readthedocs-preview to readthedocs/actions/. Now, the preview action is under readthedocs/actions/preview and is tagged as v1

:books: Documentation preview :books:: https://datasette--1778.org.readthedocs.build/en/1778/

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1778/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1343732788 I_kwDOBm6k_c5QF7w0 1788 Make it more obvious that Datasette publish can publish multiple databases simonw 9599 closed 0     0 2022-08-18T22:57:51Z 2022-08-18T23:06:16Z 2022-08-18T23:06:16Z OWNER  

Feedback initially for datasette-publish-fly but it applies to the others too.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1788/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1342374388 PR_kwDOCGYnMM49Wv9T 466 Use Read the Docs action v1 (#463) simonw 9599 closed 0     0 2022-08-17T23:11:50Z 2022-08-17T23:11:54Z 2022-08-17T23:11:54Z OWNER simonw/sqlite-utils/pulls/466

Read the Docs repository was renamed from readthedocs/readthedocs-preview to readthedocs/actions/. Now, the preview action is under readthedocs/actions/preview and is tagged as v1

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/466/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1292368833 I_kwDOBm6k_c5NB_vB 1764 Keep track of config_dir in directory mode (for plugins) eyeseast 25778 closed 0     0 2022-07-03T16:57:49Z 2022-07-18T01:12:45Z 2022-07-18T01:12:45Z CONTRIBUTOR  

I started working on using config_dir with my datasette-query-files plugin and realized Datasette doesn't actually hold onto the config_dir argument. It gets used in __init__ but then forgotten. It would be nice to be able to use it in plugins, though.

Here's the reference issue: https://github.com/eyeseast/datasette-query-files/issues/4

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1764/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1243715381 I_kwDOCGYnMM5KIZc1 436 Add "copy to clipboard" button to code examples in documentation simonw 9599 closed 0     0 2022-05-20T21:53:23Z 2022-05-20T21:57:53Z 2022-05-20T21:57:53Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/436/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1223312279 PR_kwDOBm6k_c43MIU0 1736 Clean up compatibility with Pyodide simonw 9599 closed 0     0 2022-05-02T20:14:38Z 2022-05-02T20:15:28Z 2022-05-02T20:15:27Z OWNER simonw/datasette/pulls/1736

Closes #1735, closes #1733

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1736/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1177059481 I_kwDODFdgUs5GKICZ 71 Store commit parents carltongibson 64686 closed 0     0 2022-03-22T17:06:48Z 2022-04-22T12:44:04Z 2022-04-22T12:44:04Z NONE  

Hi @simonw 👋

Currently, stored commit data doesn't quite give me the information I'm needing...

Committer date and author date are not 100% reliable for dividing a commit history up by release or branch. A PR created before a release but merged after can have earlier dates… — this can be quite frustrating if you're trying to pin down commits for a release: It should be there!, but then isn't. (This gets worse using release branches.)

Would you be open to adding the sha of a parent of a commit to the commit table? (As an FK? 🤔 — likely not feasible.)

It's part of the response body:

"parents": [ { "url": "https://api.github.com/repos/octocat/Hello-World/commits/6dcb09b5b57875f334f61aebed695e2e4193db5e", "sha": "6dcb09b5b57875f334f61aebed695e2e4193db5e" } ],

I think this list should only have a single entry. (🤔 — not sure why it's a list then...)

With this it would be possible to build/reconstruct a chain of commits from the history, that I don't think is available as yet (unless you know a better way).

It is certainly possible to get sequential lists of commits out of git directly, so the same would be possible combining tools, but wondering if a single tool could do it.

What do you think? Thanks! 🏅

github-to-sqlite 207052882 issue     
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/71/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1194790504 I_kwDOBm6k_c5HNw5o 1701 Use + for spaces instead of ~20 simonw 9599 closed 0   Datasette 1.0 3268330 0 2022-04-06T15:40:48Z 2022-04-06T15:55:10Z 2022-04-06T15:55:05Z OWNER  

Tilde encoding introduced in #1657 means that database files with spaces in the name - e.g. the Apple Mail Envelope Index database - end up with URLs like this:

http://127.0.0.1:8001/Envelope~20Index

I think this would be prettier:

http://127.0.0.1:9933/Envelope+Index
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1701/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1182143895 I_kwDOBm6k_c5GdhWX 1691 Bug in pytest-httpx example simonw 9599 closed 0     0 2022-03-26T22:45:30Z 2022-03-26T22:46:09Z 2022-03-26T22:46:09Z OWNER  

https://docs.datasette.io/en/0.61.1/testing_plugins.html#testing-outbound-http-calls-with-pytest-httpx says:

python async def test_outbound_http_call(httpx_mock): httpx_mock.add_response( url='https://www.example.com/', data='Hello world', ) That's wrong - data= should be text=.

https://github.com/Colin-b/pytest_httpx/blob/v0.20.0/README.md#reply-with-custom-body

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1691/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1178456794 I_kwDOCGYnMM5GPdLa 418 Add generated files to .gitignore eyeseast 25778 closed 0     0 2022-03-23T17:48:12Z 2022-03-24T21:01:44Z 2022-03-24T21:01:44Z CONTRIBUTOR  

I end up with these in my local directory:

.hypothesis/
Pipfile
Pipfile.lock
pyproject.toml

Might as well gitignore them.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/418/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1179928510 I_kwDOBm6k_c5GVEe- 1683 allow_facet: False should be respected by column cog menu simonw 9599 closed 0     0 2022-03-24T19:05:06Z 2022-03-24T19:16:36Z 2022-03-24T19:16:36Z OWNER  

The column cog menu currently shows "Facet by this" even if faceting is disabled for the Datasette instance.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1683/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1122413719 I_kwDOBm6k_c5C5qyX 1621 Test against Python 3.11 dev version simonw 9599 closed 0   Datasette 1.0 3268330 0 2022-02-02T21:38:57Z 2022-03-19T04:04:49Z 2022-02-02T21:58:54Z OWNER  

To avoid another surprise like we got with 3.10: https://simonwillison.net/2021/Oct/9/finding-and-reporting-a-bug/

From a quick GitHub code search it looks like 3.11-dev should work: https://cs.github.com/urllib3/urllib3/blob/7bec77e81aa0a194c98381053225813f5347c9d2/.github/workflows/ci.yml#L60

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1621/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1112633417 PR_kwDOBm6k_c4xfryi 1610 Update asgiref requirement from <3.5.0,>=3.2.10 to >=3.2.10,<3.6.0 dependabot[bot] 49699333 closed 0     0 2022-01-24T13:14:18Z 2022-03-06T01:30:27Z 2022-03-06T01:30:27Z CONTRIBUTOR simonw/datasette/pulls/1610

Updates the requirements on asgiref to permit the latest version.

Changelog

Sourced from asgiref's changelog.

3.5.0 (2022-01-22)

  • Python 3.6 is no longer supported, and asyncio calls have been changed to use only the modern versions of the APIs as a result

  • Several causes of RuntimeErrors in cases where an event loop was assigned to a thread but not running

  • Speed improvements in the Local class

3.4.1 (2021-07-01)

  • Fixed an issue with the deadlock detection where it had false positives during exception handling.

3.4.0 (2021-06-27)

  • Calling sync_to_async directly from inside itself (which causes a deadlock when in the default, thread-sensitive mode) now has deadlock detection.

  • asyncio usage has been updated to use the new versions of get_event_loop, ensure_future, wait and gather, avoiding deprecation warnings in Python 3.10. Python 3.6 installs continue to use the old versions; this is only for 3.7+

  • sync_to_async and async_to_sync now have improved type hints that pass through the underlying function type correctly.

  • All Websocket* types are now spelled WebSocket, to match our specs and the official spelling. The old names will work until release 3.5.0, but will raise deprecation warnings.

  • The typing for WebSocketScope and HTTPScope's extensions key has been fixed.

3.3.4 (2021-04-06)

  • The async_to_sync type error is now a warning due the high false negative rate when trying to detect coroutine-returning callables in Python.

3.3.3 (2021-04-06)

... (truncated)

Commits
  • 8b61513 Releasing 3.5.0
  • b2e1c9d Fixed pytest_asyncio deprecation warning.
  • 2eda551 Added testing for Python 3.10.
  • 02fecb6 Drop Python 3.6 (#307)
  • 6689c0a Added stacklevel to warning in AsyncToSync.
  • 4364f9b Changed how StatelessServer handles event loops
  • 7bc055c Update implementations.rst (#295)
  • c758984 Move current_task import choice to module definition time
  • dfe87b2 Fixed #292: Use get_event_loop in class-level code
  • b3a65e3 Removed class variable which has been unused since a0bbe90
  • Additional commits viewable in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1610/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1123851690 I_kwDOCGYnMM5C_J2q 396 mypy failure, sqlite_utils/utils.py:56 simonw 9599 closed 0     0 2022-02-04T06:08:09Z 2022-02-04T06:10:33Z 2022-02-04T06:10:33Z OWNER  

https://github.com/simonw/sqlite-utils/runs/5062725880?check_suite_focus=true

sqlite_utils/utils.py:56: error: Incompatible return value type (got "None", expected "str")

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/396/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1099897648 I_kwDOCGYnMM5Bjxsw 384 Add examples to every `--help` simonw 9599 closed 0     0 2022-01-12T05:31:25Z 2022-01-26T03:15:02Z 2022-01-26T03:15:02Z OWNER  

Everything on https://sqlite-utils.datasette.io/en/stable/cli-reference.html would benefit from an example.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/384/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1107557831 I_kwDOCGYnMM5CA_3H 386 Better "contributing" documentation simonw 9599 closed 0     0 2022-01-19T02:11:48Z 2022-01-19T02:15:21Z 2022-01-19T02:15:21Z OWNER  

This page jumps straight into running the tests: https://sqlite-utils.datasette.io/en/latest/contributing.html

It should add a little more about expected collaboration styles - opening an issue before filing a pull request - and probably link to https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/386/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1082746149 I_kwDOBm6k_c5AiWUl 1560 Table page title has "where where" in it simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-17T00:05:48Z 2022-01-13T22:28:35Z 2022-01-13T22:20:15Z OWNER  

Just noticed this while working on #1518.

% curl -s 'https://latest.datasette.io/fixtures/facetable?_sort=pk&on_earth__exact=1' | grep -C 1 '<title>' <head> <title>fixtures: facetable: 14 rows where where on_earth = 1 sorted by pk</title>

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1560/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083927147 I_kwDOBm6k_c5Am2pr 1571 Track number of executions for execute_write_many() in traces simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-18T19:16:17Z 2022-01-13T22:27:49Z 2021-12-19T20:30:40Z OWNER  

Spotted while working on #1555

There's no indication there of how many times execute_write_many() executed the SQL.

Solving this is a tiny bit tricky because params_seq is an iterator that we don't want to exhaust before passing it to conn.executemany() - so we need to instead wrap it in something that counts how many times it was called.

But then we need a way to attach that to the trace here: https://github.com/simonw/datasette/blob/d637ed46762fdbbd8e32b86f258cd9a53c1cfdc7/datasette/database.py#L115-L122

So probably need to redesign the trace() decorator to allow extra pairs to be attached to it within the with statement.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1571/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1084007781 I_kwDOBm6k_c5AnKVl 1572 "Query took" should be "Queries took" simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-19T04:03:00Z 2022-01-13T22:27:43Z 2021-12-19T04:03:24Z OWNER  

This is misleading, since usually there have been more than one query executed:

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1572/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083718998 I_kwDOBm6k_c5AmD1W 1567 Remove undocumented sqlite_functions mechanism simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-18T01:51:10Z 2022-01-13T22:27:04Z 2021-12-18T01:54:46Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1567/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1098544628 I_kwDOCGYnMM5BenX0 379 CLI options for running ANALYZE simonw 9599 closed 0   3.21 7558727 0 2022-01-11T01:09:16Z 2022-01-11T01:38:01Z 2022-01-11T01:36:48Z OWNER  

The Python methods are all done now, next step is the CLI options. I'll do those in a separate issue.

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009508865

  • [x] sqlite-utils analyze command
  • [x] sqlite-utils create-index --analyze option (see #365)
  • [x] sqlite-utils insert --analyze option
  • [x] sqlite-utils upsert --analyze option

In #378 I also added .delete_where(..., analyze=True) but there isn't currently a sqlite-utils delete-where CLI command - deletions via CLI are expected to be handled using SQL queries.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/379/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1098309897 I_kwDOCGYnMM5BduEJ 378 analyze=True parameter for some methods simonw 9599 closed 0   3.21 7558727 0 2022-01-10T19:54:52Z 2022-01-11T01:08:11Z 2022-01-11T01:08:09Z OWNER  

This would cause ANALYZE to be run against the relevant table at the end of executing the method.

Having browsed the API reference I think the methods that would benefit from an analyze=True parameter are:

  • [x] table.create_index
  • [x] table.insert_all
  • [x] table.upsert_all
  • [x] table.delete_where

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009288898

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/378/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1097436959 I_kwDOCGYnMM5BaY8f 376 `--nl` mode should ignore blank lines simonw 9599 closed 0   3.21 7558727 0 2022-01-10T04:10:54Z 2022-01-10T19:27:41Z 2022-01-10T04:12:46Z OWNER  

Spotted this while manually testing #364 - there's no reason --nl should crash if you feed it an empty line in between JSON objects.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/376/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1094974713 I_kwDOCGYnMM5BQ_z5 362 upsert --detect-types is broken simonw 9599 closed 0     0 2022-01-06T05:12:10Z 2022-01-06T06:54:45Z 2022-01-06T06:28:34Z OWNER  

Noticed this thanks to syntax highlighting in VS Code showing an unused variable - need to fix it and add a test.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/362/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1053136495 I_kwDOCGYnMM4-xZZv 341 `hash_id: Optional[Any]` should be `hash_id: Optional[str]` simonw 9599 closed 0     0 2021-11-15T02:12:39Z 2021-11-15T02:19:31Z 2021-11-15T02:19:31Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/341/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1053087862 I_kwDOCGYnMM4-xNh2 338 dict, list, tuple should all map to TEXT simonw 9599 closed 0     0 2021-11-15T00:28:01Z 2021-11-15T00:36:03Z 2021-11-15T00:36:03Z OWNER  

This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947

So the COLUMN_TYPE_MAPPING should include those too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/322#issuecomment-968401459

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/338/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1041778507 I_kwDOCGYnMM4-GEdL 334 Filter by datetime objects using rows_where() viseshrp 11642379 closed 0     0 2021-11-02T00:44:08Z 2021-11-13T19:23:21Z 2021-11-13T19:23:21Z NONE  

Firstly, thanks for this nice utility. It would be nice to have an example in the docs on how to filter by date range using rows_where(). This doesn't seem to work: table.rows_where('datetime(created) between datetime("2021-10-31T17:29:59.277428-04:00") AND datetime("2021-11-01T03:44:04.544651+00:00")')

I could probably just use db.query(), which works for the above, but it would be nice if I could pass in datetime objects in rows_where(). Thanks.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/334/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
988553806 MDU6SXNzdWU5ODg1NTM4MDY= 1457 suggestion: distinguish names in `--static` documentation ctb 51016 closed 0     0 2021-09-05T17:04:27Z 2021-10-14T18:39:55Z 2021-10-14T18:39:55Z CONTRIBUTOR  

Over in https://docs.datasette.io/en/stable/custom_templates.html#serving-static-files, there is the slightly comical example command -

datasette -m metadata.json --static static:static/ --memory

(now, with MORE STATIC!)

It took me a while to sort out all the URLs and paths involved because I wasn't being very clever. But in the interests of simplification and distinction, I might suggest something like

datasette -m metadata.json --static loc:static-files/ --memory

I will submit a PR for your consideration.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1457/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1026664511 PR_kwDOBm6k_c4tNtoe 1490 Upgrade to httpx 0.20 simonw 9599 closed 0     0 2021-10-14T17:51:05Z 2021-10-14T18:03:45Z 2021-10-14T18:03:44Z OWNER simonw/datasette/pulls/1490

Refs #1488

datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1490/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1000779422 PR_kwDOBm6k_c4r9CTw 1474 Update full_text_search.rst MichaelTiemannOSC 72577720 closed 0     0 2021-09-20T09:59:45Z 2021-10-13T21:10:23Z 2021-10-13T21:10:23Z CONTRIBUTOR simonw/datasette/pulls/1474

Change "above" to "below" to correct correspondence of reference to example.

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1474/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1023243105 I_kwDOBm6k_c48_XNh 1486 pipx installation instructions for plugins don't reference pipx inject RhetTbull 41546558 closed 0     0 2021-10-12T00:43:42Z 2021-10-13T21:09:11Z 2021-10-13T21:09:11Z CONTRIBUTOR  

The datasette installation instructions discuss how to install with pipx, how to upgrade with pipx, and how to upgrade plugins with pipx but do not mention how to install a plugin with pipx. You discussed this on your blog but looks like this didn't make it in when you updated the docs for pipx (#756).

I'll submit a PR shortly to fix this.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1486/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1021849766 I_kwDOBm6k_c486DCm 1483 Running a search on page 2 of results should not preserve ?_next= simonw 9599 closed 0     0 2021-10-10T01:18:12Z 2021-10-13T21:08:10Z 2021-10-13T21:08:10Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1483/reactions",
    "total_count": 2,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1022294524 PR_kwDOBm6k_c4s_4Cw 1484 GitHub Actions: Add Python 3.10 to the tests cclauss 3709715 closed 0     0 2021-10-11T06:03:03Z 2021-10-11T06:03:31Z 2021-10-11T06:03:28Z FIRST_TIME_CONTRIBUTOR simonw/datasette/pulls/1484   datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1484/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
975158266 MDU6SXNzdWU5NzUxNTgyNjY= 19 table activity_summary has no column named appleMoveTime simonw 9599 closed 0     0 2021-08-20T00:46:44Z 2021-08-20T00:54:34Z 2021-08-20T00:54:34Z MEMBER  

Got this error today against a fresh export:

table activity_summary has no column named appleMoveTime
healthkit-to-sqlite 197882382 issue     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/19/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
975049826 MDExOlB1bGxSZXF1ZXN0NzE2MjYyODI5 1444 Ability to deploy demos of branches simonw 9599 closed 0     0 2021-08-19T21:08:04Z 2021-08-19T21:09:44Z 2021-08-19T21:09:39Z OWNER simonw/datasette/pulls/1444

See #1442.

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1444/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
931752773 MDU6SXNzdWU5MzE3NTI3NzM= 294 Add a `sqlite-utils memory` example to the README simonw 9599 closed 0     0 2021-06-28T16:35:59Z 2021-08-18T21:40:03Z 2021-08-18T21:40:03Z OWNER  
sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/294/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
965166058 MDU6SXNzdWU5NjUxNjYwNTg= 313 `.add_foreign_keys()` doesn't reject being called with a View simonw 9599 closed 0     0 2021-08-10T17:22:17Z 2021-08-10T17:25:34Z 2021-08-10T17:25:34Z OWNER  

Spotted this bug using mypy while working on #311 / #312!

% mypy sqlite_utils sqlite_utils/db.py:725: error: Item "View" of "Union[Table, View]" has no attribute "foreign_keys" Found 1 error in 1 file (checked 5 source files) Refers to this code: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L710-L720

It's a bug! We run some checks earlier but none of them ensure that it's a view:

https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L697-L709

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/313/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
959305209 MDU6SXNzdWU5NTkzMDUyMDk= 307 codespell to spell check documentation simonw 9599 closed 0     0 2021-08-03T16:48:19Z 2021-08-03T16:48:53Z 2021-08-03T16:48:53Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/307/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
959276629 MDU6SXNzdWU5NTkyNzY2Mjk= 1416 Use rich to render tracebacks on errors, if available simonw 9599 closed 0     0 2021-08-03T16:12:08Z 2021-08-03T16:12:51Z 2021-08-03T16:12:51Z OWNER  

Now thinking I should try adding Rich as an optional dependency to Datasette - if it's there, show tracebacks using it. Could be really handy for development https://twitter.com/simonw/status/1422576091055616003

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1416/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
957383814 MDU6SXNzdWU5NTczODM4MTQ= 301 insert-files should get a --silent option simonw 9599 closed 0     0 2021-08-01T04:11:03Z 2021-08-02T19:12:21Z 2021-08-02T19:12:21Z OWNER  

The new sqlite-utils convert command I'm adding in #251 will have a --silent option for turning off the progress bars. The only other command that has progress bars right now is insert-files so it should get this option too, for consistency.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/301/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
946553953 MDExOlB1bGxSZXF1ZXN0NjkxNzA3NDA5 1397 Fix for race condition in refresh_schemas(), closes #1231 simonw 9599 closed 0     0 2021-07-16T19:44:43Z 2021-07-16T19:45:00Z 2021-07-16T19:44:58Z OWNER simonw/datasette/pulls/1397
datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1397/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
927789811 MDU6SXNzdWU5Mjc3ODk4MTE= 292 Add contributing documentation simonw 9599 closed 0     0 2021-06-23T02:13:05Z 2021-06-25T17:53:51Z 2021-06-25T17:53:51Z OWNER  

Like https://docs.datasette.io/en/latest/contributing.html (but simpler) - should cover how to run black and flake8 and mypy and how to run the tests.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/292/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
925544070 MDU6SXNzdWU5MjU1NDQwNzA= 287 Update rowid examples in the docs simonw 9599 closed 0     0 2021-06-20T08:03:00Z 2021-06-20T18:26:21Z 2021-06-20T18:26:21Z OWNER  

Changed in #284 - a couple of examples need updating on https://github.com/simonw/sqlite-utils/blob/3.10/docs/cli.rst.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/287/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
925545468 MDU6SXNzdWU5MjU1NDU0Njg= 288 sqlite-utils memory blah.json --schema simonw 9599 closed 0     0 2021-06-20T08:10:40Z 2021-06-20T18:26:21Z 2021-06-20T18:26:21Z OWNER  

Like --dump but only outputs the schema - useful for understanding what you are about to run queries against.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/288/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
924991194 MDU6SXNzdWU5MjQ5OTExOTQ= 280 Add --encoding option to sqlite-utils memory simonw 9599 closed 0     0 2021-06-18T15:03:32Z 2021-06-18T15:29:46Z 2021-06-18T15:29:46Z OWNER  

Follow-on from #272 - this will work like --encoding on sqlite-utils insert and will affect all CSV files processed by sqlite-utils memory.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/280/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
923602693 MDU6SXNzdWU5MjM2MDI2OTM= 276 support small help flag -h mcint 601708 closed 0     0 2021-06-17T07:59:31Z 2021-06-18T14:56:59Z 2021-06-18T14:56:59Z CONTRIBUTOR  
sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/276/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
922832113 MDU6SXNzdWU5MjI4MzIxMTM= 274 sqlite-utils dump my.db command simonw 9599 closed 0     0 2021-06-16T16:30:14Z 2021-06-16T23:51:54Z 2021-06-16T23:51:54Z OWNER  

Inspired by the --dump mechanism I added to sqlite-utils memory here: https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862018937

Can use .iterdump() to implement this: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.iterdump

Maybe instead (or as-well-as) offer --dump which dumps out the SQL from that.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/274/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
919733213 MDU6SXNzdWU5MTk3MzMyMTM= 33 Searching for whitespace throws an error simonw 9599 closed 0     0 2021-06-13T06:57:57Z 2021-06-13T14:36:39Z 2021-06-13T14:36:39Z MEMBER  

https://datasette.io/-/beta?q=+ returns a 500

fts5: syntax error near ""

 dogsheep-beta 197431109 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/33/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
912959264 MDU6SXNzdWU5MTI5NTkyNjQ= 1364 Don't truncate columns on the list of databases simonw 9599 closed 0     0 2021-06-06T22:01:56Z 2021-06-06T22:07:50Z 2021-06-06T22:07:50Z OWNER  

https://covid-19.datasettes.com/covid currently truncates at 9 database columns:

Django SQL Dashboard showed me that this is a bad idea - having the full list of columns is actually really useful documentation for crafting custom SQL queries.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1364/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
912419349 MDU6SXNzdWU5MTI0MTkzNDk= 1359 `?_trace=1` should only be available with a new `trace_debug` setting simonw 9599 closed 0     0 2021-06-05T19:59:27Z 2021-06-05T20:18:46Z 2021-06-05T20:18:46Z OWNER  

Just like template debug mode is controlled by this off-by-default setting: https://github.com/simonw/datasette/blob/368aa5f1b16ca35f82d90ff747023b9a2bfa27c1/datasette/app.py#L160-L164

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1359/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
864969683 MDU6SXNzdWU4NjQ5Njk2ODM= 1305 Index view crashes when any database table is not accessible to actor gfrmin 416374 closed 0     0 2021-04-22T13:44:22Z 2021-06-02T04:26:29Z 2021-06-02T04:26:29Z CONTRIBUTOR  

Because of https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L63, the tables dict built does not include invisible tables; however, if https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L80 is reached (because table_counts was not successfully initialized, e.g. due to a very large database) then as db.get_all_foreign_keys() returns ALL tables, a KeyError will be raised.

This error can be recreated with the fixtures.db if any table is hidden, e.g. by adding something like "foreign_key_references": { "allow": {} } to fixtures-metadata.json and deleting or not table_counts from https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L77.

I'm not sure how to fix this error; perhaps by testing if the table is in the aforementions tables dict.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1305/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
906355849 MDExOlB1bGxSZXF1ZXN0NjU3MzczNzI2 262 Ability to add descending order indexes simonw 9599 closed 0     0 2021-05-29T04:51:04Z 2021-05-29T05:01:42Z 2021-05-29T05:01:39Z OWNER simonw/sqlite-utils/pulls/262

Refs #260

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/262/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
858501079 MDU6SXNzdWU4NTg1MDEwNzk= 255 transform --help should tell you the available types simonw 9599 closed 0     0 2021-04-15T05:24:48Z 2021-05-29T03:55:52Z 2021-05-29T03:55:52Z OWNER  

``` Usage: sqlite-utils transform [OPTIONS] PATH TABLE

Transform a table beyond the capabilities of ALTER TABLE

Options: --type <TEXT TEXT>... Change column type to X ``` This should specify that the possible types are 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/255/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
904582277 MDExOlB1bGxSZXF1ZXN0NjU1NzI2Mzg3 1347 Test docker platform blair only blairdrummond 10801138 closed 0     0 2021-05-28T02:47:09Z 2021-05-28T02:47:28Z 2021-05-28T02:47:28Z CONTRIBUTOR simonw/datasette/pulls/1347
datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1347/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
899169307 MDU6SXNzdWU4OTkxNjkzMDc= 1338 Fix jinja2 warnings simonw 9599 closed 0     0 2021-05-24T01:38:23Z 2021-05-24T01:41:55Z 2021-05-24T01:41:55Z OWNER  

Lots of these in the test suite now, after the Jinja upgrade in #1331: ``` tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:45: DeprecationWarning: 'jinja2.escape' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.escape' instead. label=jinja2.escape(data["label"] or "") or " ",

tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:41: DeprecationWarning: 'jinja2.Markup' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.Markup' instead. return jinja2.Markup( ```

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1338/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
868188068 MDU6SXNzdWU4NjgxODgwNjg= 257 Insert from JSON containing strings with non-ascii characters are escaped as unicode for lists, tuples, dicts. dylan-wu 6586811 closed 0     0 2021-04-26T20:46:25Z 2021-05-19T02:57:05Z 2021-05-19T02:57:05Z CONTRIBUTOR  

JSON Test File (test.json):

json [ { "id": 123, "text": "FR Théâtre" }, { "id": 223, "text": [ "FR Théâtre" ] } ]

Command to import:

bash sqlite-utils insert test.db text test.json --pk=id

Resulting table view from datasette:

Original, db.py line 2225:

python return json.dumps(value, default=repr)

Fix, db.py line 2225:

python return json.dumps(value, default=repr, ensure_ascii=False)

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/257/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
807433181 MDU6SXNzdWU4MDc0MzMxODE= 1224 can't start immutable databases from configuration dir mode camallen 295329 closed 0     0 2021-02-12T17:50:13Z 2021-03-29T00:17:31Z 2021-03-29T00:17:31Z CONTRIBUTOR  

Say I have a /databases/ directory with multiple sqlite db files in that dir (1.db & 2.db) and an inspect-data.json file.

If I start datasette via datasette -h 0.0.0.0 /databases/ then the resulting databases are set to is_mutable: true as inspected via http://127.0.0.1:8001/-/databases.json

I don't want to have to list out the databases by name, e.g. datasette -i /databases/1.db -i /databases/2.db as i want the system to autodetect the sqlite dbs i have in the configuration directory

According to the docs outlined in https://docs.datasette.io/en/latest/settings.html?highlight=immutable#configuration-directory-mode this should be possible

inspect-data.json the result of running datasette inspect - any database files listed here will be treated as immutable, so they should not be changed while Datasette is running

I believe that if the inspect-json.json file present, then in theory the databases will be automatically set to immutable via this code https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L211-L216

However it appears the Click Multiple Options will return a tuple via https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/cli.py#L311-L317

The resulting tuple is passed to the Datasette app via kwargs and overrides the behaviour to set the databases to immutable via this arg https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L182

If you think this is a bug and needs fixing, I am willing to make a PR to check for the empty immutable tuple before calling the Datasette class initializer as I think leaving that class interface alone is the best path here.

Thoughts?

Also - i'm loving Datasette, it truly is a wonderful tool, thank you :)

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1224/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
763207948 MDU6SXNzdWU3NjMyMDc5NDg= 1141 Default styling for bullet point lists simonw 9599 closed 0     0 2020-12-12T02:49:33Z 2021-03-29T00:14:05Z 2021-03-29T00:14:05Z OWNER  

I just noticed that https://datasette.io/content/recent_releases (which uses datasette-render-markdown) is missing its bullet points:

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1141/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
808036774 MDU6SXNzdWU4MDgwMzY3NzQ= 232 Run tests against Windows in GitHub Actions simonw 9599 closed 0     0 2021-02-14T20:09:45Z 2021-02-14T20:39:55Z 2021-02-14T20:39:55Z OWNER  

I'm going to try and get the test suite to run in Windows on GitHub Actions.

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
808037010 MDExOlB1bGxSZXF1ZXN0NTczMTQ3MTY4 233 Run tests against Ubuntu, macOS and Windows simonw 9599 closed 0     0 2021-02-14T20:11:02Z 2021-02-14T20:39:54Z 2021-02-14T20:39:54Z OWNER simonw/sqlite-utils/pulls/233

Refs #232

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/233/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
806861312 MDExOlB1bGxSZXF1ZXN0NTcyMjA5MjQz 1222 --ssl-keyfile and --ssl-certfile, refs #1221 simonw 9599 closed 0     0 2021-02-12T00:45:58Z 2021-02-12T00:52:18Z 2021-02-12T00:52:17Z OWNER simonw/datasette/pulls/1222
datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1222/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
748370021 MDExOlB1bGxSZXF1ZXN0NTI1MzcxMDI5 8 fix import error if note has no "updated" element mkorosec 4028322 closed 0     0 2020-11-22T22:51:05Z 2021-02-11T22:34:06Z 2021-02-11T22:34:06Z CONTRIBUTOR dogsheep/evernote-to-sqlite/pulls/8

I got the following error when executing evernote-to-sqlite enex evernote.db evernote.enex ... File "evernote_to_sqlite/cli.py", line 31, in enex save_note(db, note) File "evernote_to_sqlite/utils.py", line 28, in save_note updated = note.find("updated").text AttributeError: 'NoneType' object has no attribute 'text'

Seems that in some cases the updated element is not added to the note, this is a part of the problematic note:

<created>20201019T074518Z</created> <note-attributes> <source>web.clip7</source> <source-application>webclipper.evernote</source-application> </note-attributes>

 evernote-to-sqlite 303218369 pull     
{
    "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/8/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
802583450 MDU6SXNzdWU4MDI1ODM0NTA= 226 3.4 release is broken - includes a rogue line simonw 9599 closed 0     0 2021-02-06T02:08:01Z 2021-02-06T02:10:26Z 2021-02-06T02:10:26Z OWNER  

I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3

That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199

I'm surprised the tests didn't catch this!

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
792958773 MDExOlB1bGxSZXF1ZXN0NTYwNzI1NzE0 1203 Easier way to run Prettier locally simonw 9599 closed 0     0 2021-01-25T01:39:06Z 2021-01-25T01:41:46Z 2021-01-25T01:41:46Z OWNER simonw/datasette/pulls/1203

Refs #1167

 datasette 107914493 pull     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1203/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
783714076 MDU6SXNzdWU3ODM3MTQwNzY= 1184 request.full_path property simonw 9599 closed 0   Datasette 0.54 6346396 0 2021-01-11T21:21:58Z 2021-01-24T21:21:16Z 2021-01-11T21:34:47Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1184/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
760605882 MDU6SXNzdWU3NjA2MDU4ODI= 1135 Feature: --create option to create database file if it does not yet exist simonw 9599 closed 0     0 2020-12-09T19:23:58Z 2021-01-24T21:19:39Z 2020-12-09T19:45:52Z OWNER  

I'd like to be able to tell people to run the following in the Datasette documentation to get started:

brew install datasette
datasette install datasette-upload-csvs
datasette data.db --create --root --open

This would give them a local Datasette instance with the ability to drag-and-drop CSV files directly into it.

Just one catch: I don't want to have to talk them through creating an empty SQLite database file. So I want to add a new --create option which means "If any of the database files passed on the command-line do not yet exist, create them".

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1135/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
787900412 MDU6SXNzdWU3ODc5MDA0MTI= 222 .m2m() should accept alter=True parameter simonw 9599 closed 0     0 2021-01-18T04:15:43Z 2021-01-18T04:26:10Z 2021-01-18T04:26:10Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
783910901 MDU6SXNzdWU3ODM5MTA5MDE= 221 .add_missing_columns() does not take case insensitivity into account simonw 9599 closed 0     0 2021-01-12T05:01:00Z 2021-01-12T23:17:33Z 2021-01-12T23:17:33Z OWNER  

SQLite columns are case insensitive - but the .add_missing_columns() method doesn't know that. This means that it can crash if it identifies a column that is a case-insensitive duplicate of an existing column. https://github.com/simonw/sqlite-utils/blob/4cc82fd0bccc9d2eeb3510beb4e691d7da099f84/sqlite_utils/db.py#L1974-L1980

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);