id,node_id,number,title,user,user_label,state,locked,assignee,assignee_label,milestone,milestone_label,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,repo_label,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
1802613340,PR_kwDOBm6k_c5VZhfw,2100,Make primary key view accessible to render_cell hook,1563881,meowcat,open,0,,,,,0,2023-07-13T09:30:36Z,2023-08-10T13:15:41Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2100,"
----
:books: Documentation preview :books:: https://datasette--2100.org.readthedocs.build/en/2100/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2100/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1838469176,I_kwDOBm6k_c5tlNA4,2127,Context base class to support documenting the context,9599,simonw,open,0,,,3268330,Datasette 1.0,3,2023-08-07T00:01:02Z,2023-08-10T01:30:25Z,,OWNER,,"This idea first came up here:
- https://github.com/simonw/datasette/issues/2112#issuecomment-1652751140
If `datasette.render_template(...)` takes an optional `Context` subclass as an alternative to a context dictionary, I could then use dataclasses to define the context made available to specific templates - which then gives me something I can use to help document what they are.
Also refs:
- https://github.com/simonw/datasette/issues/1510",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2127/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1843391585,I_kwDOBm6k_c5t3-xh,2134,Add writable canned query demo to latest.datasette.io,9599,simonw,closed,0,,,,,5,2023-08-09T14:31:30Z,2023-08-10T01:22:46Z,2023-08-10T01:05:56Z,OWNER,,"This would be useful while working on:
- #2114",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2134/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1844213115,I_kwDOBm6k_c5t7HV7,2138,on_success_message_sql option for writable canned queries,9599,simonw,closed,0,,,8755003,Datasette 1.0a-next,2,2023-08-10T00:20:14Z,2023-08-10T00:39:40Z,2023-08-10T00:34:26Z,OWNER,,"> Or... how about if the `on_success_message` option could define a SQL query to be executed to generate that message? Maybe `on_success_message_sql`.
- https://github.com/simonw/datasette/issues/2134",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2138/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1841501975,I_kwDOBm6k_c5twxcX,2133,[feature request]`datasette install plugins.json` options,54462,HaveF,closed,0,,,,,9,2023-08-08T15:06:50Z,2023-08-10T00:31:24Z,2023-08-09T22:04:46Z,NONE,,"Hi, simon ❤️
`datasette plugins --all > plugins.json` could generate all plugins info. On another machine, it would be great to install all plugins just by `datasette install plugins.json`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2133/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
627794879,MDU6SXNzdWU2Mjc3OTQ4Nzk=,782,Redesign default .json format,9599,simonw,closed,0,,,8755003,Datasette 1.0a-next,55,2020-05-30T18:47:07Z,2023-08-10T00:07:17Z,2023-08-10T00:07:17Z,OWNER,,The default JSON just isn't right. I find myself using `?_shape=array` for almost everything I build against the API.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/782/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1843600087,I_kwDOBm6k_c5t4xrX,2135,Release notes for 1.0a3,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,3,2023-08-09T16:09:26Z,2023-08-09T19:17:07Z,2023-08-09T19:17:06Z,OWNER,,118 commits! https://github.com/simonw/datasette/compare/1.0a2...26be9f0445b753fb84c802c356b0791a72269f25,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2135/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1843710170,I_kwDOBm6k_c5t5Mja,2136,Query view shouldn't return `columns`,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,4,2023-08-09T17:23:57Z,2023-08-09T19:03:04Z,2023-08-09T19:03:04Z,OWNER,,"I just noticed that https://latest.datasette.io/fixtures/roadside_attraction_characteristics.json?_labels=on&_size=1 returns:
```json
{
""ok"": true,
""next"": ""1"",
""rows"": [
{
""rowid"": 1,
""attraction_id"": {
""value"": 1,
""label"": ""The Mystery Spot""
},
""characteristic_id"": {
""value"": 2,
""label"": ""Paranormal""
}
}
],
""truncated"": false
}
```
But https://latest.datasette.io/fixtures.json?sql=select+rowid%2C+attraction_id%2C+characteristic_id+from+roadside_attraction_characteristics+order+by+rowid+limit+1 returns:
```json
{
""rows"": [
{
""rowid"": 1,
""attraction_id"": 1,
""characteristic_id"": 2
}
],
""columns"": [
""rowid"",
""attraction_id"",
""characteristic_id""
],
""ok"": true,
""truncated"": false
}
```
The `columns` key in the query response is inconsistent with the table response.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2136/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1843821954,I_kwDOBm6k_c5t5n2C,2137,Redesign row default JSON,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2023-08-09T18:49:11Z,2023-08-09T19:02:47Z,,OWNER,,"This URL here:
https://latest.datasette.io/fixtures/simple_primary_key/1.json?_extras=foreign_key_tables
```json
{
""database"": ""fixtures"",
""table"": ""simple_primary_key"",
""rows"": [
{
""id"": ""1"",
""content"": ""hello""
}
],
""columns"": [
""id"",
""content""
],
""primary_keys"": [
""id""
],
""primary_key_values"": [
""1""
],
""units"": {},
""foreign_key_tables"": [
{
""other_table"": ""foreign_key_references"",
""column"": ""id"",
""other_column"": ""foreign_key_with_blank_label"",
""count"": 0,
""link"": ""/fixtures/foreign_key_references?foreign_key_with_blank_label=1""
},
{
""other_table"": ""foreign_key_references"",
""column"": ""id"",
""other_column"": ""foreign_key_with_label"",
""count"": 1,
""link"": ""/fixtures/foreign_key_references?foreign_key_with_label=1""
},
{
""other_table"": ""complex_foreign_keys"",
""column"": ""id"",
""other_column"": ""f3"",
""count"": 1,
""link"": ""/fixtures/complex_foreign_keys?f3=1""
},
{
""other_table"": ""complex_foreign_keys"",
""column"": ""id"",
""other_column"": ""f2"",
""count"": 0,
""link"": ""/fixtures/complex_foreign_keys?f2=1""
},
{
""other_table"": ""complex_foreign_keys"",
""column"": ""id"",
""other_column"": ""f1"",
""count"": 1,
""link"": ""/fixtures/complex_foreign_keys?f1=1""
}
],
""query_ms"": 4.226590999678592,
""source"": ""tests/fixtures.py"",
""source_url"": ""https://github.com/simonw/datasette/blob/main/tests/fixtures.py"",
""license"": ""Apache License 2.0"",
""license_url"": ""https://github.com/simonw/datasette/blob/main/LICENSE"",
""ok"": true,
""truncated"": false
}
```
That `?_extras=` should be `?_extra=` - plus the row JSON should be redesigned to fit the new default JSON representation.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2137/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1822939274,I_kwDOBm6k_c5sp9iK,2113,Implement and document extras for the new query view page,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,3,2023-07-26T18:24:01Z,2023-08-09T17:35:22Z,,OWNER,,- #2109 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2113/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1560662739,I_kwDOBm6k_c5dBdLT,2007,`render_cell()` hook should take an optional `request` argument,9599,simonw,closed,0,,,,,1,2023-01-28T03:13:00Z,2023-08-09T17:15:03Z,2023-01-28T03:34:26Z,OWNER,,From Discord: https://discordapp.com/channels/823971286308356157/996877076982415491/1068227071156965486,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2007/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1840417903,I_kwDOBm6k_c5tsoxv,2131,Refactor code that supports templates_considered comment,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2023-08-08T01:28:36Z,2023-08-09T15:27:41Z,,OWNER,,"I ended up duplicating it here: https://github.com/simonw/datasette/blob/7532feb424b1dce614351e21b2265c04f9669fe2/datasette/views/database.py#L164-L167
I think it should move to `datasette.render_template()` - and maybe have a renamed template variable too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2131/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1822940263,I_kwDOBm6k_c5sp9xn,2114,Implement canned queries against new query JSON work,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,3,2023-07-26T18:24:50Z,2023-08-09T15:26:58Z,2023-08-09T15:26:57Z,OWNER,,- #2109 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2114/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1841343173,I_kwDOBm6k_c5twKrF,2132,Get form fields on query page working again ,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,1,2023-08-08T13:39:05Z,2023-08-08T13:45:10Z,2023-08-08T13:45:09Z,OWNER,,"Caused by:
- #2112
https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+where+%22pk1%22+%3D+%3Ap0+order+by+pk1%2C+pk2%2C+pk3+limit+101&p0=b
The `:p0` form field is missing. Submitting the form results in this error:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2132/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1840324765,I_kwDOBm6k_c5tsSCd,2129,CSV ?sql= should indicate errors,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2023-08-07T23:13:04Z,2023-08-08T02:02:21Z,,OWNER,,"> https://latest.datasette.io/_memory.csv?sql=select+blah is a blank page right now:
```bash
curl -I 'https://latest.datasette.io/_memory.csv?sql=select+blah'
```
```
HTTP/2 200
access-control-allow-origin: *
access-control-allow-headers: Authorization, Content-Type
access-control-expose-headers: Link
access-control-allow-methods: GET, POST, HEAD, OPTIONS
access-control-max-age: 3600
content-type: text/plain; charset=utf-8
x-databases: _memory, _internal, fixtures, fixtures2, extra_database, ephemeral
date: Mon, 07 Aug 2023 23:12:15 GMT
server: Google Frontend
```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/2118#issuecomment-1668688947_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2129/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1822982933,I_kwDOBm6k_c5sqIMV,2117,Figure out what to do about `DatabaseView.name`,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,1,2023-07-26T18:58:06Z,2023-08-08T02:02:07Z,2023-08-08T02:02:07Z,OWNER,,"In the old code:
https://github.com/simonw/datasette/blob/08181823990a71ffa5a1b57b37259198eaa43e06/datasette/views/database.py#L34-L35
This `name` class attribute was later used by some of the plugin hooks, passed as `view_name`: https://github.com/simonw/datasette/blob/18dd88ee4d78fe9d760e9da96028ae06d938a85c/datasette/hookspecs.py#L50-L54
Figure out how that should work once I've refactored those classes to view functions instead.
Refs:
- #2109 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2117/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822940964,I_kwDOBm6k_c5sp98k,2115,Ensure all tests pass against new query view JSON,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,0,2023-07-26T18:25:20Z,2023-08-08T02:01:39Z,2023-08-08T02:01:38Z,OWNER,,- #2109 ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2115/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822938661,I_kwDOBm6k_c5sp9Yl,2112,Build HTML version of /content?sql=...,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,5,2023-07-26T18:23:34Z,2023-08-08T02:01:09Z,2023-08-08T02:01:01Z,OWNER,,"This will help make the hook as robust as possible.
- #2109 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2112/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822937426,I_kwDOBm6k_c5sp9FS,2111,Implement new /content.json?sql=...,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,4,2023-07-26T18:22:39Z,2023-08-08T02:00:37Z,2023-08-08T02:00:22Z,OWNER,,"This will be the base that the remaining work builds on top of. Refs:
- #2109 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2111/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1840329615,I_kwDOBm6k_c5tsTOP,2130,Render plugin mechanism needs `error` and `truncated` fields,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,2,2023-08-07T23:19:19Z,2023-08-08T01:51:54Z,2023-08-08T01:47:42Z,OWNER,,"While working on:
- https://github.com/simonw/datasette/pull/2118
It became clear that the `render` callback function documented here: https://docs.datasette.io/en/0.64.3/plugin_hooks.html#register-output-renderer-datasette
Needs to grow the ability to be told if an error occurred (an `error` string) and if the results were truncated (a `truncated` boolean).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2130/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1823352380,PR_kwDOBm6k_c5Wfgd9,2118,New JSON design for query views,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,11,2023-07-26T23:29:21Z,2023-08-08T01:47:40Z,2023-08-08T01:47:39Z,OWNER,simonw/datasette/pulls/2118,"WIP. Refs:
- #2109
----
:books: Documentation preview :books:: https://datasette--2118.org.readthedocs.build/en/2118/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2118/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1833193570,PR_kwDOBm6k_c5XArm3,2125,Bump sphinx from 6.1.3 to 7.1.2,49699333,dependabot[bot],closed,0,,,,,2,2023-08-02T13:28:39Z,2023-08-07T16:20:30Z,2023-08-07T16:20:27Z,CONTRIBUTOR,simonw/datasette/pulls/2125,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 7.1.2.
Release notes
Sourced from sphinx's releases .
Sphinx 7.1.2
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Sphinx 7.1.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Sphinx 7.1.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0rc1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 7.1.2 (released Aug 02, 2023)
Bugs fixed
#11542 : linkcheck: Properly respect :confval:linkcheck_anchors
and do not spuriously report failures to validate anchors.
Patch by James Addison.
Release 7.1.1 (released Jul 27, 2023)
Bugs fixed
#11514 : Fix SOURCE_DATE_EPOCH
in multi-line copyright footer.
Patch by Bénédikt Tran.
Release 7.1.0 (released Jul 24, 2023)
Incompatible changes
Deprecated
#11412 : Emit warnings on using a deprecated Python-specific index entry type
(namely, module
, keyword
, operator
, object
, exception
,
statement
, and builtin
) in the :rst:dir:index
directive, and
set the removal version to Sphinx 9. Patch by Adam Turner.
Features added
#11415 : Add a checksum to JavaScript and CSS asset URIs included within
generated HTML, using the CRC32 algorithm.
:meth:~sphinx.application.Sphinx.require_sphinx
now allows the version
requirement to be specified as (major, minor)
.
#11011 : Allow configuring a line-length limit for object signatures, via
:confval:maximum_signature_line_length
and the domain-specific variants.
If the length of the signature (in characters) is greater than the configured
limit, each parameter in the signature will be split to its own logical line.
This behaviour may also be controlled by options on object description
directives, for example :rst:dir:py:function:single-line-parameter-list
.
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=7.1.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2125.org.readthedocs.build/en/2125/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2125/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1824399610,PR_kwDOBm6k_c5WjCS8,2121,Bump furo from 2023.3.27 to 2023.7.26,49699333,dependabot[bot],closed,0,,,,,2,2023-07-27T13:40:48Z,2023-08-07T16:20:23Z,2023-08-07T16:20:20Z,CONTRIBUTOR,simonw/datasette/pulls/2121,"Bumps [furo](https://github.com/pradyunsg/furo) from 2023.3.27 to 2023.7.26.
Changelog
Sourced from furo's changelog .
Changelog
2023.07.26 -- Vigilant Volt
Fix compatiblity with Sphinx 7.1.
Improve how content overflow is handled.
Improve how literal blocks containing inline code are handled.
2023.05.20 -- Unassuming Ultramarine
✨ Add support for Sphinx 7.
Drop support for Sphinx 5.
Improve the screen-reader label for sidebar collapse.
Make it easier to create derived themes from Furo.
Bump all JS dependencies (NodeJS and npm packages).
2023.03.27 -- Tasty Tangerine
Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
Add missing class to Font Awesome examples
2023.03.23 -- Sassy Saffron
Update Python version classifiers.
Increase the icon size in mobile header.
Increase admonition title bg opacity.
Change the default API background to transparent.
Transition the API background change.
Remove the "indent" of API entries which have a background.
Break long inline code literals.
2022.12.07 -- Reverent Raspberry
✨ Add support for Sphinx 6.
✨ Improve footnote presentation with docutils 0.18+.
Drop support for Sphinx 4.
Improve documentation about what the edit button does.
Improve handling of empty-flexboxes for better print experience on Chrome.
Improve styling for inline signatures.
... (truncated)
Commits
35f5307
Prepare release: 2023.07.26
0a8bedc
Update changelog
a92dd0c
Make _add_asset_hashes
a no-op with Sphinx 7.1
f8db95b
Improve literals with inline code are handled
1680dbe
Document the use of figclass
with figure
directive
beebd7e
Increase the specificity of the admonition title selector
834e951
Setup uploads to Percy
27bf2c0
[pre-commit.ci] pre-commit autoupdate (#672 )
c8b51d0
Fix how content overflow is handled
80afa27
[pre-commit.ci] pre-commit autoupdate (#652 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2023.3.27&new-version=2023.7.26)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2121.org.readthedocs.build/en/2121/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2121/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1796830110,PR_kwDOBm6k_c5VFw3j,2098,Bump blacken-docs from 1.14.0 to 1.15.0,49699333,dependabot[bot],closed,0,,,,,2,2023-07-10T13:49:12Z,2023-08-07T16:20:22Z,2023-08-07T16:20:20Z,CONTRIBUTOR,simonw/datasette/pulls/2098,"Bumps [blacken-docs](https://github.com/asottile/blacken-docs) from 1.14.0 to 1.15.0.
Changelog
Sourced from blacken-docs's changelog .
1.15.0 (2023-07-09)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=blacken-docs&package-manager=pip&previous-version=1.14.0&new-version=1.15.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2098.org.readthedocs.build/en/2098/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2098/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1839766197,PR_kwDOBm6k_c5XWhWF,2128,"Bump blacken-docs, furo, blacken-docs",49699333,dependabot[bot],closed,0,,,,,1,2023-08-07T15:50:40Z,2023-08-07T16:19:25Z,2023-08-07T16:19:24Z,CONTRIBUTOR,simonw/datasette/pulls/2128,"Bumps the python-packages group with 3 updates: [sphinx](https://github.com/sphinx-doc/sphinx), [furo](https://github.com/pradyunsg/furo) and [blacken-docs](https://github.com/asottile/blacken-docs).
Updates `sphinx` from 6.1.3 to 7.1.2
Release notes
Sourced from sphinx's releases .
Sphinx 7.1.2
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Sphinx 7.1.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Sphinx 7.1.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0rc1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 7.1.2 (released Aug 02, 2023)
Bugs fixed
#11542 : linkcheck: Properly respect :confval:linkcheck_anchors
and do not spuriously report failures to validate anchors.
Patch by James Addison.
Release 7.1.1 (released Jul 27, 2023)
Bugs fixed
#11514 : Fix SOURCE_DATE_EPOCH
in multi-line copyright footer.
Patch by Bénédikt Tran.
Release 7.1.0 (released Jul 24, 2023)
Incompatible changes
Deprecated
#11412 : Emit warnings on using a deprecated Python-specific index entry type
(namely, module
, keyword
, operator
, object
, exception
,
statement
, and builtin
) in the :rst:dir:index
directive, and
set the removal version to Sphinx 9. Patch by Adam Turner.
Features added
#11415 : Add a checksum to JavaScript and CSS asset URIs included within
generated HTML, using the CRC32 algorithm.
:meth:~sphinx.application.Sphinx.require_sphinx
now allows the version
requirement to be specified as (major, minor)
.
#11011 : Allow configuring a line-length limit for object signatures, via
:confval:maximum_signature_line_length
and the domain-specific variants.
If the length of the signature (in characters) is greater than the configured
limit, each parameter in the signature will be split to its own logical line.
This behaviour may also be controlled by options on object description
directives, for example :rst:dir:py:function:single-line-parameter-list
.
... (truncated)
Commits
Updates `furo` from 2023.3.27 to 2023.7.26
Changelog
Sourced from furo's changelog .
Changelog
2023.07.26 -- Vigilant Volt
Fix compatiblity with Sphinx 7.1.
Improve how content overflow is handled.
Improve how literal blocks containing inline code are handled.
2023.05.20 -- Unassuming Ultramarine
✨ Add support for Sphinx 7.
Drop support for Sphinx 5.
Improve the screen-reader label for sidebar collapse.
Make it easier to create derived themes from Furo.
Bump all JS dependencies (NodeJS and npm packages).
2023.03.27 -- Tasty Tangerine
Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
Add missing class to Font Awesome examples
2023.03.23 -- Sassy Saffron
Update Python version classifiers.
Increase the icon size in mobile header.
Increase admonition title bg opacity.
Change the default API background to transparent.
Transition the API background change.
Remove the "indent" of API entries which have a background.
Break long inline code literals.
2022.12.07 -- Reverent Raspberry
✨ Add support for Sphinx 6.
✨ Improve footnote presentation with docutils 0.18+.
Drop support for Sphinx 4.
Improve documentation about what the edit button does.
Improve handling of empty-flexboxes for better print experience on Chrome.
Improve styling for inline signatures.
... (truncated)
Commits
35f5307
Prepare release: 2023.07.26
0a8bedc
Update changelog
a92dd0c
Make _add_asset_hashes
a no-op with Sphinx 7.1
f8db95b
Improve literals with inline code are handled
1680dbe
Document the use of figclass
with figure
directive
beebd7e
Increase the specificity of the admonition title selector
834e951
Setup uploads to Percy
27bf2c0
[pre-commit.ci] pre-commit autoupdate (#672 )
c8b51d0
Fix how content overflow is handled
80afa27
[pre-commit.ci] pre-commit autoupdate (#652 )
Additional commits viewable in compare view
Updates `blacken-docs` from 1.14.0 to 1.15.0
Changelog
Sourced from blacken-docs's changelog .
1.15.0 (2023-07-09)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
----
:books: Documentation preview :books:: https://datasette--2128.org.readthedocs.build/en/2128/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2128/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1826424151,PR_kwDOBm6k_c5Wp6Hs,2124,Bump sphinx from 6.1.3 to 7.1.1,49699333,dependabot[bot],closed,0,,,,,2,2023-07-28T13:23:11Z,2023-08-02T13:28:47Z,2023-08-02T13:28:44Z,CONTRIBUTOR,simonw/datasette/pulls/2124,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 7.1.1.
Release notes
Sourced from sphinx's releases .
Sphinx 7.1.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Sphinx 7.1.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0rc1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 7.1.1 (released Jul 27, 2023)
Bugs fixed
#11514 : Fix SOURCE_DATE_EPOCH
in multi-line copyright footer.
Patch by Bénédikt Tran.
Release 7.1.0 (released Jul 24, 2023)
Incompatible changes
Deprecated
#11412 : Emit warnings on using a deprecated Python-specific index entry type
(namely, module
, keyword
, operator
, object
, exception
,
statement
, and builtin
) in the :rst:dir:index
directive, and
set the removal version to Sphinx 9. Patch by Adam Turner.
Features added
#11415 : Add a checksum to JavaScript and CSS asset URIs included within
generated HTML, using the CRC32 algorithm.
:meth:~sphinx.application.Sphinx.require_sphinx
now allows the version
requirement to be specified as (major, minor)
.
#11011 : Allow configuring a line-length limit for object signatures, via
:confval:maximum_signature_line_length
and the domain-specific variants.
If the length of the signature (in characters) is greater than the configured
limit, each parameter in the signature will be split to its own logical line.
This behaviour may also be controlled by options on object description
directives, for example :rst:dir:py:function:single-line-parameter-list
.
Patch by Thomas Louf, Adam Turner, and Jean-François B.
#10983 : Support for multiline copyright statements in the footer block.
Patch by Stefanie Molin
sphinx.util.display.status_iterator
now clears the current line
with ANSI control codes, rather than overprinting with space characters.
#11431 : linkcheck: Treat SSL failures as broken links.
Patch by Bénédikt Tran
#11157 : Keep the translated
attribute on translated nodes.
#11451 : Improve the traceback displayed when using :option:sphinx-build -T
in parallel builds. Patch by Bénédikt Tran
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=7.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2124.org.readthedocs.build/en/2124/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2124/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
774332247,MDExOlB1bGxSZXF1ZXN0NTQ1MjY0NDM2,1159,Improve the display of facets information,552629,lovasoa,open,0,,,3268330,Datasette 1.0,9,2020-12-24T11:01:47Z,2023-07-31T18:57:59Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1159,"This PR changes the display of facets to hopefully make them more readable.
Before | After
---|---
![image](https://user-images.githubusercontent.com/552629/103084609-b1ec2980-45df-11eb-85bc-68ab8df3e8d9.png) | ![image](https://user-images.githubusercontent.com/552629/103085220-620e6200-45e1-11eb-8189-5dd5d3e2569e.png)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1159/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1820346348,PR_kwDOBm6k_c5WVYor,2107,Bump sphinx from 6.1.3 to 7.1.0,49699333,dependabot[bot],closed,0,,,,,2,2023-07-25T13:28:30Z,2023-07-28T13:23:19Z,2023-07-28T13:23:17Z,CONTRIBUTOR,simonw/datasette/pulls/2107,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 7.1.0.
Release notes
Sourced from sphinx's releases .
Sphinx 7.1.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0rc1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 7.1.0 (released Jul 24, 2023)
Incompatible changes
Deprecated
#11412 : Emit warnings on using a deprecated Python-specific index entry type
(namely, module
, keyword
, operator
, object
, exception
,
statement
, and builtin
) in the :rst:dir:index
directive, and
set the removal version to Sphinx 9. Patch by Adam Turner.
Features added
#11415 : Add a checksum to JavaScript and CSS asset URIs included within
generated HTML, using the CRC32 algorithm.
:meth:~sphinx.application.Sphinx.require_sphinx
now allows the version
requirement to be specified as (major, minor)
.
#11011 : Allow configuring a line-length limit for object signatures, via
:confval:maximum_signature_line_length
and the domain-specific variants.
If the length of the signature (in characters) is greater than the configured
limit, each parameter in the signature will be split to its own logical line.
This behaviour may also be controlled by options on object description
directives, for example :rst:dir:py:function:single-line-parameter-list
.
Patch by Thomas Louf, Adam Turner, and Jean-François B.
#10983 : Support for multiline copyright statements in the footer block.
Patch by Stefanie Molin
sphinx.util.display.status_iterator
now clears the current line
with ANSI control codes, rather than overprinting with space characters.
#11431 : linkcheck: Treat SSL failures as broken links.
Patch by Bénédikt Tran
#11157 : Keep the translated
attribute on translated nodes.
#11451 : Improve the traceback displayed when using :option:sphinx-build -T
in parallel builds. Patch by Bénédikt Tran
#11324 : linkcheck: Use session-basd HTTP requests.
#11438 : Add support for the :rst:dir:py:class
and :rst:dir:py:function
directives for PEP 695 (generic classes and functions declarations) and
PEP 696 (default type parameters). Multi-line support (#11011 ) is enabled
for type parameters list and can be locally controlled on object description
directives, e.g., :rst:dir:py:function:single-line-type-parameter-list
.
Patch by Bénédikt Tran.
#11484 : linkcheck: Allow HTML anchors to be ignored on a per-URL basis
via :confval:linkcheck_anchors_ignore_for_url
while
... (truncated)
Commits
e560f63
Bump to 7.1.0 final
066e0fa
Add translation progress information (#11509 )
0882914
Target PyPI in create-release.yml
21fbee5
Fix OIDC token payload
1a403e4
Add informational log messaging
258b0ea
Revert "Switch to using github.request
"
f9c89e5
Switch to using github.request
52c7f66
Use the correct token minting URL for TestPyPI
6079f28
Install twine in PyPI publish workflow
3d43b9e
Fix github-script syntax in create-release.yml
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=7.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2107.org.readthedocs.build/en/2107/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2107/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1824457306,I_kwDOBm6k_c5svwJa,2122,Parameters on canned queries: fixed or query-generated list?,1563881,meowcat,open,0,,,,,0,2023-07-27T14:07:07Z,2023-07-27T14:07:07Z,,NONE,,"Hi,
currently parameters in canned queries are just text fields. It would be cool to have one of the options below. Would you accept a PR doing something in this direction? (Possibly this could even work as a plugin.)
* adding facets, which would work like facets on tables or views, giving a list of selectable options (and leaving parameters as is)
* making it possible to provide a query which returns selectable values for a parameter, e.g.
```
calendar_entries_current_instrument:
sql: |
select * from calendar_entries
where
DTEND_UNIX > UNIXEPOCH() and
DTSTART_UNIX < UNIXEPOCH() + :days *24*60*60 and
current = 1 and
MACHINE = :instrument
order by
DTSTART_UNIX
params:
days:
sql: ""SELECT VALUE FROM generate_series(1, 30, 1)""
# this obviously requires the corresponding sqlite extension
instrument:
sql: ""SELECT DISTINCT MACHINE FROM calendar_entries""
```
* making it possible to provide a fixed list of parameters
```
calendar_entries_current_instrument:
sql: |
select * from calendar_entries
where
DTEND_UNIX > UNIXEPOCH() and
DTSTART_UNIX < UNIXEPOCH() + :days *24*60*60 and
current = 1 and
MACHINE = :instrument
order by
DTSTART_UNIX
params:
days:
values: [1, 2, 3, 5, 10, 20, 30]
instrument:
values: [supermachine, crappymachine, boringmachine]
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2122/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1719759468,PR_kwDOBm6k_c5RBXH_,2077,Bump furo from 2023.3.27 to 2023.5.20,49699333,dependabot[bot],closed,0,,,,,3,2023-05-22T13:58:16Z,2023-07-27T13:40:55Z,2023-07-27T13:40:53Z,CONTRIBUTOR,simonw/datasette/pulls/2077,"Bumps [furo](https://github.com/pradyunsg/furo) from 2023.3.27 to 2023.5.20.
Changelog
Sourced from furo's changelog .
Changelog
2023.05.20 -- Unassuming Ultramarine
✨ Add support for Sphinx 7.
Drop support for Sphinx 5.
Improve the screen-reader label for sidebar collapse.
Make it easier to create derived themes from Furo.
Bump all JS dependencies (NodeJS and npm packages).
2023.03.27 -- Tasty Tangerine
Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
Add missing class to Font Awesome examples
2023.03.23 -- Sassy Saffron
Update Python version classifiers.
Increase the icon size in mobile header.
Increase admonition title bg opacity.
Change the default API background to transparent.
Transition the API background change.
Remove the "indent" of API entries which have a background.
Break long inline code literals.
2022.12.07 -- Reverent Raspberry
✨ Add support for Sphinx 6.
✨ Improve footnote presentation with docutils 0.18+.
Drop support for Sphinx 4.
Improve documentation about what the edit button does.
Improve handling of empty-flexboxes for better print experience on Chrome.
Improve styling for inline signatures.
Replace the meta
generator tag with a comment.
Tweak labels with icons to prevent users selecting icons as text on touch.
2022.09.29 -- Quaint Quartz
Add ability to set arbitrary URLs for edit button.
... (truncated)
Commits
d2c9ca8
Prepare release: 2023.05.20
662d21b
Update changelog
591780b
Bump compatible Sphinx version
c2e7837
Bump NodeJS and package versions
dd85574
Use the reference HtmlFormatter class defined on PygmentsBridge. (#657 )
6bff419
Fix broken link (#654 )
e7f732e
Improve the screen-reader label for sidebar collapse
48c0bf2
Drop the check for the theme name
1b17d81
[pre-commit.ci] pre-commit autoupdate (#646 )
4904fd5
Remove Python 3.8 constraint from Black pre-commit config (#647 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2023.3.27&new-version=2023.5.20)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2077.org.readthedocs.build/en/2077/
> **Note**
> Automatic rebases have been disabled on this pull request as it has been open for over 30 days.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2077/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1823428714,I_kwDOBm6k_c5sr1Bq,2120,Add __all__ to datasette/__init__.py,9599,simonw,open,0,,,,,0,2023-07-27T01:07:10Z,2023-07-27T01:07:10Z,,OWNER,,"Currently looks like this: https://github.com/simonw/datasette/blob/08181823990a71ffa5a1b57b37259198eaa43e06/datasette/__init__.py#L1-L6
Adding `__all__ = [""Permission"", ""Forbidden""...]` would let me get rid of those `# noqa` comments.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2120/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1822934563,I_kwDOBm6k_c5sp8Yj,2109,Plan for getting the new JSON format query views working,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,5,2023-07-26T18:20:18Z,2023-07-27T00:24:47Z,2023-07-26T18:25:34Z,OWNER,,"I've been stuck on this for too long. I'm breaking it down into a full milestone:
https://github.com/simonw/datasette/milestone/29",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2109/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822936521,I_kwDOBm6k_c5sp83J,2110,Merge database index page and query view,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,1,2023-07-26T18:21:57Z,2023-07-26T19:53:25Z,2023-07-26T19:53:25Z,OWNER,,"Refs:
- #2109
The idea here is that hitting `/content` without a `?sql=` will show an empty result set AND default to including a bunch of extras about the list of tables in the database.
Then I won't have to think about `/content` and `/content?sql=` as separate pages any more.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2110/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822949756,I_kwDOBm6k_c5sqAF8,2116,Turn DatabaseDownload into an async view function,9599,simonw,closed,0,,,9700784,Datasette 1.0a3,3,2023-07-26T18:31:59Z,2023-07-26T18:44:00Z,2023-07-26T18:44:00Z,OWNER,,"A minor refactor, but it is a good starting point for this new branch. Refs:
- #2109",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2116/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1656432059,PR_kwDOBm6k_c5NuBNG,2053,WIP new JSON for queries,9599,simonw,closed,0,,,,,12,2023-04-05T23:26:15Z,2023-07-26T18:28:59Z,2023-07-26T18:26:45Z,OWNER,simonw/datasette/pulls/2053,"Refs:
- #2049
TODO:
- [x] Read queries JSON
- Implement error display with `""ok"": false` and an errors key
- Read queries HTML
- Read queries other formats (plugins)
- Canned read queries (dispatched to from table)
- Write queries (a canned query thing)
- Implement different shapes, refactoring to share code with table
- Implement a sensible subset of extras, also refactoring to share code with table
- Get all tests passing
----
:books: Documentation preview :books:: https://datasette--2053.org.readthedocs.build/en/2053/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2053/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1,
1816857442,I_kwDOBm6k_c5sSwti,2106,`datasette install -e` option,9599,simonw,closed,0,,,,,3,2023-07-22T18:33:42Z,2023-07-26T18:28:33Z,2023-07-22T18:42:54Z,OWNER,,"As seen in LLM and now in `sqlite-utils` too:
- https://github.com/simonw/sqlite-utils/issues/570
Useful for developing plugins, see tutorial at https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2106/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1822813627,I_kwDOBm6k_c5spe27,2108,some (many?) SQL syntax errors are not throwing errors with a .csv endpoint,536941,fgregg,open,0,,,,,0,2023-07-26T16:57:45Z,2023-07-26T16:58:07Z,,CONTRIBUTOR,,"here's a CTE query that should always fail with a syntax error:
```sql
with foo as (nonsense)
select
*
from
foo;
```
when we make this query against the default endpoint, we do indeed get a 400 status code the problem is returned to the user: https://global-power-plants.datasettes.com/global-power-plants?sql=with+foo+as+%28nonsense%29+select+*+from+foo%3B
but, if we use the csv endpoint, we get a 200 status code and no indication of a problem: https://global-power-plants.datasettes.com/global-power-plants.csv?sql=with+foo+as+%28nonsense%29+select+*+from+foo%3B
same with this bad sql
```sql
select
a,
from
foo;
```
https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++a%2C%0D%0Afrom%0D%0A++foo%3B
vs
https://global-power-plants.datasettes.com/global-power-plants.csv?sql=select%0D%0A++a%2C%0D%0Afrom%0D%0A++foo%3B
but, datasette catches this bad sql at both endpoints:
```sql
slect
a
from
foo;
```
https://global-power-plants.datasettes.com/global-power-plants?sql=slect%0D%0A++a%0D%0Afrom%0D%0A++foo%3B
https://global-power-plants.datasettes.com/global-power-plants.csv?sql=slect%0D%0A++a%0D%0Afrom%0D%0A++foo%3B
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2108/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1710164693,PR_kwDOBm6k_c5QhIL2,2075,Bump sphinx from 6.1.3 to 7.0.1,49699333,dependabot[bot],closed,0,,,,,2,2023-05-15T13:59:31Z,2023-07-25T13:28:39Z,2023-07-25T13:28:36Z,CONTRIBUTOR,simonw/datasette/pulls/2075,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 7.0.1.
Release notes
Sourced from sphinx's releases .
v7.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0rc1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 7.0.1 (released May 12, 2023)
Dependencies
#11411 : Support Docutils 0.20
_. Patch by Adam Turner.
.. _Docutils 0.20: https://docutils.sourceforge.io/RELEASE-NOTES.html#release-0-20-2023-05-04
Bugs fixed
#11418 : Clean up remaining references to sphinx.setup_command
following the removal of support for setuptools.
Patch by Willem Mulder.
Release 7.0.0 (released Apr 29, 2023)
Incompatible changes
#11359 : Remove long-deprecated aliases for MecabSplitter
and
DefaultSplitter
in sphinx.search.ja
.
#11360 : Remove deprecated make_old_id
functions in domain object
description classes.
#11363 : Remove the Setuptools integration (build_sphinx
hook in
setup.py
).
#11364 : Remove deprecated sphinx.ext.napoleon.iterators
module.
#11365 : Remove support for the jsdump
format in sphinx.search
.
#11366 : Make locale
a required argument to
sphinx.util.i18n.format_date()
.
#11370 : Remove deprecated sphinx.util.stemmer
module.
#11371 : Remove deprecated sphinx.pycode.ast.parse()
function.
#11372 : Remove deprecated sphinx.io.read_doc()
function.
#11373 : Removed deprecated sphinx.util.get_matching_files()
function.
#11378 : Remove deprecated sphinx.util.docutils.is_html5_writer_available()
function.
#11379 : Make the env
argument to Builder
subclasses required.
#11380 : autosummary: Always emit grouped import exceptions.
#11381 : Remove deprecated style
key for HTML templates.
#11382 : Remove deprecated sphinx.writers.latex.LaTeXTranslator.docclasses
attribute.
#11383 : Remove deprecated sphinx.builders.html.html5_ready
and
sphinx.builders.html.HTMLTranslator
attributes.
#11385 : Remove support for HTML 4 output.
Release 6.2.1 (released Apr 25, 2023)
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=7.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2075.org.readthedocs.build/en/2075/
> **Note**
> Automatic rebases have been disabled on this pull request as it has been open for over 30 days.
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2075/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1798901709,PR_kwDOBm6k_c5VM2MK,2099,Bump black from 23.3.0 to 23.7.0,49699333,dependabot[bot],closed,0,,,,,0,2023-07-11T13:05:53Z,2023-07-21T21:19:25Z,2023-07-21T21:19:24Z,CONTRIBUTOR,simonw/datasette/pulls/2099,"Bumps [black](https://github.com/psf/black) from 23.3.0 to 23.7.0.
Release notes
Sourced from black's releases .
23.7.0
Highlights
Runtime support for Python 3.7 has been removed. Formatting 3.7 code will still be
supported until further notice (#3765 )
Stable style
Fix a bug where an illegal trailing comma was added to return type annotations using
PEP 604 unions (#3735 )
Fix several bugs and crashes where comments in stub files were removed or mishandled
under some circumstances (#3745 )
Fix a crash with multi-line magic comments like type: ignore
within parentheses
(#3740 )
Fix error in AST validation when Black removes trailing whitespace in a type comment
(#3773 )
Preview style
Implicitly concatenated strings used as function args are no longer wrapped inside
parentheses (#3640 )
Remove blank lines between a class definition and its docstring (#3692 )
Configuration
The --workers
argument to Black can now be specified via the BLACK_NUM_WORKERS
environment variable (#3743 )
.pytest_cache
, .ruff_cache
and .vscode
are now excluded by default (#3691 )
Fix Black not honouring pyproject.toml
settings when running --stdin-filename
and the pyproject.toml
found isn't in the current working directory (#3719 )
Black will now error if exclude
and extend-exclude
have invalid data types in
pyproject.toml
, instead of silently doing the wrong thing (#3764 )
Packaging
Upgrade mypyc from 0.991 to 1.3 (#3697 )
Remove patching of Click that mitigated errors on Python 3.6 with LANG=C
(#3768 )
Parser
Add support for the new PEP 695 syntax in Python 3.12 (#3703 )
Performance
Speed up Black significantly when the cache is full (#3751 )
Avoid importing IPython
in a case where we wouldn't need it (#3748 )
Output
... (truncated)
Changelog
Sourced from black's changelog .
23.7.0
Highlights
Runtime support for Python 3.7 has been removed. Formatting 3.7 code will still be
supported until further notice (#3765 )
Stable style
Fix a bug where an illegal trailing comma was added to return type annotations using
PEP 604 unions (#3735 )
Fix several bugs and crashes where comments in stub files were removed or mishandled
under some circumstances (#3745 )
Fix a crash with multi-line magic comments like type: ignore
within parentheses
(#3740 )
Fix error in AST validation when Black removes trailing whitespace in a type comment
(#3773 )
Preview style
Implicitly concatenated strings used as function args are no longer wrapped inside
parentheses (#3640 )
Remove blank lines between a class definition and its docstring (#3692 )
Configuration
The --workers
argument to Black can now be specified via the BLACK_NUM_WORKERS
environment variable (#3743 )
.pytest_cache
, .ruff_cache
and .vscode
are now excluded by default (#3691 )
Fix Black not honouring pyproject.toml
settings when running --stdin-filename
and the pyproject.toml
found isn't in the current working directory (#3719 )
Black will now error if exclude
and extend-exclude
have invalid data types in
pyproject.toml
, instead of silently doing the wrong thing (#3764 )
Packaging
Upgrade mypyc from 0.991 to 1.3 (#3697 )
Remove patching of Click that mitigated errors on Python 3.6 with LANG=C
(#3768 )
Parser
Add support for the new PEP 695 syntax in Python 3.12 (#3703 )
Performance
Speed up Black significantly when the cache is full (#3751 )
Avoid importing IPython
in a case where we wouldn't need it (#3748 )
Output
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=23.3.0&new-version=23.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2099.org.readthedocs.build/en/2099/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2099/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1811824307,I_kwDOBm6k_c5r_j6z,2105,When reverse proxying datasette with nginx an URL element gets erronously added,2235371,aki-k,open,0,,,,,3,2023-07-19T12:16:53Z,2023-07-21T21:17:09Z,,NONE,,"I use this nginx config:
```
location /datasette-llm {
return 302 /datasette-llm/;
}
location /datasette-llm/ {
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection ""Upgrade"";
proxy_http_version 1.1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header Host $host;
proxy_max_temp_file_size 0;
proxy_pass http://127.0.0.1:8001/datasette-llm/;
proxy_redirect http:// https://;
proxy_buffering off;
proxy_request_buffering off;
proxy_set_header Origin '';
client_max_body_size 0;
auth_basic ""datasette-llm"";
auth_basic_user_file /etc/nginx/custom-userdb;
}
```
Then I start datasette with this command:
```
datasette serve --setting base_url /datasette-llm/ $(llm logs path)
```
Everything else works right, except the links in ""This data as json, CSV"".
They get an extra URL element ""datasette-llm"" like this:
https://192.168.1.3:5432/datasette-llm/datasette-llm/logs.json?sql=select+*+from+_llm_migrations
https://192.168.1.3:5432/datasette-llm/datasette-llm/logs.csv?sql=select+*+from+_llm_migrations&_size=max
When I remove that extra ""datasette-llm"" from the URL, those links work too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2105/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1808215339,I_kwDOBm6k_c5rxy0r,2104,Tables starting with an underscore should be treated as hidden,9599,simonw,open,0,,,,,2,2023-07-17T17:13:53Z,2023-07-18T22:41:37Z,,OWNER,,"Plugins can then take advantage of this pattern, for example:
- https://github.com/simonw/datasette-auth-tokens/pull/8",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2104/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1808116827,I_kwDOBm6k_c5rxaxb,2103,data attribute on Datasette tables exposing the primary key of the row,9599,simonw,open,0,,,,,0,2023-07-17T16:18:25Z,2023-07-17T16:18:25Z,,OWNER,,Maybe put it on the `
` but probably better to go on the `td.type-pk`.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2103/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1765870617,I_kwDOBm6k_c5pQQwZ,2087,`--settings settings.json` option,9599,simonw,open,0,,,,,2,2023-06-20T17:48:45Z,2023-07-14T17:02:03Z,,OWNER,,"https://discord.com/channels/823971286308356157/823971286941302908/1120705940728066080
> May I add a request to the whole metadata / settings ? Allow to pass `--settings path/to/settings.json` instead of having to rely exclusively on directory mode to centralize settings (this would reflect the behavior of providing metadata)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2087/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1803264272,I_kwDOBm6k_c5re6EQ,2101,alter: true support for JSON write API,9599,simonw,open,0,,,,,1,2023-07-13T15:24:11Z,2023-07-13T15:24:18Z,,OWNER,,"Requested here: https://discord.com/channels/823971286308356157/823971286941302908/1129034187073134642
> The former datasette-insert plugin had an option `?alter=1` to auto-add new columns. Does the JSON write API also have this?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2101/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
771202454,MDU6SXNzdWU3NzEyMDI0NTQ=,1153,"Use YAML examples in documentation by default, not JSON",9599,simonw,closed,0,,,,,22,2020-12-18T22:20:15Z,2023-07-08T20:09:48Z,2023-07-08T20:08:13Z,OWNER,,"YAML configuration is much better for multi-line strings, and I'm increasingly adding configuration options to Datasette that benefit from that - fragments of HTML in `description_html` or SQL queries used to configure things like https://github.com/simonw/datasette-atom for example.
Rather than confusing things by showing both in the documentation, I should switch all of the default examples to use YAML instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1153/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1794604602,PR_kwDOBm6k_c5U-akg,2096,Clarify docs for descriptions in metadata,15906,garthk,open,0,,,,,0,2023-07-08T01:57:58Z,2023-07-08T01:58:13Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2096,"G'day! I got confused while debugging, earlier today. That's on me, but it does strike me a little repetition in the metadata documentation might help those flicking around it rather than reading it from top to bottom. No worries if you think otherwise.
----
:books: Documentation preview :books:: https://datasette--2096.org.readthedocs.build/en/2096/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2096/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1794097871,I_kwDOBm6k_c5q78LP,2095,"Introduce ""dark mode"" CSS",3315059,jamietanna,open,0,,,,,0,2023-07-07T19:15:58Z,2023-07-07T19:15:58Z,,NONE,,Using [the CSS media query `prefers-color-scheme`](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-color-scheme) we can provide a dark-mode version of Datasette,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2095/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1355148385,I_kwDOBm6k_c5Qxexh,1796,Research an upgrade to CodeMirror 6,9599,simonw,closed,0,,,,,4,2022-08-30T04:27:46Z,2023-07-03T04:58:21Z,2023-07-03T04:58:21Z,OWNER,,"There are still a bunch of bugs in CodeMirror 5 that affect various mobile browsers - see Datasette Discord report here: https://discord.com/channels/823971286308356157/823971286941302908/1013878624992108645
https://user-images.githubusercontent.com/9599/187349269-7b7c0c8c-3894-4810-82f0-de7c1eb940b3.mp4
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1796/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1783304750,I_kwDOBm6k_c5qSxIu,2094,JS Plugin Hooks for the Code Editor,15178711,asg017,open,0,,,,,0,2023-07-01T00:51:57Z,2023-07-01T00:51:57Z,,CONTRIBUTOR,,"When #2052 merges, I'd like to add support to add extensions/functions to the Datasette code editor.
I'd eventually like to build a JS plugin for [`sqlite-docs`](https://github.com/asg017/sqlite-docs), to add things like:
- Inline documentation for tables/columns on hover
- Inline docs for custom functions that are loaded in
- More detailed autocomplete for tables/columns/functions
I did some hacking to see what this would look like, see here:
There can be a new hook that allows JS plugins to add new ""extension"" in the CodeMirror editorview here:
https://github.com/simonw/datasette/blob/8cd60fd1d899952f1153460469b3175465f33f80/datasette/static/cm-editor-6.0.1.js#L25
Will need some more planning. For example, the Codemirror bundle in Datasette has functions that we could re-export for plugins to use (so we don't load 2 version of `""@codemirror/autocomplete""`, for example. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2094/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1781047747,I_kwDOBm6k_c5qKKHD,2092,test_homepage intermittent failure,9599,simonw,closed,0,,,,,2,2023-06-29T15:20:37Z,2023-06-29T15:26:28Z,2023-06-29T15:24:13Z,OWNER,,"e.g. in https://github.com/simonw/datasette/actions/runs/5413590227/jobs/9839373852
```
=================================== FAILURES ===================================
________________________________ test_homepage _________________________________
[gw0] linux -- Python 3.7.17 /opt/hostedtoolcache/Python/3.7.17/x64/bin/python
ds_client =
@pytest.mark.asyncio
async def test_homepage(ds_client):
response = await ds_client.get(""/.json"")
assert response.status_code == 200
assert ""application/json; charset=utf-8"" == response.headers[""content-type""]
data = response.json()
assert data.keys() == {""fixtures"": 0}.keys()
d = data[""fixtures""]
assert d[""name""] == ""fixtures""
assert d[""tables_count""] == 24
assert len(d[""tables_and_views_truncated""]) == 5
assert d[""tables_and_views_more""] is True
# 4 hidden FTS tables + no_primary_key (hidden in metadata)
assert d[""hidden_tables_count""] == 6
# 201 in no_primary_key, plus 6 in other hidden tables:
> assert d[""hidden_table_rows_sum""] == 207, data
E AssertionError: {'fixtures': {'color': '9403e5', 'hash': None, 'hidden_table_rows_sum': 0, 'hidden_tables_count': 6, ...}}
E assert 0 == 207
```
My guess is that this is a timing error, where very occasionally the ""count rows but stop counting if it exceeds a time limit"" thing fails.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2092/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1781005740,I_kwDOBm6k_c5qJ_2s,2090,Adopt ruff for linting,9599,simonw,open,0,,,,,2,2023-06-29T14:56:43Z,2023-06-29T15:05:04Z,,OWNER,,https://beta.ruff.rs/docs/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2090/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1780973290,I_kwDOBm6k_c5qJ37q,2089,codespell test failure,9599,simonw,closed,0,,,,,5,2023-06-29T14:40:10Z,2023-06-29T14:48:11Z,2023-06-29T14:48:10Z,OWNER,,"https://github.com/simonw/datasette/actions/runs/5413443676/jobs/9838999356
```
codespell docs/*.rst --ignore-words docs/codespell-ignore-words.txt
codespell datasette -S datasette/static --ignore-words docs/codespell-ignore-words.txt
shell: /usr/bin/bash -e {0}
env:
pythonLocation: /opt/hostedtoolcache/Python/3.9.17/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.17/x64/lib
docs/metadata.rst:192: displaing ==> displaying
```
This failure is legit, it found a spelling mistake: https://github.com/simonw/datasette/blob/ede62036180993dbd9d4e5d280fc21c183cda1c3/docs/metadata.rst#L192",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2089/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1756975532,PR_kwDOBm6k_c5S_5Jl,2083,Bump blacken-docs from 1.13.0 to 1.14.0,49699333,dependabot[bot],closed,0,,,,,0,2023-06-14T13:57:52Z,2023-06-29T14:31:55Z,2023-06-29T14:31:54Z,CONTRIBUTOR,simonw/datasette/pulls/2083,"Bumps [blacken-docs](https://github.com/asottile/blacken-docs) from 1.13.0 to 1.14.0.
Changelog
Sourced from blacken-docs's changelog .
1.14.0 (2023-06-13)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=blacken-docs&package-manager=pip&previous-version=1.13.0&new-version=1.14.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2083.org.readthedocs.build/en/2083/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2083/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1054244712,I_kwDOBm6k_c4-1n9o,1510,Datasette 1.0 documented template context (maybe via API docs),9599,simonw,open,0,,,3268330,Datasette 1.0,3,2021-11-15T23:23:58Z,2023-06-28T02:05:21Z,,OWNER,,Documented context plus protective unit tests. Goal is that custom templates built for 1.x will not break without a 2.x release.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1510/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
323223872,MDU6SXNzdWUzMjMyMjM4NzI=,260,Validate metadata.json on startup,9599,simonw,open,0,,,,,7,2018-05-15T13:42:56Z,2023-06-21T12:51:22Z,,OWNER,,"It's easy to misspell the name of a database or table and then be puzzled when the metadata settings silently fail.
To avoid this, let's sanity check the provided metadata.json on startup and quit with a useful error message if we find any obvious mistakes.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/260/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1764792125,I_kwDOBm6k_c5pMJc9,2086,Show information on startup in directory configuration mode,9599,simonw,open,0,,,,,0,2023-06-20T07:13:33Z,2023-06-20T07:13:33Z,,OWNER,,"https://discord.com/channels/823971286308356157/823971286941302908/1120516587036889098
> One thing that would be helpful would be message at launch indicating a metadata.json is getting picked up. I'm using directory mode and was editing the wrong file for awhile before I realize nothing I was doing was having any effect.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2086/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1762180409,I_kwDOBm6k_c5pCL05,2085,Interactive row selection in Datasette ,24938923,learning4life,open,0,,,,,0,2023-06-18T08:29:45Z,2023-06-18T08:31:23Z,,NONE,,"Simon did a excellent [prototype](https://til.simonwillison.net/datasette/row-selection-prototype) of an interactive row selection in Datasette.
I hope this [functionality](https://camo.githubusercontent.com/3d4a0f31fb6a27fd279f809af5b53dc3b76faa63c7721e228951c5252b645a77/68747470733a2f2f7374617469632e73696d6f6e77696c6c69736f6e2e6e65742f7374617469632f323032332f6461746173657474652d7069636b65722e676966) can be turned into a Datasette plugin.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2085/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1761613778,I_kwDOBm6k_c5pABfS,2084,Support facets for columns that contain timestamps,19492893,devxpy,open,0,,,,,0,2023-06-17T03:33:54Z,2023-06-17T03:33:54Z,,NONE,,"
Django has this very nice filter for datetime fields -
It would be nice to have something similar to facet by a field that contains a timestamp in datasette too - Which doesn't seem to do anything with timestamps right now...
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2084/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1734786661,PR_kwDOBm6k_c5R0fcK,2082,Catch query interrupted on facet suggest row count,10843208,redraw,open,0,,,,,0,2023-05-31T18:42:46Z,2023-05-31T18:45:26Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2082,"Just like facet's `suggest()` is trapping `QueryInterrupted` for facet columns, we also need to trap `get_row_count()`, which can reach timeout if database tables are big enough.
I've included `get_columns()` inside the block as that's just another query, despite it's a really cheap one and might never raise the exception.
----
:books: Documentation preview :books:: https://datasette--2082.org.readthedocs.build/en/2082/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2082/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1727478903,I_kwDOBm6k_c5m9zx3,2081,Update Endpoints defined in metadata throws 403 Forbidden after a while,15085007,cutmasta-kun,open,0,,,,,0,2023-05-26T11:52:30Z,2023-05-26T11:52:30Z,,NONE,,"Hello. I expose an endpoint to update `tasks`:
```
{
""title"": ""My Datasette Instance"",
""databases"": {
""tasks"": {
""queries"": {
""update_task"": {
""sql"": ""UPDATE tasks SET status = :status, result = :result, systemMessage = :systemMessage WHERE queueID = :queueID"",
""write"": true,
""on_success_message"": ""Task updated"",
""on_success_redirect"": ""/tasks/tasks.json"",
""on_error_message"": ""Task update failed"",
""on_error_redirect"": ""/tasks.json"",
""params"": [""queueID"", ""taskData"", ""status"", ""result"", ""systemMessage""]
}
}
}
}
}
```
This works really well! But after a while, the Datasette Instanz answers with **403 Forbidden**.
I have to delete the database and recreate it in order to work again.
Any help here? (´。_。`)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2081/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1726236847,I_kwDOBm6k_c5m5Eiv,2078,Resolve the difference between `wrap_view()` and `BaseView`,9599,simonw,closed,0,,,,,16,2023-05-25T17:44:32Z,2023-05-26T00:18:46Z,2023-05-26T00:18:46Z,OWNER,,"There are two patterns for implementing views in Datasette at the moment. I want to combine those.
Part of:
- #2053",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2078/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1726603778,PR_kwDOBm6k_c5RYvTU,2080,New View base class,9599,simonw,closed,0,,,,,3,2023-05-25T23:22:55Z,2023-05-26T00:18:45Z,2023-05-26T00:18:44Z,OWNER,simonw/datasette/pulls/2080,"Refs:
- #2078
TODO:
- [x] Teach router layer how to handle this
- [x] Use it for something
----
:books: Documentation preview :books:: https://datasette--2080.org.readthedocs.build/en/2080/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2080/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1726531350,I_kwDOBm6k_c5m6McW,2079,Datasette should serve Access-Control-Max-Age,9599,simonw,closed,0,,,,,8,2023-05-25T21:50:50Z,2023-05-25T22:56:28Z,2023-05-25T22:08:35Z,OWNER,,"Currently the CORS headers served are:
https://github.com/simonw/datasette/blob/9584879534ff0556e04e4c420262972884cac87b/datasette/utils/__init__.py#L1139-L1143
Serving `Access-Control-Max-Age: 600` would allow browsers to cache that for 10 minutes, avoiding additional CORS pre-flight OPTIONS requests during that time.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2079/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1715468032,PR_kwDOBm6k_c5QzEAM,2076,Datsette gpt plugin,130708713,StudioCordillera,open,0,,,,,0,2023-05-18T11:22:30Z,2023-05-18T11:22:45Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2076,"
----
:books: Documentation preview :books:: https://datasette--2076.org.readthedocs.build/en/2076/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2076/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1690842199,PR_kwDOBm6k_c5PgNaA,2068,Bump sphinx from 6.1.3 to 7.0.0,49699333,dependabot[bot],closed,0,,,,,1,2023-05-01T13:58:46Z,2023-05-15T13:59:38Z,2023-05-15T13:59:36Z,CONTRIBUTOR,simonw/datasette/pulls/2068,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 7.0.0.
Release notes
Sourced from sphinx's releases .
v7.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v7.0.0rc1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 7.0.0 (released Apr 29, 2023)
Incompatible changes
#11359 : Remove long-deprecated aliases for MecabSplitter
and
DefaultSplitter
in sphinx.search.ja
.
#11360 : Remove deprecated make_old_id
functions in domain object
description classes.
#11363 : Remove the Setuptools integration (build_sphinx
hook in
setup.py
).
#11364 : Remove deprecated sphinx.ext.napoleon.iterators
module.
#11365 : Remove support for the jsdump
format in sphinx.search
.
#11366 : Make locale
a required argument to
sphinx.util.i18n.format_date()
.
#11370 : Remove deprecated sphinx.util.stemmer
module.
#11371 : Remove deprecated sphinx.pycode.ast.parse()
function.
#11372 : Remove deprecated sphinx.io.read_doc()
function.
#11373 : Removed deprecated sphinx.util.get_matching_files()
function.
#11378 : Remove deprecated sphinx.util.docutils.is_html5_writer_available()
function.
#11379 : Make the env
argument to Builder
subclasses required.
#11380 : autosummary: Always emit grouped import exceptions.
#11381 : Remove deprecated style
key for HTML templates.
#11382 : Remove deprecated sphinx.writers.latex.LaTeXTranslator.docclasses
attribute.
#11383 : Remove deprecated sphinx.builders.html.html5_ready
and
sphinx.builders.html.HTMLTranslator
attributes.
#11385 : Remove support for HTML 4 output.
Release 6.2.1 (released Apr 25, 2023)
Bugs fixed
#11355 : Revert the default type of :confval:nitpick_ignore
and
:confval:nitpick_ignore_regex
to list
.
Release 6.2.0 (released Apr 23, 2023)
Dependencies
Require Docutils 0.18.1 or greater.
Incompatible changes
... (truncated)
Commits
d568b2f
Bump to 7.0.0 final
ff79edf
Remove jsdump
references post removal
1a5133a
Bump to 7.0.0rc1 final
5795fc7
Update sphinx.deprecation
for Sphinx 7.0 (#11386 )
6202087
Add a note to CHANGES for PR 11385
ad47373
Remove HTML 4 support (#11385 )
3e3251d
Remove HTMLTranslator
and html5_ready
from sphinx.builders.html
(...
77fd819
Remove deprecated LaTeXTranslator.docclasses
attribute (#11382 )
4be56f3
Remove deprecated style
key for HTML templates (#11381 )
49027a9
Autosummary: Always emit grouped ImportError
exceptions (#11380 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=7.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2068.org.readthedocs.build/en/2068/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2068/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1708981860,PR_kwDOBm6k_c5QdMea,2074,sort files by mtime,3919561,abbbi,open,0,,,,,0,2023-05-14T15:25:15Z,2023-05-14T15:25:29Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2074,"serving multiple database files and getting tired by the default sort, changes so the sort order puts the latest changed databases to be on top of the list so don't have to scroll down, lazy as i am ;)
----
:books: Documentation preview :books:: https://datasette--2074.org.readthedocs.build/en/2074/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2074/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1708030220,I_kwDOBm6k_c5lznkM,2073,Faceting doesn't work against integer columns in views,9599,simonw,open,0,,,,,2,2023-05-12T18:20:10Z,2023-05-12T18:24:07Z,,OWNER,,"Spotted this issue here: https://til.simonwillison.net/datasette/baseline
I had to do this workaround:
```sql
create view baseline as select
_key,
spec,
'' || json_extract(status, '$.is_baseline') as is_baseline,
json_extract(status, '$.since') as baseline_since,
json_extract(status, '$.support.chrome') as baseline_chrome,
json_extract(status, '$.support.edge') as baseline_edge,
json_extract(status, '$.support.firefox') as baseline_firefox,
json_extract(status, '$.support.safari') as baseline_safari,
compat_features,
caniuse,
usage_stats,
status
from
[index]
```
I think the core issue here is that, against a table, `select * from x where integer_column = '1'` works correctly, due to some kind of column type conversion mechanism... but this mechanism doesn't work against views.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2073/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1702354223,I_kwDOBm6k_c5ld90v,2070,Mechanism for deploying a preview of a branch using Vercel,9599,simonw,closed,0,,,,,2,2023-05-09T16:21:45Z,2023-05-09T16:25:00Z,2023-05-09T16:24:31Z,OWNER,,"I prototyped that here: https://github.com/simonw/one-off-actions/blob/main/.github/workflows/deploy-datasette-branch-preview.yml
It deployed the `json-extras-query` branch here: https://datasette-preview-json-extras-query.vercel.app/",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2070/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1698865182,I_kwDOBm6k_c5lQqAe,2069,[BUG] Cannot insert new data to deployed instance,31861128,yqlbu,open,0,,,,,1,2023-05-07T02:59:42Z,2023-05-07T03:17:35Z,,NONE,,"## Summary
Recently, I deployed an instance of datasette to Vercel with the following plugins:
- datasette-auth-tokens
- datasette-insert
With the above plugins, I was able to insert new data to local sqlite db. However, when it comes to the deployment on Vercel, things behave differently. I observed some errors from the logs console on Vercel:
```console
File ""/var/task/datasette/database.py"", line 179, in _execute_writes
conn = self.connect(write=True)
File ""/var/task/datasette/database.py"", line 93, in connect
assert not (write and not self.is_mutable)
AssertionError
```
I think it is a potential bug.
## Reproduce
metadata.json
```json
{
""plugins"": {
""datasette-insert"": {
""allow"": {
""id"": ""*""
}
},
""datasette-auth-tokens"": {
""tokens"": [
{
""token"": {
""$env"": ""INSERT_TOKEN""
},
""actor"": {
""id"": ""repeater""
}
}
],
""param"": ""_auth_token""
}
}
}
```
commands
```bash
# deploy
datasette publish vercel remote.db \
--project=repeater-bot-sqlite \
--metadata metadata.json \
--install datasette-auth-tokens \
--install datasette-insert \
--vercel-json=vercel.json
# test insert
cat fixtures/dogs.json | curl --request POST -d @- -H ""Authorization: Bearer "" \
'https://repeater-bot-sqlite.vercel.app/-/insert/remote/dogs?pk=id'
```
logs
```console
Traceback (most recent call last):
File ""/var/task/datasette/app.py"", line 1354, in route_path
response = await view(request, send)
File ""/var/task/datasette/app.py"", line 1500, in async_view_fn
response = await async_call_with_supported_arguments(
File ""/var/task/datasette/utils/__init__.py"", line 1005, in async_call_with_supported_arguments
return await fn(*call_with)
File ""/var/task/datasette_insert/__init__.py"", line 14, in insert_or_upsert
response = await insert_or_upsert_implementation(request, datasette)
File ""/var/task/datasette_insert/__init__.py"", line 91, in insert_or_upsert_implementation
table_count = await db.execute_write_fn(write_in_thread, block=True)
File ""/var/task/datasette/database.py"", line 167, in execute_write_fn
raise result
File ""/var/task/datasette/database.py"", line 179, in _execute_writes
conn = self.connect(write=True)
File ""/var/task/datasette/database.py"", line 93, in connect
assert not (write and not self.is_mutable)
AssertionError
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2069/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1690765434,I_kwDOBm6k_c5kxwh6,2067,Litestream-restored db: errors on 3.11 and 3.10.8; but works on py3.10.7 and 3.10.6,39538958,justmars,open,0,,,,,1,2023-05-01T12:42:28Z,2023-05-03T00:16:03Z,,NONE,,"Hi! Wondering if this issue is limited to my local system or if it affects others as well.
It seems like 3.11 errors out on a ""litestream-restored"" database. On further investigation, it also appears to conk out on 3.10.8 but works on 3.10.7 and 3.10.6.
To demo issue I created a test database, replicated it to an aws s3 bucket, then restored the same under various .pyenv-versioned shells where I test whether I can read the database via the sqlite3 cli.
```sh
# create new shell with 3.11.3
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
sqlite3 data/db.sqlite
# SQLite version 3.41.2 2023-03-22 11:56:21
# Enter "".help"" for usage hints.
# sqlite> .tables
# _litestream_lock _litestream_seq movie
# sqlite>
```
However this get me an `OperationalError` when reading via datasette:
Error on 3.11.3 and 3.10.8
```sh
datasette data/db.sqlite
```
```console
/tester/.venv/lib/python3.11/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn(""pkg_resources is deprecated as an API"", DeprecationWarning)
Traceback (most recent call last):
File ""/tester/.venv/bin/datasette"", line 8, in
sys.exit(cli())
^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/click/core.py"", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 143, in wrapped
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 615, in serve
asyncio.get_event_loop().run_until_complete(check_databases(ds))
File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/asyncio/base_events.py"", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/cli.py"", line 660, in check_databases
await database.execute_fn(check_connection)
File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 213, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/mv/.pyenv/versions/3.11.3/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/database.py"", line 211, in in_thread
return fn(conn)
^^^^^^^^
File ""/tester/.venv/lib/python3.11/site-packages/datasette/utils/__init__.py"", line 951, in check_connection
for r in conn.execute(
^^^^^^^^^^^^^
sqlite3.OperationalError: unable to open database file
```
Works on 3.10.7, 3.10.6
```sh
# create new shell with 3.10.7 / 3.10.6
litestream restore -o data/db.sqlite s3://mytestbucketxx/db
datasette data/db.sqlite
# ...
# INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
```
In both scenarios, the only dependencies were the pinned python version and the latest Datasette version 0.64.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2067/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1683229834,PR_kwDOBm6k_c5PG0wF,2064,Bump sphinx from 6.1.3 to 6.2.1,49699333,dependabot[bot],closed,0,,,,,1,2023-04-25T13:57:49Z,2023-05-01T13:58:53Z,2023-05-01T13:58:52Z,CONTRIBUTOR,simonw/datasette/pulls/2064,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 6.2.1.
Release notes
Sourced from sphinx's releases .
v6.2.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 6.2.1 (released Apr 25, 2023)
Bugs fixed
#11355 : Revert the default type of :confval:nitpick_ignore
and
:confval:nitpick_ignore_regex
to list
.
Release 6.2.0 (released Apr 23, 2023)
Dependencies
Require Docutils 0.18.1 or greater.
Incompatible changes
LaTeX: removal of some internal TeX \dimen
registers (not previously
publicly documented) as per 5.1.0 code comments in sphinx.sty
:
\sphinxverbatimsep
, \sphinxverbatimborder
, \sphinxshadowsep
,
\sphinxshadowsize
, and \sphinxshadowrule
. (refs: #11105 )
Remove .egg
support from pycode ModuleAnalyser
; Python eggs are a
now-obsolete binary distribution format
#11089 : Remove deprecated code in sphinx.builders.linkcheck
.
Patch by Daniel Eades
Remove internal-only sphinx.locale.setlocale
Deprecated
#11247 : Deprecate the legacy intersphinx_mapping
format
sphinx.util.osutil.cd
is deprecated in favour of contextlib.chdir
.
Features added
#11277 : :rst:dir:autoproperty
allows the return type to be specified as
a type comment (e.g., # type: () -> int
). Patch by Bénédikt Tran
#10811 : Autosummary: extend __all__
to imported members for template rendering
when option autosummary_ignore_module_all
is set to False
. Patch by
Clement Pinard
#11147 : Add a content_offset
parameter to nested_parse_with_titles()
,
allowing for correct line numbers during nested parsing.
Patch by Jeremy Maitin-Shepard
Update to Unicode CLDR 42
Add a --jobs
synonym for -j
. Patch by Hugo van Kemenade
LaTeX: a command \sphinxbox
for styling text elements with a (possibly
... (truncated)
Commits
ec993dd
Bump to 6.2.1 final
d2aa91f
Revert the default type of nitpick_ignore[_regex]
to list
60d8fa1
Bump version
70102ac
Bump to 6.2.0 final
4e27a5f
Remove unneeded JavaScript from sphinx13
theme
bffb547
Note correct deprecation version for sphinx.util.osutil.cd
59de8d5
Revert "Support and prefer .jinja
to _t
for static templates (#11165 )...
aee3c0a
Partially revert "Disable localisation when SOURCE_DATE_EPOCH is set (#10949 )...
186d596
Use overwrite_file
context manager in test_ext_autodoc_configs
(#11320 )
77483f2
Add missing test decorator for test_util_inspect
(#11321 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=6.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2064.org.readthedocs.build/en/2064/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2064/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1686033652,I_kwDOBm6k_c5kftT0,2065,Datasette cannot be installed with Rye,9599,simonw,closed,0,,,,,4,2023-04-27T03:35:42Z,2023-04-27T05:09:36Z,2023-04-27T05:09:36Z,OWNER,,"https://github.com/mitsuhiko/rye
I tried this:
rye install datasette
But now:
```
% ~/.rye/shims/datasette
Traceback (most recent call last):
File ""/Users/simon/.rye/shims/datasette"", line 5, in
from datasette.cli import cli
File ""/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/cli.py"", line 17, in
from .app import (
File ""/Users/simon/.rye/tools/datasette/lib/python3.11/site-packages/datasette/app.py"", line 14, in
import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
```
I think that's because `setuptools` is not included in Rye.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2065/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1686042269,I_kwDOBm6k_c5kfvad,2066,Failing test: httpx.InvalidURL: URL too long,9599,simonw,closed,0,,,,,10,2023-04-27T03:48:47Z,2023-04-27T04:27:50Z,2023-04-27T04:27:50Z,OWNER,,"https://github.com/simonw/datasette/actions/runs/4815723640/jobs/8574667731
```
def urlparse(url: str = """", **kwargs: typing.Optional[str]) -> ParseResult:
# Initial basic checks on allowable URLs.
# ---------------------------------------
# Hard limit the maximum allowable URL length.
if len(url) > MAX_URL_LENGTH:
> raise InvalidURL(""URL too long"")
E httpx.InvalidURL: URL too long
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/httpx/_urlparse.py:155: InvalidURL
=========================== short test summary info ============================
FAILED tests/test_csv.py::test_max_csv_mb - httpx.InvalidURL: URL too long
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2066/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1681339696,PR_kwDOBm6k_c5PAcGt,2063,Bump sphinx from 6.1.3 to 6.2.0,49699333,dependabot[bot],closed,0,,,,,1,2023-04-24T13:58:21Z,2023-04-25T13:57:55Z,2023-04-25T13:57:53Z,CONTRIBUTOR,simonw/datasette/pulls/2063,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.3 to 6.2.0.
Release notes
Sourced from sphinx's releases .
v6.2.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 6.2.0 (released Apr 23, 2023)
Dependencies
Require Docutils 0.18.1 or greater.
Incompatible changes
LaTeX: removal of some internal TeX \dimen
registers (not previously
publicly documented) as per 5.1.0 code comments in sphinx.sty
:
\sphinxverbatimsep
, \sphinxverbatimborder
, \sphinxshadowsep
,
\sphinxshadowsize
, and \sphinxshadowrule
. (refs: #11105 )
Remove .egg
support from pycode ModuleAnalyser
; Python eggs are a
now-obsolete binary distribution format
#11089 : Remove deprecated code in sphinx.builders.linkcheck
.
Patch by Daniel Eades
Remove internal-only sphinx.locale.setlocale
Deprecated
#11247 : Deprecate the legacy intersphinx_mapping
format
sphinx.util.osutil.cd
is deprecated in favour of contextlib.chdir
.
Features added
#11277 : :rst:dir:autoproperty
allows the return type to be specified as
a type comment (e.g., # type: () -> int
). Patch by Bénédikt Tran
#10811 : Autosummary: extend __all__
to imported members for template rendering
when option autosummary_ignore_module_all
is set to False
. Patch by
Clement Pinard
#11147 : Add a content_offset
parameter to nested_parse_with_titles()
,
allowing for correct line numbers during nested parsing.
Patch by Jeremy Maitin-Shepard
Update to Unicode CLDR 42
Add a --jobs
synonym for -j
. Patch by Hugo van Kemenade
LaTeX: a command \sphinxbox
for styling text elements with a (possibly
rounded) box, optional background color and shadow, has been added.
See :ref:sphinxbox
. (refs: #11224 )
LaTeX: add \sphinxstylenotetitle
, ..., \sphinxstylewarningtitle
, ...,
for an extra layer of mark-up freeing up \sphinxstrong
for other uses.
See :ref:latex-macros
. (refs: #11267 )
LaTeX: :dudir:note
, :dudir:hint
, :dudir:important
and :dudir:tip
can
now each be styled as the other admonitions, i.e. possibly with a background
color, individual border widths and paddings, possibly rounded corners, and
optional shadow. See :ref:additionalcss
. (refs: #11234 )
... (truncated)
Commits
e7d4c36
Bump to 6.2.0 final
4e27a5f
Remove unneeded JavaScript from sphinx13
theme
bffb547
Note correct deprecation version for sphinx.util.osutil.cd
59de8d5
Revert "Support and prefer .jinja
to _t
for static templates (#11165 )...
aee3c0a
Partially revert "Disable localisation when SOURCE_DATE_EPOCH is set (#10949 )...
186d596
Use overwrite_file
context manager in test_ext_autodoc_configs
(#11320 )
77483f2
Add missing test decorator for test_util_inspect
(#11321 )
d8f15c7
Increase timeout threshold for linkcheck
tests (#11326 )
b430e05
Create a 'search field' component for themes (#11045 )
e2f66ce
Update CHANGES for PR #11333
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.3&new-version=6.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2063.org.readthedocs.build/en/2063/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2063/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1674322631,PR_kwDOBm6k_c5OpEz_,2061,"Add ""Packaging a plugin using Poetry"" section in docs",1238873,rclement,open,0,,,,,0,2023-04-19T07:23:28Z,2023-04-19T07:27:18Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2061,"This PR adds a new section about packaging a plugin using `poetry` within the ""Writing plugins"" page of the documentation.
----
:books: Documentation preview :books:: https://datasette--2061.org.readthedocs.build/en/2061/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2061/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1661860507,PR_kwDOBm6k_c5N_bMw,2056,GitHub Action to lint Python code with ruff,3709715,cclauss,open,0,,,,,6,2023-04-11T06:41:27Z,2023-04-15T14:24:46Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2056,"[Ruff](https://beta.ruff.rs/) supports [over 500 lint rules](https://beta.ruff.rs/docs/rules) and can be used to replace [Flake8](https://pypi.org/project/flake8/) (plus dozens of plugins), [isort](https://pypi.org/project/isort/), [pydocstyle](https://pypi.org/project/pydocstyle/), [yesqa](https://github.com/asottile/yesqa), [eradicate](https://pypi.org/project/eradicate/), [pyupgrade](https://pypi.org/project/pyupgrade/), and [autoflake](https://pypi.org/project/autoflake/), all while executing (in Rust) tens or hundreds of times faster than any individual tool.
The ruff Action uses minimal steps to run in ~5 seconds, rapidly providing intuitive GitHub Annotations to contributors.
![image](https://user-images.githubusercontent.com/3709715/223758136-afc386d2-70aa-4eff-953a-2c2d82ceea23.png)
----
:books: Documentation preview :books:: https://datasette--2056.org.readthedocs.build/en/2056/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2056/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1663399821,I_kwDOBm6k_c5jJXeN,2058,"500 ""attempt to write a readonly database"" error caused by ""PRAGMA schema_version""",9599,simonw,open,0,,,,,9,2023-04-11T23:57:50Z,2023-04-13T16:35:21Z,,OWNER,,"I've not been able to replicate this myself yet, but I've seen log files from a user affected by it.
```
File ""/usr/local/lib/python3.11/site-packages/datasette/views/base.py"", line 89, in dispatch_request
await self.ds.refresh_schemas()
File ""/usr/local/lib/python3.11/site-packages/datasette/app.py"", line 371, in refresh_schemas
await self._refresh_schemas()
File ""/usr/local/lib/python3.11/site-packages/datasette/app.py"", line 386, in _refresh_schemas
schema_version = (await db.execute(""PRAGMA schema_version"")).first()[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 267, in execute
results = await self.execute_fn(sql_operation_in_thread)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 213, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 211, in in_thread
return fn(conn)
^^^^^^^^
File ""/usr/local/lib/python3.11/site-packages/datasette/database.py"", line 237, in sql_operation_in_thread
cursor.execute(sql, params if params is not None else {})
sqlite3.OperationalError: attempt to write a readonly database
```
That's running the official Datasette Docker image on https://fly.io/ - it's causing 500 errors on every page of their site.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2058/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1665510265,I_kwDOBm6k_c5jRat5,2060,Clean up a bunch of warnings from ruff,9599,simonw,open,0,,,,,0,2023-04-13T01:23:02Z,2023-04-13T01:23:02Z,,OWNER,,"See:
- #2056
`ruff` spots a bunch of warnings about things like unused variables - would be good to clean up as many of these as possible.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2060/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1665053646,I_kwDOBm6k_c5jPrPO,2059,"""Deceptive site ahead"" alert on Heroku deployment",1186275,mtdukes,open,0,,,,,1,2023-04-12T18:34:51Z,2023-04-13T01:13:01Z,,NONE,,"I deployed a fairly basic instance of Datasette (`datasette-auth-passwords` is the only plugin) using Heroku. The deployed URL now gives a ""Deceptive site ahead"" warning to users.
Is there way around this? Maybe a way to add ownership verification [through Google's search console](https://search.google.com/search-console/welcome)? ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2059/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1657861026,I_kwDOBm6k_c5i0POi,2054,"Make detailed notes on how table, query and row views work right now",9599,simonw,open,0,,,,,13,2023-04-06T18:21:09Z,2023-04-07T20:14:38Z,,OWNER,,"Research to help influence the following:
- #2049
- #2053
- #2050
- #262 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2054/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1646734246,I_kwDOBm6k_c5iJyum,2049,Custom SQL queries should use new JSON ?_extra= format,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,4,2023-03-30T00:42:53Z,2023-04-05T23:29:27Z,,OWNER,,"Related:
- #262
I've made the change to the table view, now I need the new format to work for arbitrary SQL queries too.
Note that this incorporates both arbitrary SQL queries and canned queries.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2049/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1649791661,I_kwDOBm6k_c5iVdKt,2050,Row page JSON should use new ?_extra= format,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2023-03-31T17:56:53Z,2023-03-31T17:59:49Z,,OWNER,,"https://latest.datasette.io/fixtures/facetable/2.json
Related:
- #2049
- #1709 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2050/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1649793525,I_kwDOBm6k_c5iVdn1,2051,`?_extra=row_urls` for table pages,9599,simonw,open,0,,,,,0,2023-03-31T17:58:36Z,2023-03-31T17:58:36Z,,OWNER,,Provides URLs to the JSON version of those rows. Maybe it persists the `?_shape=` option too? Not sure about that.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2051/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1615692818,I_kwDOBm6k_c5gTYQS,2035,Potential feature: special support for `?a=1&a=2` on the query page,9599,simonw,open,0,,,3268330,Datasette 1.0,14,2023-03-08T18:05:03Z,2023-03-31T16:09:08Z,,OWNER,,"From a discussion on Discord: https://discord.com/channels/823971286308356157/996877076982415491/1082789517062320138
The key idea is to make it easier for people to implement `where id in (...)` that's populated from query string arguments.
What if you could add `?id=11&id=32&id=62` to the URL and have that made available as a list that can be used in the query?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2035/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1531991339,I_kwDOBm6k_c5bUFUr,1989,Suggestion: Hiding columns,116795,pax,open,0,,,,,3,2023-01-13T09:33:32Z,2023-03-31T06:18:05Z,,NONE,,As there's the possibility of [hiding tables](https://docs.datasette.io/en/stable/metadata.html#hiding-tables) - I've run into the **need of hiding specific columns** - data that's either not relevant for public or can't be shown due to privacy reasons. ,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1989/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1613974869,PR_kwDOBm6k_c5LgPS-,2034,remove an unused `app` var in cli.py,4370201,wenhoujx,open,0,,,,,2,2023-03-07T18:19:05Z,2023-03-29T20:56:20Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2034,"this var `app` isn't actually used? unless init it does some side-effect outside of the event loop, idon't think it's necessary.
Feel free to ignore this PR if the deleted line actually does something.
----
:books: Documentation preview :books:: https://datasette--2034.org.readthedocs.build/en/2034/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2034/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1646068413,I_kwDOBm6k_c5iHQK9,2048,Test failures encountered while packaging for GNU Guix,8332263,Apteryks,open,0,,,,,0,2023-03-29T15:36:54Z,2023-03-29T15:36:54Z,,NONE,,"Hello,
While reviewing a packaged submitted to Guix to add `datasette`, the test suite produces the following errors:
```
=================================== FAILURES ===================================
_________________________ test_row_strange_table_name __________________________
[gw21] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
def test_row_strange_table_name(app_client):
response = app_client.get(
""/fixtures/table~2Fwith~2Fslashes~2Ecsv/3.json?_shape=objects""
)
> assert response.status == 200
E assert 400 == 200
E + where 400 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:701: AssertionError
----------------------------- Captured stderr call -----------------------------
ERROR: conn=, sql = 'select rowid, * from [table%7E2Fwith%7E2Fslashes%7E2Ecsv] where ""rowid""=:p0', params = {'p0': '3'}: no such table: table%7E2Fwith%7E2Fslashes%7E2Ecsv
_______________ test_database_page_for_database_with_dot_in_name _______________
[gw15] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client_with_dot =
def test_database_page_for_database_with_dot_in_name(app_client_with_dot):
response = app_client_with_dot.get(""/fixtures~2Edot.json"")
> assert response.status == 200
E assert 302 == 200
E + where 302 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:633: AssertionError
___________________ test_tilde_encoded_database_names[fo%o] ____________________
[gw6] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
db_name = 'fo%o'
@pytest.mark.asyncio
@pytest.mark.parametrize(""db_name"", (""foo"", r""fo%o"", ""f~/c.d""))
async def test_tilde_encoded_database_names(db_name):
ds = Datasette()
ds.add_memory_database(db_name)
response = await ds.client.get(""/.json"")
assert db_name in response.json().keys()
path = response.json()[db_name][""path""]
# And the JSON for that database
response2 = await ds.client.get(path + "".json"")
> assert response2.status_code == 200
E assert 302 == 200
E + where 302 = .status_code
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:983: AssertionError
__________________ test_tilde_encoded_database_names[f~/c.d] ___________________
[gw7] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
db_name = 'f~/c.d'
@pytest.mark.asyncio
@pytest.mark.parametrize(""db_name"", (""foo"", r""fo%o"", ""f~/c.d""))
async def test_tilde_encoded_database_names(db_name):
ds = Datasette()
ds.add_memory_database(db_name)
response = await ds.client.get(""/.json"")
assert db_name in response.json().keys()
path = response.json()[db_name][""path""]
# And the JSON for that database
response2 = await ds.client.get(path + "".json"")
> assert response2.status_code == 200
E assert 302 == 200
E + where 302 = .status_code
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:983: AssertionError
______________ test_database_with_space_in_name[/searchable.json] ______________
[gw21] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client_two_attached_databases =
path = '/searchable.json'
@pytest.mark.parametrize(
""path"",
(
""/"",
"".json"",
""/searchable"",
""/searchable.json"",
""/searchable_view"",
""/searchable_view.json"",
),
)
def test_database_with_space_in_name(app_client_two_attached_databases, path):
> response = app_client_two_attached_databases.get(
""/extra~20database"" + path, follow_redirects=True
)
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
request =
follow_redirects = True
history = [, , , , , , ...]
async def _send_handling_redirects(
self,
request: Request,
follow_redirects: bool,
history: typing.List[Response],
) -> Response:
while True:
if len(history) > self.max_redirects:
> raise TooManyRedirects(
""Exceeded maximum allowed redirects."", request=request
)
E httpx.TooManyRedirects: Exceeded maximum allowed redirects.
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
___________________ test_database_with_space_in_name[.json] ____________________
[gw19] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client_two_attached_databases =
path = '.json'
@pytest.mark.parametrize(
""path"",
(
""/"",
"".json"",
""/searchable"",
""/searchable.json"",
""/searchable_view"",
""/searchable_view.json"",
),
)
def test_database_with_space_in_name(app_client_two_attached_databases, path):
> response = app_client_two_attached_databases.get(
""/extra~20database"" + path, follow_redirects=True
)
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
request =
follow_redirects = True
history = [, , , , , , ...]
async def _send_handling_redirects(
self,
request: Request,
follow_redirects: bool,
history: typing.List[Response],
) -> Response:
while True:
if len(history) > self.max_redirects:
> raise TooManyRedirects(
""Exceeded maximum allowed redirects."", request=request
)
E httpx.TooManyRedirects: Exceeded maximum allowed redirects.
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
______________ test_database_with_space_in_name[/searchable_view] ______________
[gw22] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client_two_attached_databases =
path = '/searchable_view'
@pytest.mark.parametrize(
""path"",
(
""/"",
"".json"",
""/searchable"",
""/searchable.json"",
""/searchable_view"",
""/searchable_view.json"",
),
)
def test_database_with_space_in_name(app_client_two_attached_databases, path):
> response = app_client_two_attached_databases.get(
""/extra~20database"" + path, follow_redirects=True
)
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
request =
follow_redirects = True
history = [, , , , , , ...]
async def _send_handling_redirects(
self,
request: Request,
follow_redirects: bool,
history: typing.List[Response],
) -> Response:
while True:
if len(history) > self.max_redirects:
> raise TooManyRedirects(
""Exceeded maximum allowed redirects."", request=request
)
E httpx.TooManyRedirects: Exceeded maximum allowed redirects.
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
_____________________ test_database_with_space_in_name[/] ______________________
[gw18] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client_two_attached_databases =
path = '/'
@pytest.mark.parametrize(
""path"",
(
""/"",
"".json"",
""/searchable"",
""/searchable.json"",
""/searchable_view"",
""/searchable_view.json"",
),
)
def test_database_with_space_in_name(app_client_two_attached_databases, path):
> response = app_client_two_attached_databases.get(
""/extra~20database"" + path, follow_redirects=True
)
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
request =
follow_redirects = True
history = [, , , , , , ...]
async def _send_handling_redirects(
self,
request: Request,
follow_redirects: bool,
history: typing.List[Response],
) -> Response:
while True:
if len(history) > self.max_redirects:
> raise TooManyRedirects(
""Exceeded maximum allowed redirects."", request=request
)
E httpx.TooManyRedirects: Exceeded maximum allowed redirects.
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
________________ test_database_with_space_in_name[/searchable] _________________
[gw20] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client_two_attached_databases =
path = '/searchable'
@pytest.mark.parametrize(
""path"",
(
""/"",
"".json"",
""/searchable"",
""/searchable.json"",
""/searchable_view"",
""/searchable_view.json"",
),
)
def test_database_with_space_in_name(app_client_two_attached_databases, path):
> response = app_client_two_attached_databases.get(
""/extra~20database"" + path, follow_redirects=True
)
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
request =
follow_redirects = True
history = [, , , , , , ...]
async def _send_handling_redirects(
self,
request: Request,
follow_redirects: bool,
history: typing.List[Response],
) -> Response:
while True:
if len(history) > self.max_redirects:
> raise TooManyRedirects(
""Exceeded maximum allowed redirects."", request=request
)
E httpx.TooManyRedirects: Exceeded maximum allowed redirects.
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
___________ test_database_with_space_in_name[/searchable_view.json] ____________
[gw23] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client_two_attached_databases =
path = '/searchable_view.json'
@pytest.mark.parametrize(
""path"",
(
""/"",
"".json"",
""/searchable"",
""/searchable.json"",
""/searchable_view"",
""/searchable_view.json"",
),
)
def test_database_with_space_in_name(app_client_two_attached_databases, path):
> response = app_client_two_attached_databases.get(
""/extra~20database"" + path, follow_redirects=True
)
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_api.py:920:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:223: in __call__
return call_result.result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:438: in result
return self.__get_result()
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/concurrent/futures/_base.py:390: in __get_result
raise self._exception
/gnu/store/mcclmphjgbrgpa0v037a4nlq336482g8-python-asgiref-3.4.1/lib/python3.9/site-packages/asgiref/sync.py:292: in main_wrap
result = await self.awaitable(*args, **kwargs)
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:66: in get
return await self._request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:156: in _request
httpx_response = await self.ds.client.request(
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/app.py:1602: in request
return await client.request(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1527: in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1614: in send
response = await self._send_handling_auth(
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1642: in _send_handling_auth
response = await self._send_handling_redirects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
request =
follow_redirects = True
history = [, , , , , , ...]
async def _send_handling_redirects(
self,
request: Request,
follow_redirects: bool,
history: typing.List[Response],
) -> Response:
while True:
if len(history) > self.max_redirects:
> raise TooManyRedirects(
""Exceeded maximum allowed redirects."", request=request
)
E httpx.TooManyRedirects: Exceeded maximum allowed redirects.
/gnu/store/bj5lb299rfb4cbbq5kczq9imdk9a7y64-python-httpx-0.23.0/lib/python3.9/site-packages/httpx/_client.py:1672: TooManyRedirects
________________ test_weird_database_names[database (1).sqlite] ________________
[gw7] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
tmpdir = local('/tmp/guix-build-datasette-0.64.2.drv-0/pytest-of-nixbld/pytest-0/popen-gw7/test_weird_database_names_data0')
filename = 'database (1).sqlite'
@pytest.mark.parametrize(
""filename"", [""test-database (1).sqlite"", ""database (1).sqlite""]
)
def test_weird_database_names(tmpdir, filename):
# https://github.com/simonw/datasette/issues/1181
runner = CliRunner()
db_path = str(tmpdir / filename)
sqlite3.connect(db_path).execute(""vacuum"")
result1 = runner.invoke(cli, [db_path, ""--get"", ""/""])
assert result1.exit_code == 0, result1.output
filename_no_stem = filename.rsplit(""."", 1)[0]
expected_link = '{} '.format(
tilde_encode(filename_no_stem), filename_no_stem
)
assert expected_link in result1.output
# Now try hitting that database page
result2 = runner.invoke(
cli, [db_path, ""--get"", ""/{}"".format(tilde_encode(filename_no_stem))]
)
> assert result2.exit_code == 0, result2.output
E AssertionError:
E
E assert 1 == 0
E + where 1 = .exit_code
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_cli.py:321: AssertionError
_____________ test_weird_database_names[test-database (1).sqlite] ______________
[gw6] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
tmpdir = local('/tmp/guix-build-datasette-0.64.2.drv-0/pytest-of-nixbld/pytest-0/popen-gw6/test_weird_database_names_test0')
filename = 'test-database (1).sqlite'
@pytest.mark.parametrize(
""filename"", [""test-database (1).sqlite"", ""database (1).sqlite""]
)
def test_weird_database_names(tmpdir, filename):
# https://github.com/simonw/datasette/issues/1181
runner = CliRunner()
db_path = str(tmpdir / filename)
sqlite3.connect(db_path).execute(""vacuum"")
result1 = runner.invoke(cli, [db_path, ""--get"", ""/""])
assert result1.exit_code == 0, result1.output
filename_no_stem = filename.rsplit(""."", 1)[0]
expected_link = '{} '.format(
tilde_encode(filename_no_stem), filename_no_stem
)
assert expected_link in result1.output
# Now try hitting that database page
result2 = runner.invoke(
cli, [db_path, ""--get"", ""/{}"".format(tilde_encode(filename_no_stem))]
)
> assert result2.exit_code == 0, result2.output
E AssertionError:
E
E assert 1 == 0
E + where 1 = .exit_code
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_cli.py:321: AssertionError
_ test_row_html_compound_primary_key[/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd-expected1] _
[gw11] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
path = '/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd'
expected = [['a/b ', '.c-d ', 'c ']]
@pytest.mark.parametrize(
""path,expected"",
(
(
""/fixtures/compound_primary_key/a,b"",
[
[
'a ',
'b ',
'c ',
]
],
),
(
""/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd"",
[
[
'a/b ',
'.c-d ',
'c ',
]
],
),
),
)
def test_row_html_compound_primary_key(app_client, path, expected):
response = app_client.get(path)
> assert response.status == 200
E assert 302 == 200
E + where 302 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:370: AssertionError
_ test_css_classes_on_body[/fixtures/table~2Fwith~2Fslashes~2Ecsv-expected_classes5] _
[gw3] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
path = '/fixtures/table~2Fwith~2Fslashes~2Ecsv'
expected_classes = ['table', 'db-fixtures', 'table-tablewithslashescsv-fa7563']
@pytest.mark.parametrize(
""path,expected_classes"",
[
(""/"", [""index""]),
(""/fixtures"", [""db"", ""db-fixtures""]),
(""/fixtures?sql=select+1"", [""query"", ""db-fixtures""]),
(
""/fixtures/simple_primary_key"",
[""table"", ""db-fixtures"", ""table-simple_primary_key""],
),
(
""/fixtures/neighborhood_search"",
[""query"", ""db-fixtures"", ""query-neighborhood_search""],
),
(
""/fixtures/table~2Fwith~2Fslashes~2Ecsv"",
[""table"", ""db-fixtures"", ""table-tablewithslashescsv-fa7563""],
),
(
""/fixtures/simple_primary_key/1"",
[""row"", ""db-fixtures"", ""table-simple_primary_key""],
),
],
)
def test_css_classes_on_body(app_client, path, expected_classes):
response = app_client.get(path)
> assert response.status == 200
E assert 302 == 200
E + where 302 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:238: AssertionError
_ test_templates_considered[/fixtures/table~2Fwith~2Fslashes~2Ecsv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html] _
[gw3] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
path = '/fixtures/table~2Fwith~2Fslashes~2Ecsv'
expected_considered = 'table-fixtures-tablewithslashescsv-fa7563.html, *table.html'
@pytest.mark.parametrize(
""path,expected_considered"",
[
(""/"", ""*index.html""),
(""/fixtures"", ""database-fixtures.html, *database.html""),
(
""/fixtures/simple_primary_key"",
""table-fixtures-simple_primary_key.html, *table.html"",
),
(
""/fixtures/table~2Fwith~2Fslashes~2Ecsv"",
""table-fixtures-tablewithslashescsv-fa7563.html, *table.html"",
),
(
""/fixtures/simple_primary_key/1"",
""row-fixtures-simple_primary_key.html, *row.html"",
),
],
)
def test_templates_considered(app_client, path, expected_considered):
response = app_client.get(path)
> assert response.status == 200
E assert 302 == 200
E + where 302 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:264: AssertionError
_ test_alternate_url_json[/fixtures/table~2Fwith~2Fslashes~2Ecsv-http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json] _
[gw21] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
path = '/fixtures/table~2Fwith~2Fslashes~2Ecsv'
expected = 'http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json'
@pytest.mark.parametrize(
""path,expected"",
(
# Instance index page
(""/"", ""http://localhost/.json""),
# Table page
(""/fixtures/facetable"", ""http://localhost/fixtures/facetable.json""),
(
""/fixtures/table~2Fwith~2Fslashes~2Ecsv"",
""http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json"",
),
# Row page
(
""/fixtures/no_primary_key/1"",
""http://localhost/fixtures/no_primary_key/1.json"",
),
# Database index page
(
""/fixtures"",
""http://localhost/fixtures.json"",
),
# Custom query page
(
""/fixtures?sql=select+*+from+facetable"",
""http://localhost/fixtures.json?sql=select+*+from+facetable"",
),
# Canned query page
(
""/fixtures/neighborhood_search?text=town"",
""http://localhost/fixtures/neighborhood_search.json?text=town"",
),
# /-/ pages
(
""/-/plugins"",
""http://localhost/-/plugins.json"",
),
),
)
def test_alternate_url_json(app_client, path, expected):
response = app_client.get(path)
> assert response.status == 200
E assert 302 == 200
E + where 302 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:948: AssertionError
_ test_edit_sql_link_on_canned_queries[/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC-/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B] _
[gw18] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
path = '/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC'
expected = '/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B'
@pytest.mark.parametrize(
""path,expected"",
[
(
""/fixtures/neighborhood_search"",
""/fixtures?sql=%0Aselect+_neighborhood%2C+facet_cities.name%2C+state%0Afrom+facetable%0A++++join+facet_cities%0A++++++++on+facetable._city_id+%3D+facet_cities.id%0Awhere+_neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0Aorder+by+_neighborhood%3B%0A&text="",
),
(
""/fixtures/neighborhood_search?text=ber"",
""/fixtures?sql=%0Aselect+_neighborhood%2C+facet_cities.name%2C+state%0Afrom+facetable%0A++++join+facet_cities%0A++++++++on+facetable._city_id+%3D+facet_cities.id%0Awhere+_neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0Aorder+by+_neighborhood%3B%0A&text=ber"",
),
(""/fixtures/pragma_cache_size"", None),
(
# /fixtures/𝐜𝐢𝐭𝐢𝐞𝐬
""/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC"",
""/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B"",
),
(""/fixtures/magic_parameters"", None),
],
)
def test_edit_sql_link_on_canned_queries(app_client, path, expected):
response = app_client.get(path)
> assert response.status == 200
E assert 302 == 200
E + where 302 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_html.py:841: AssertionError
_______________________ test_table_with_slashes_in_name ________________________
[gw9] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
def test_table_with_slashes_in_name(app_client):
response = app_client.get(
""/fixtures/table~2Fwith~2Fslashes~2Ecsv.json?_shape=objects""
)
> assert response.status == 200
E assert 302 == 200
E + where 302 = .status
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:141: AssertionError
__________________ test_custom_query_with_unicode_characters ___________________
[gw8] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
def test_custom_query_with_unicode_characters(app_client):
# /fixtures/𝐜𝐢𝐭𝐢𝐞𝐬.json
response = app_client.get(
""/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC.json?_shape=array""
)
> assert [{""id"": 1, ""name"": ""San Francisco""}] == response.json
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:1042:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/tmp/guix-build-datasette-0.64.2.drv-0/source/datasette/utils/testing.py:40: in json
return json.loads(self.text)
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/json/__init__.py:346: in loads
return _default_decoder.decode(s)
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/json/decoder.py:337: in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = , s = '', idx = 0
def raw_decode(self, s, idx=0):
""""""Decode a JSON document from ``s`` (a ``str`` beginning with
a JSON document) and return a 2-tuple of the Python
representation and the index in ``s`` where the document ended.
This can be used to decode a JSON document from a string that may
have extraneous data at the end.
""""""
try:
obj, end = self.scan_once(s, idx)
except StopIteration as err:
> raise JSONDecodeError(""Expecting value"", s, err.value) from None
E json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
/gnu/store/65i3nhcwmz0p8rqbg48gaavyky4g4hwk-python-3.9.9/lib/python3.9/json/decoder.py:355: JSONDecodeError
_ test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3] _
[gw13] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
app_client =
path = '/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw'
expected_rows = [[1, 'barry cat', 'terry dog', 'panther'], [2, 'terry dog', 'sara weasel', 'puma']]
@pytest.mark.parametrize(
""path,expected_rows"",
[
(
""/fixtures/searchable.json?_search=dog"",
[
[1, ""barry cat"", ""terry dog"", ""panther""],
[2, ""terry dog"", ""sara weasel"", ""puma""],
],
),
(
# Special keyword shouldn't break FTS query
""/fixtures/searchable.json?_search=AND"",
[],
),
(
# Without _searchmode=raw this should return no results
""/fixtures/searchable.json?_search=te*+AND+do*"",
[],
),
(
# _searchmode=raw
""/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw"",
[
[1, ""barry cat"", ""terry dog"", ""panther""],
[2, ""terry dog"", ""sara weasel"", ""puma""],
],
),
(
# _searchmode=raw combined with _search_COLUMN
""/fixtures/searchable.json?_search_text2=te*&_searchmode=raw"",
[
[1, ""barry cat"", ""terry dog"", ""panther""],
],
),
(
""/fixtures/searchable.json?_search=weasel"",
[[2, ""terry dog"", ""sara weasel"", ""puma""]],
),
(
""/fixtures/searchable.json?_search_text2=dog"",
[[1, ""barry cat"", ""terry dog"", ""panther""]],
),
(
""/fixtures/searchable.json?_search_name%20with%20.%20and%20spaces=panther"",
[[1, ""barry cat"", ""terry dog"", ""panther""]],
),
],
)
def test_searchable(app_client, path, expected_rows):
response = app_client.get(path)
> assert expected_rows == response.json[""rows""]
E AssertionError: assert [[1, 'barry cat', 'terry dog', 'panther'],\n [2, 'terry dog', 'sara weasel', 'puma']] == []
E Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']
E Full diff:
E [
E - ,
E + [1,
E + 'barry cat',
E + 'terry dog',
E + 'panther'],
E + [2,
E + 'terry dog',
E + 'sara weasel',
E + 'puma'],
E ]
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:402: AssertionError
_____ test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1] ______
[gw20] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
table_metadata = {'searchmode': 'raw'}, querystring = '_search=te*+AND+do*'
expected_rows = [[1, 'barry cat', 'terry dog', 'panther'], [2, 'terry dog', 'sara weasel', 'puma']]
@pytest.mark.parametrize(
""table_metadata,querystring,expected_rows"",
[
(
{},
""_search=te*+AND+do*"",
[],
),
(
{""searchmode"": ""raw""},
""_search=te*+AND+do*"",
_SEARCHMODE_RAW_RESULTS,
),
(
{},
""_search=te*+AND+do*&_searchmode=raw"",
_SEARCHMODE_RAW_RESULTS,
),
# Can be over-ridden with _searchmode=escaped
(
{""searchmode"": ""raw""},
""_search=te*+AND+do*&_searchmode=escaped"",
[],
),
],
)
def test_searchmode(table_metadata, querystring, expected_rows):
with make_app_client(
metadata={""databases"": {""fixtures"": {""tables"": {""searchable"": table_metadata}}}}
) as client:
response = client.get(""/fixtures/searchable.json?"" + querystring)
> assert expected_rows == response.json[""rows""]
E AssertionError: assert [[1, 'barry cat', 'terry dog', 'panther'],\n [2, 'terry dog', 'sara weasel', 'puma']] == []
E Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']
E Full diff:
E [
E - ,
E + [1,
E + 'barry cat',
E + 'terry dog',
E + 'panther'],
E + [2,
E + 'terry dog',
E + 'sara weasel',
E + 'puma'],
E ]
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:442: AssertionError
_ test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2] _
[gw20] linux -- Python 3.9.9 /gnu/store/slsh0qjv5j68xda2bb6h8gsxwyi1j25a-python-wrapper-3.9.9/bin/python
table_metadata = {}, querystring = '_search=te*+AND+do*&_searchmode=raw'
expected_rows = [[1, 'barry cat', 'terry dog', 'panther'], [2, 'terry dog', 'sara weasel', 'puma']]
@pytest.mark.parametrize(
""table_metadata,querystring,expected_rows"",
[
(
{},
""_search=te*+AND+do*"",
[],
),
(
{""searchmode"": ""raw""},
""_search=te*+AND+do*"",
_SEARCHMODE_RAW_RESULTS,
),
(
{},
""_search=te*+AND+do*&_searchmode=raw"",
_SEARCHMODE_RAW_RESULTS,
),
# Can be over-ridden with _searchmode=escaped
(
{""searchmode"": ""raw""},
""_search=te*+AND+do*&_searchmode=escaped"",
[],
),
],
)
def test_searchmode(table_metadata, querystring, expected_rows):
with make_app_client(
metadata={""databases"": {""fixtures"": {""tables"": {""searchable"": table_metadata}}}}
) as client:
response = client.get(""/fixtures/searchable.json?"" + querystring)
> assert expected_rows == response.json[""rows""]
E AssertionError: assert [[1, 'barry cat', 'terry dog', 'panther'],\n [2, 'terry dog', 'sara weasel', 'puma']] == []
E Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']
E Full diff:
E [
E - ,
E + [1,
E + 'barry cat',
E + 'terry dog',
E + 'panther'],
E + [2,
E + 'terry dog',
E + 'sara weasel',
E + 'puma'],
E ]
/tmp/guix-build-datasette-0.64.2.drv-0/source/tests/test_table_api.py:442: AssertionError
=========================== short test summary info ============================
FAILED tests/test_api.py::test_row_strange_table_name - assert 400 == 200
FAILED tests/test_api.py::test_database_page_for_database_with_dot_in_name - ...
FAILED tests/test_api.py::test_tilde_encoded_database_names[fo%o] - assert 30...
FAILED tests/test_api.py::test_tilde_encoded_database_names[f~/c.d] - assert ...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable.json]
FAILED tests/test_api.py::test_database_with_space_in_name[.json] - httpx.Too...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable_view]
FAILED tests/test_api.py::test_database_with_space_in_name[/] - httpx.TooMany...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable] - htt...
FAILED tests/test_api.py::test_database_with_space_in_name[/searchable_view.json]
FAILED tests/test_cli.py::test_weird_database_names[database (1).sqlite] - As...
FAILED tests/test_cli.py::test_weird_database_names[test-database (1).sqlite]
FAILED tests/test_html.py::test_row_html_compound_primary_key[/fixtures/compound_primary_key/a~2Fb,~2Ec~2Dd-expected1]
FAILED tests/test_html.py::test_css_classes_on_body[/fixtures/table~2Fwith~2Fslashes~2Ecsv-expected_classes5]
FAILED tests/test_html.py::test_templates_considered[/fixtures/table~2Fwith~2Fslashes~2Ecsv-table-fixtures-tablewithslashescsv-fa7563.html, *table.html]
FAILED tests/test_html.py::test_alternate_url_json[/fixtures/table~2Fwith~2Fslashes~2Ecsv-http://localhost/fixtures/table~2Fwith~2Fslashes~2Ecsv.json]
FAILED tests/test_html.py::test_edit_sql_link_on_canned_queries[/fixtures/~F0~9D~90~9C~F0~9D~90~A2~F0~9D~90~AD~F0~9D~90~A2~F0~9D~90~9E~F0~9D~90~AC-/fixtures?sql=select+id%2C+name+from+facet_cities+order+by+id+limit+1%3B]
FAILED tests/test_table_api.py::test_table_with_slashes_in_name - assert 302 ...
FAILED tests/test_table_api.py::test_custom_query_with_unicode_characters - j...
FAILED tests/test_table_api.py::test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]
FAILED tests/test_table_api.py::test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1]
FAILED tests/test_table_api.py::test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2]
=========== 22 failed, 1049 passed, 3 skipped in 1522.28s (0:25:22) ============
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: ""/gnu/store/ziqwkzz6znb5d3c245xn0cq5ra2ly0w3-python-pytest-7.1.3/bin/pytest"" arguments: (""-vv"" ""-n"" ""24"" ""-m"" ""not serial"") exit-status: 1 term-signal: #f stop-signal: #f>
phase `check' failed after 1523.3 seconds
```
The tests run in a private namespace without internet connectivity, and the Python dependencies are at:
```
python-aiofiles@0.6.0 python-asgi-csrf@0.9 python-asgiref@3.4.1
+ python-beautifulsoup4@4.11.1 python-black@22.3.0 python-click-default-group@1.2.2 python-click@8.1.3
+ python-cogapp@3.3.0 python-httpx@0.23.0 python-hupper@1.10.3 python-itsdangerous@2.0.1
+ python-janus@1.0.0 python-jinja2@3.1.1 python-mergedeep@1.3.4 python-pint@0.20.1 python-pluggy@1.0.0
+ python-pytest-asyncio@0.17.2 python-pytest-runner@5.2 python-pytest-timeout@2.0.2
+ python-pytest-xdist@2.5.0 python-pytest@7.1.3 python-pyyaml@6.0 python-setuptools@64.0.3
+ python-trustme@0.9.0 python-uvicorn@0.17.6
```
With Python 3.9.9.
Thank you!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2048/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
323658641,MDU6SXNzdWUzMjM2NTg2NDE=,262,Add ?_extra= mechanism for requesting extra properties in JSON,9599,simonw,open,0,,,3268330,Datasette 1.0,27,2018-05-16T14:55:42Z,2023-03-29T06:22:22Z,,OWNER,,"Datasette views currently work by creating a set of data that should be returned as JSON, then defining an additional, optional `template_data()` function which is called if the view is being rendered as HTML.
This `template_data()` function calculates extra template context variables which are necessary for the HTML view but should not be included in the JSON.
Example of how that is used today: https://github.com/simonw/datasette/blob/2b79f2bdeb1efa86e0756e741292d625f91cb93d/datasette/views/table.py#L672-L704
With features like Facets in #255 I'm beginning to want to move more items into the `template_data()` - in the case of facets it's the `suggested_facets` array. This saves that feature from being calculated (involving several SQL queries) for the JSON case where it is unlikely to be used.
But... as an API user, I want to still optionally be able to access that information.
Solution: Add a `?_extra=suggested_facets&_extra=table_metadata` argument which can be used to optionally request additional blocks to be added to the JSON API.
Then redefine as many of the current `template_data()` features as extra arguments instead, and teach Datasette to return certain extras by default when rendering templates.
This could allow the JSON representation to be slimmed down further (removing e.g. the `table_definition` and `view_definition` keys) while still making that information available to API users who need it.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/262/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1645098678,PR_kwDOBm6k_c5NIQri,2047,Bump black from 22.12.0 to 23.3.0,49699333,dependabot[bot],closed,0,,,,,0,2023-03-29T06:09:06Z,2023-03-29T06:12:21Z,2023-03-29T06:12:05Z,CONTRIBUTOR,simonw/datasette/pulls/2047,"Bumps [black](https://github.com/psf/black) from 22.12.0 to 23.3.0.
Release notes
Sourced from black's releases .
23.3.0
Highlights
This release fixes a longstanding confusing behavior in Black's GitHub action, where the
version of the action did not determine the version of Black being run (issue #3382 ). In
addition, there is a small bug fix around imports and a number of improvements to the
preview style.
Please try out the
preview style
with black --preview
and tell us your feedback. All changes in the preview style are
expected to become part of Black's stable style in January 2024.
Stable style
Import lines with # fmt: skip
and # fmt: off
no longer have an extra blank line
added when they are right after another import line (#3610 )
Preview style
Add trailing commas to collection literals even if there's a comment after the last
entry (#3393 )
async def
, async for
, and async with
statements are now formatted consistently
compared to their non-async version. (#3609 )
with
statements that contain two context managers will be consistently wrapped in
parentheses (#3589 )
Let string splitters respect East Asian Width
(#3445 )
Now long string literals can be split after East Asian commas and periods (、
U+3001
IDEOGRAPHIC COMMA, 。
U+3002 IDEOGRAPHIC FULL STOP, & ,
U+FF0C FULLWIDTH COMMA)
besides before spaces (#3445 )
For stubs, enforce one blank line after a nested class with a body other than just
...
(#3564 )
Improve handling of multiline strings by changing line split behavior (#1879 )
Parser
Added support for formatting files with invalid type comments (#3594 )
Integrations
Update GitHub Action to use the version of Black equivalent to action's version if
version input is not specified (#3543 )
Fix missing Python binary path in autoload script for vim (#3508 )
Documentation
Document that only the most recent release is supported for security issues;
vulnerabilities should be reported through Tidelift (#3612 )
... (truncated)
Changelog
Sourced from black's changelog .
23.3.0
Highlights
This release fixes a longstanding confusing behavior in Black's GitHub action, where the
version of the action did not determine the version of Black being run (issue #3382 ). In
addition, there is a small bug fix around imports and a number of improvements to the
preview style.
Please try out the
preview style
with black --preview
and tell us your feedback. All changes in the preview style are
expected to become part of Black's stable style in January 2024.
Stable style
Import lines with # fmt: skip
and # fmt: off
no longer have an extra blank line
added when they are right after another import line (#3610 )
Preview style
Add trailing commas to collection literals even if there's a comment after the last
entry (#3393 )
async def
, async for
, and async with
statements are now formatted consistently
compared to their non-async version. (#3609 )
with
statements that contain two context managers will be consistently wrapped in
parentheses (#3589 )
Let string splitters respect East Asian Width
(#3445 )
Now long string literals can be split after East Asian commas and periods (、
U+3001
IDEOGRAPHIC COMMA, 。
U+3002 IDEOGRAPHIC FULL STOP, & ,
U+FF0C FULLWIDTH COMMA)
besides before spaces (#3445 )
For stubs, enforce one blank line after a nested class with a body other than just
...
(#3564 )
Improve handling of multiline strings by changing line split behavior (#1879 )
Parser
Added support for formatting files with invalid type comments (#3594 )
Integrations
Update GitHub Action to use the version of Black equivalent to action's version if
version input is not specified (#3543 )
Fix missing Python binary path in autoload script for vim (#3508 )
Documentation
Document that only the most recent release is supported for security issues;
vulnerabilities should be reported through Tidelift (#3612 )
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.12.0&new-version=23.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2047.org.readthedocs.build/en/2047/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2047/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1534904478,PR_kwDOBm6k_c5HdwRg,1992,Bump blacken-docs from 1.12.1 to 1.13.0,49699333,dependabot[bot],closed,0,,,,,1,2023-01-16T13:05:05Z,2023-03-29T06:11:35Z,2023-03-29T06:11:34Z,CONTRIBUTOR,simonw/datasette/pulls/1992,"Bumps [blacken-docs](https://github.com/asottile/blacken-docs) from 1.12.1 to 1.13.0.
Changelog
Sourced from blacken-docs's changelog .
1.13.0 (2023-01-16)
Note Adam Johnson is new maintainer.
Require Black 22.1.0+.
Add --rst-literal-blocks
option, to also format text in reStructuredText literal blocks, starting with ::
.
Sphinx highlights these with the project’s default language, which defaults to Python.
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=blacken-docs&package-manager=pip&previous-version=1.12.1&new-version=1.13.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1992.org.readthedocs.build/en/1992/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1992/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1528995601,PR_kwDOBm6k_c5HJ55o,1986,Bump sphinx from 6.1.2 to 6.1.3,49699333,dependabot[bot],closed,0,,,,,0,2023-01-11T13:02:36Z,2023-03-29T06:09:50Z,2023-03-29T06:09:49Z,CONTRIBUTOR,simonw/datasette/pulls/1986,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.1.2 to 6.1.3.
Release notes
Sourced from sphinx's releases .
v6.1.3
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 6.1.3 (released Jan 10, 2023)
Bugs fixed
#11116 : Reverted to previous Sphinx 5 node copying method
#11117 : Reverted changes to parallel image processing from Sphinx 6.1.0
#11119 : Supress ValueError
in the linkcheck
builder
Commits
776d01e
Bump to 6.1.3 final
a2e922a
CHANGES for Sphinx 6.1.3
31162a9
Handle exceptions for get_node_source
and get_node_line
dcb4429
Restore Sphinx 5 nodes.Element
copying behaviour
2a7c40d
Undo parallel image changes
7841d3d
Ignore more checks in Ruff 0.0.214
ddbc5b5
Bump version
See full diff in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=6.1.2&new-version=6.1.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1986.org.readthedocs.build/en/1986/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1986/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1566081801,PR_kwDOBm6k_c5JAcGy,2014,Bump black from 22.12.0 to 23.1.0,49699333,dependabot[bot],closed,0,,,,,2,2023-02-01T13:06:16Z,2023-03-29T06:09:14Z,2023-03-29T06:09:12Z,CONTRIBUTOR,simonw/datasette/pulls/2014,"Bumps [black](https://github.com/psf/black) from 22.12.0 to 23.1.0.
Release notes
Sourced from black's releases .
23.1.0
Highlights
This is the first release of 2023, and following our stability policy , it comes with a number of improvements to our stable style, notably improvements to empty line handling and the removal of redundant parentheses in several contexts.
There are also many changes to the preview style; try out black --preview
and give us feedback to help us set the stable style for next year.
In addition to style changes, Black now automatically infers the supported Python versions from your pyproject.toml
file, removing the need to set Black's target versions separately.
Stable style
Introduce the 2023 stable style, which incorporates most aspects of last year's preview style (#3418 ). Specific changes:
Enforce empty lines before classes and functions with sticky leading comments (#3302 ) (22.12.0)
Reformat empty and whitespace-only files as either an empty file (if no newline is present) or as a single newline character (if a newline is present) (#3348 ) (22.12.0)
Correctly handle trailing commas that are inside a line's leading non-nested parens (#3370 ) (22.12.0)
--skip-string-normalization
/ -S
now prevents docstring prefixes from being normalized as expected (#3168 ) (since 22.8.0)
When using --skip-magic-trailing-comma
or -C
, trailing commas are stripped from subscript expressions with more than 1 element (#3209 ) (22.8.0)
Fix a string merging/split issue when a comment is present in the middle of implicitly concatenated strings on its own line (#3227 ) (22.8.0)
Docstring quotes are no longer moved if it would violate the line length limit (#3044 , #3430 ) (22.6.0)
Parentheses around return annotations are now managed (#2990 ) (22.6.0)
Remove unnecessary parentheses around awaited objects (#2991 ) (22.6.0)
Remove unnecessary parentheses in with
statements (#2926 ) (22.6.0)
Remove trailing newlines after code block open (#3035 ) (22.6.0)
Code cell separators #%%
are now standardised to # %%
(#2919 ) (22.3.0)
Remove unnecessary parentheses from except
statements (#2939 ) (22.3.0)
Remove unnecessary parentheses from tuple unpacking in for
loops (#2945 ) (22.3.0)
Avoid magic-trailing-comma in single-element subscripts (#2942 ) (22.3.0)
Fix a crash when a colon line is marked between # fmt: off
and # fmt: on
(#3439 )
Preview style
Format hex codes in unicode escape sequences in string literals (#2916 )
Add parentheses around if
-else
expressions (#2278 )
Improve performance on large expressions that contain many strings (#3467 )
Fix a crash in preview style with assert + parenthesized string (#3415 )
Fix crashes in preview style with walrus operators used in function return annotations and except clauses (#3423 )
Fix a crash in preview advanced string processing where mixed implicitly concatenated regular and f-strings start with an empty span (#3463 )
Fix a crash in preview advanced string processing where a standalone comment is placed before a dict's value (#3469 )
Fix an issue where extra empty lines are added when a decorator has # fmt: skip
applied or there is a standalone comment between decorators (#3470 )
Do not put the closing quotes in a docstring on a separate line, even if the line is too long (#3430 )
Long values in dict literals are now wrapped in parentheses; correspondingly unnecessary parentheses around short values in dict literals are now removed; long string lambda values are now wrapped in parentheses (#3440 )
Fix two crashes in preview style involving edge cases with docstrings (#3451 )
Exclude string type annotations from improved string processing; fix crash when the return type annotation is stringified and spans across multiple lines (#3462 )
Wrap multiple context managers in parentheses when targeting Python 3.9+ (#3489 )
Fix several crashes in preview style with walrus operators used in with
statements or tuples (#3473 )
Fix an invalid quote escaping bug in f-string expressions where it produced invalid code. Implicitly concatenated f-strings with different quotes can now be merged or quote-normalized by changing the quotes used in expressions. (#3509 )
... (truncated)
Changelog
Sourced from black's changelog .
23.1.0
Highlights
This is the first release of 2023, and following our
stability policy ,
it comes with a number of improvements to our stable style, including improvements to
empty line handling, removal of redundant parentheses in several contexts, and output
that highlights implicitly concatenated strings better.
There are also many changes to the preview style; try out black --preview
and give us
feedback to help us set the stable style for next year.
In addition to style changes, Black now automatically infers the supported Python
versions from your pyproject.toml
file, removing the need to set Black's target
versions separately.
Stable style
Introduce the 2023 stable style, which incorporates most aspects of last year's
preview style (#3418 ). Specific changes:
Enforce empty lines before classes and functions with sticky leading comments
(#3302 ) (22.12.0)
Reformat empty and whitespace-only files as either an empty file (if no newline is
present) or as a single newline character (if a newline is present) (#3348 )
(22.12.0)
Implicitly concatenated strings used as function args are now wrapped inside
parentheses (#3307 ) (22.12.0)
Correctly handle trailing commas that are inside a line's leading non-nested parens
(#3370 ) (22.12.0)
--skip-string-normalization
/ -S
now prevents docstring prefixes from being
normalized as expected (#3168 ) (since 22.8.0)
When using --skip-magic-trailing-comma
or -C
, trailing commas are stripped from
subscript expressions with more than 1 element (#3209 ) (22.8.0)
Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside
parentheses (#3162 ) (22.8.0)
Fix a string merging/split issue when a comment is present in the middle of
implicitly concatenated strings on its own line (#3227 ) (22.8.0)
Docstring quotes are no longer moved if it would violate the line length limit
(#3044 , #3430 ) (22.6.0)
Parentheses around return annotations are now managed (#2990 ) (22.6.0)
Remove unnecessary parentheses around awaited objects (#2991 ) (22.6.0)
Remove unnecessary parentheses in with
statements (#2926 ) (22.6.0)
Remove trailing newlines after code block open (#3035 ) (22.6.0)
Code cell separators #%%
are now standardised to # %%
(#2919 ) (22.3.0)
Remove unnecessary parentheses from except
statements (#2939 ) (22.3.0)
Remove unnecessary parentheses from tuple unpacking in for
loops (#2945 ) (22.3.0)
Avoid magic-trailing-comma in single-element subscripts (#2942 ) (22.3.0)
... (truncated)
Commits
b0d1fba
Prepare release 23.1.0 (#3536 )
69ca0a4
Infer target version based on project metadata (#3219 )
c4bd2e3
Draft for Black 2023 stable style (#3418 )
226cbf0
Fix unsafe cast in linegen.py w/ await yield handling (#3533 )
f4ebc68
Upgrade isort (#3534 )
6407ebb
Remove Python version in the_basics.md (#3528 )
196b1f3
Fix black --help
output for --python-cell-magics
option to be reproducibl...
d950f15
Update document now that paren wrapping CMs on Python 3.9+ is implemented (#3 ...
a36878e
Fix an invalid quote escaping bug in f-string expressions (#3509 )
eabff67
Format hex code in unicode escape sequences in string literals (#2916 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.12.0&new-version=23.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2014.org.readthedocs.build/en/2014/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2014/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1644018605,PR_kwDOBm6k_c5NEqBO,2046,Bump furo from 2022.12.7 to 2023.3.27,49699333,dependabot[bot],closed,0,,,,,0,2023-03-28T13:58:14Z,2023-03-29T06:08:02Z,2023-03-29T06:08:01Z,CONTRIBUTOR,simonw/datasette/pulls/2046,"Bumps [furo](https://github.com/pradyunsg/furo) from 2022.12.7 to 2023.3.27.
Changelog
Sourced from furo's changelog .
Changelog
2023.03.27 -- Tasty Tangerine
Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
Add missing class to Font Awesome examples
2023.03.23 -- Sassy Saffron
Update Python version classifiers.
Increase the icon size in mobile header.
Increase admonition title bg opacity.
Change the default API background to transparent.
Transition the API background change.
Remove the "indent" of API entries which have a background.
Break long inline code literals.
2022.12.07 -- Reverent Raspberry
✨ Add support for Sphinx 6.
✨ Improve footnote presentation with docutils 0.18+.
Drop support for Sphinx 4.
Improve documentation about what the edit button does.
Improve handling of empty-flexboxes for better print experience on Chrome.
Improve styling for inline signatures.
Replace the meta
generator tag with a comment.
Tweak labels with icons to prevent users selecting icons as text on touch.
2022.09.29 -- Quaint Quartz
Add ability to set arbitrary URLs for edit button.
Add support for aligning text in MyST-parser generated tables.
2022.09.15 -- Pragmatic Pistachio
Add a minimum version constraint on pygments.
Add an explicit dependency on sass
.
Change right sidebar title from "Contents" to "On this page".
Correctly position sidebars on small screens.
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.12.7&new-version=2023.3.27)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2046.org.readthedocs.build/en/2046/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2046/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1639446870,PR_kwDOBm6k_c5M1izI,2043,Bump furo from 2022.12.7 to 2023.3.23,49699333,dependabot[bot],closed,0,,,,,2,2023-03-24T13:58:08Z,2023-03-28T13:58:24Z,2023-03-28T13:58:21Z,CONTRIBUTOR,simonw/datasette/pulls/2043,"Bumps [furo](https://github.com/pradyunsg/furo) from 2022.12.7 to 2023.3.23.
Changelog
Sourced from furo's changelog .
Changelog
2023.03.23 -- Sassy Saffron
Regenerate with newer version of sphinx-theme-builder, to fix RECORD hashes.
Update Python version classifiers.
Increase the icon size in mobile header.
Increase admonition title bg opacity.
Change the default API background to transparent.
Transition the API background change.
Remove the "indent" of API entries which have a background.
Break long inline code literals.
2022.12.07 -- Reverent Raspberry
✨ Add support for Sphinx 6.
✨ Improve footnote presentation with docutils 0.18+.
Drop support for Sphinx 4.
Improve documentation about what the edit button does.
Improve handling of empty-flexboxes for better print experience on Chrome.
Improve styling for inline signatures.
Replace the meta
generator tag with a comment.
Tweak labels with icons to prevent users selecting icons as text on touch.
2022.09.29 -- Quaint Quartz
Add ability to set arbitrary URLs for edit button.
Add support for aligning text in MyST-parser generated tables.
2022.09.15 -- Pragmatic Pistachio
Add a minimum version constraint on pygments.
Add an explicit dependency on sass
.
Change right sidebar title from "Contents" to "On this page".
Correctly position sidebars on small screens.
Correctly select only Furo's own svg
in related pages nav
.
Make numpy-style documentation headers consistent.
Retitle the reference section.
Update npm dependencies.
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.12.7&new-version=2023.3.23)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--2043.org.readthedocs.build/en/2043/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2043/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1109808154,I_kwDOBm6k_c5CJlQa,1608,Documentation should clarify /stable/ vs /latest/,9599,simonw,closed,0,,,,,15,2022-01-20T22:02:59Z,2023-03-26T23:41:12Z,2022-01-20T22:53:17Z,OWNER,,"It's not currently clear what the difference between https://docs.datasette.io/en/latest/ and https://docs.datasette.io/en/stable/ is - I should fix that.
On Twitter: https://twitter.com/simonw/status/1484285006243528705",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1608/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1641013220,I_kwDOBm6k_c5hz9_k,2045,First column on a view page has no facet option in cog menu,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2023-03-26T18:02:47Z,2023-03-26T18:02:48Z,,OWNER,,"e.g. first column on this page - cog menu has no option to facet.
https://datasette.io/content/tools
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2045/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1639873822,PR_kwDOBm6k_c5M29tt,2044,Expand labels in row view as well (patch for 0.64.x branch),82332573,tmcl-it,open,0,,,,,0,2023-03-24T18:44:44Z,2023-03-24T18:44:57Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2044,"This is a version of #2031 for the 0.64.x branch.
----
:books: Documentation preview :books:: https://datasette--2044.org.readthedocs.build/en/2044/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2044/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1605481359,PR_kwDOBm6k_c5LDwrF,2031,Expand foreign key references in row view as well,82332573,tmcl-it,open,0,,,,,5,2023-03-01T18:43:09Z,2023-03-24T18:35:25Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2031,"Unlike the table view, the single row view does not resolve foreign key references into labels. This patch extracts the foreign key reference expansion code from TableView.data() into a standalone function that is then called by both TableView.data() and RowView.data().
----
:books: Documentation preview :books:: https://datasette--2031.org.readthedocs.build/en/2031/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2031/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1121583414,I_kwDOBm6k_c5C2gE2,1619,JSON link on row page is 404 if base_url setting is used,9599,simonw,open,0,,,,,5,2022-02-02T07:09:53Z,2023-03-24T15:38:04Z,,OWNER,,"On my local environment:
datasette fixtures.db -p 3344 --setting base_url /foo/bar/
Then hit http://127.0.0.1:3344/foo/bar/fixtures/table%2Fwith%2Fslashes.csv/3
But... that `json` link goes here, which is a 404:
http://127.0.0.1:3344/foo/bar/foo/bar/fixtures/table%2Fwith%2Fslashes.csv/3?_format=json",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1619/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1636616315,I_kwDOBm6k_c5hjMh7,2042,Gather feedback on new ?_extra= design,9599,simonw,open,0,,,,,0,2023-03-22T23:07:43Z,2023-03-22T23:08:19Z,,OWNER,,"Now that I've landed:
- #1999
See also:
- #262
I want to get some feedback from people on the design of the new `?_extra=` feature, before freezing it into Datasette 1.0.
The big change is that the default JSON representation is now MUCH slimmer - it only gives you keys for `""next""` and `""rows""`, where rows is a list of JSON objects (not a list of arrays as was previously the default) - for example https://latest.datasette.io/fixtures/sortable.json
If you want extra stuff you can ask for it with the new `?_extra=` parameter - e.g. https://latest.datasette.io/fixtures/sortable.json?_extra=columns&_extra=suggested_facets
You can use `?_extra=extras` to see a list of available extras: https://latest.datasette.io/fixtures/sortable.json?_extra=extras
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2042/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1551694938,PR_kwDOBm6k_c5IQeKz,1999,?_extra= support (draft),9599,simonw,closed,0,,,,,49,2023-01-21T04:55:18Z,2023-03-22T22:49:41Z,2023-03-22T22:49:40Z,OWNER,simonw/datasette/pulls/1999,"Refs:
- #262
----
:books: Documentation preview :books:: https://datasette--1999.org.readthedocs.build/en/1999/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1999/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1633077183,I_kwDOBm6k_c5hVse_,2041,Remove obsolete table POST code,9599,simonw,closed,0,,,8755003,Datasette 1.0a-next,2,2023-03-21T01:01:40Z,2023-03-21T01:17:44Z,2023-03-21T01:17:43Z,OWNER,,"Spotted this in:
- #1999
`POST /db/table` currently executes obsolete code for inserting a row - I replaced that with `/db/table/-/insert` in
https://github.com/simonw/datasette/commit/6e788b49edf4f842c0817f006eb9d865778eea5e but forgot to remove the old code.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2041/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1620515757,I_kwDOBm6k_c5glxut,2039,Subtle bug with `--load-extension` and `--static` flags with absolute Windows paths with`C:\`,15178711,asg017,open,0,,,,,0,2023-03-12T21:18:52Z,2023-03-12T21:18:52Z,,CONTRIBUTOR,,"From the Datasette discord: A user tried running the following command on windows:
```
datasette --load-extension=""C:\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll""
```
This failed with `""The specified module could not be found""`, because the entrypoint option introduced in #1789 splits the input differently. Instead of loading the extension found at `""C:\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll""`, it instead tried to load the extension at `""C""` with entrypoint `""\spatialite\mod_spatialite-5.0.1-win-x86\mod_spatialite.dll"".
This is hard because most absolute windows paths have a colon in them, like `C:\foo.txt` or `D:\bar.txt`. I'd image the `--static` flag is also vulnerable to this type of bug.
The ""solution"" is to use a relative path instead, but that doesn't feel that great. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2039/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
317001500,MDU6SXNzdWUzMTcwMDE1MDA=,236,datasette publish lambda plugin,9599,simonw,open,0,,,,,11,2018-04-23T22:10:30Z,2023-03-12T14:04:15Z,,OWNER,,"Refs #217 - create a publish plugin that can deploy to AWS Lambda.
https://docs.aws.amazon.com/lambda/latest/dg/limits.html says lambda packages can be up to 50 MB, so this would only work with smaller databases (the command can check the filesize before attempting to package and deploy it).
Lambdas do get a 512 MB `/tmp` directory too, so for larger databases the function could start and then download up to 512MB from an S3 bucket - so the plugin could take an optional S3 bucket to write to and know how to upload the `.db` file there and then have the lambda download it on startup.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/236/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1618249044,I_kwDOBm6k_c5gdIVU,2038,Consider a `strict_templates` setting,9599,simonw,open,0,,,,,2,2023-03-10T02:09:13Z,2023-03-10T02:11:06Z,,OWNER,,"A setting which turns on Jinja strict mode, so any templates that access undefined variables raise a hard error.
Prototype here:
```diff
diff --git a/datasette/app.py b/datasette/app.py
index 40416713..1428a3f0 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -200,6 +200,7 @@ SETTINGS = (
""Allow display of SQL trace debug information with ?_trace=1"",
),
Setting(""base_url"", ""/"", ""Datasette URLs should use this base path""),
+ Setting(""strict_templates"", False, ""Raise errors for undefined template variables""),
)
_HASH_URLS_REMOVED = ""The hash_urls setting has been removed, try the datasette-hashed-urls plugin instead""
OBSOLETE_SETTINGS = {
@@ -399,11 +400,14 @@ class Datasette:
),
]
)
+ env_extras = {}
+ if self.setting(""strict_templates""):
+ env_extras[""undefined""] = StrictUndefined
self.jinja_env = Environment(
loader=template_loader,
autoescape=True,
enable_async=True,
- undefined=StrictUndefined,
+ **env_extras,
)
self.jinja_env.filters[""escape_css_string""] = escape_css_string
self.jinja_env.filters[""quote_plus""] = urllib.parse.quote_plus
```
Explored this idea a bit in:
- #1999",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2038/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1615891776,I_kwDOBm6k_c5gUI1A,2037,Test failure: FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError,9599,simonw,closed,0,,,,,3,2023-03-08T20:30:06Z,2023-03-09T22:33:39Z,2023-03-09T22:33:39Z,OWNER,,"> FAILED tests/test_cli.py::test_install_requirements - FileNotFoundError: [Errno 2] No such file or directory
From https://github.com/simonw/datasette/actions/runs/4348548218/jobs/7597208191
```
=================================== FAILURES ===================================
__________________________ test_install_requirements ___________________________
run_module =
@mock.patch(""datasette.cli.run_module"")
def test_install_requirements(run_module):
runner = CliRunner()
> with runner.isolated_filesystem():
/home/runner/work/datasette/datasette/tests/test_cli.py:184:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/contextlib.py:119: in __enter__
return next(self.gen)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = , temp_dir = None
@contextlib.contextmanager
def isolated_filesystem(
self, temp_dir: t.Optional[t.Union[str, os.PathLike]] = None
) -> t.Iterator[str]:
""""""A context manager that creates a temporary directory and
changes the current working directory to it. This isolates tests
that affect the contents of the CWD to prevent them from
interfering with each other.
:param temp_dir: Create the temporary directory under this
directory. If given, the created directory is not removed
when exiting.
.. versionchanged:: 8.0
Added the ``temp_dir`` parameter.
""""""
> cwd = os.getcwd()
E FileNotFoundError: [Errno 2] No such file or directory
/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/testing.py:466: FileNotFoundError
```
Not sure why it only affected the ""[Calculate test coverage](https://github.com/simonw/datasette/actions/workflows/test-coverage.yml)"" one.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2037/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1615862295,I_kwDOBm6k_c5gUBoX,2036,"`publish cloudrun` reuses image tags, which can lead to very surprising deploy problems",9599,simonw,closed,0,,,,,6,2023-03-08T20:11:44Z,2023-03-08T20:57:34Z,2023-03-08T20:57:34Z,OWNER,,"See this issue:
- https://github.com/simonw/datasette.io/issues/141",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2036/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1590183272,I_kwDOBm6k_c5eyEVo,2027,"How to redirect from ""/"" to a specific db/table",1350673,dmick,open,0,,,,,4,2023-02-18T03:14:01Z,2023-03-08T04:42:22Z,,NONE,,"Using nginx to redirect public IP to the local uvicorn server as 'normal'. I can't figure out how to redirect such that '/' results in accessing the one db/table I want to serve; redirecting / to /db/table breaks some of the CSS; fooling with base_url doesn't seem to help. Can someone explain this, if it's possible?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2027/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1612296210,I_kwDOBm6k_c5gGbAS,2033,`datasette install -r requirements.txt`,9599,simonw,closed,0,,,,,2,2023-03-06T22:17:17Z,2023-03-06T22:54:52Z,2023-03-06T22:27:34Z,OWNER,,"Would be useful for cases where you want to install a whole set of plugins in one go, e.g. when running tutorials in GitHub Codespaces.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2033/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1590839187,PR_kwDOBm6k_c5KSs9T,2028,add Python 3.11 classifier,614233,dtrodrigues,closed,0,,,,,2,2023-02-19T20:16:03Z,2023-03-06T21:01:20Z,2023-03-06T21:01:19Z,CONTRIBUTOR,simonw/datasette/pulls/2028,"Python 3.11 is tested in CI and is used in the docker image, so add the Python 3.11 Trove classifier.
----
:books: Documentation preview :books:: https://datasette--2028.org.readthedocs.build/en/2028/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2028/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1605959201,I_kwDOBm6k_c5fuP4h,2032,datasette errors when foreign key integrity is enabled,193185,cldellow,open,0,,,,,0,2023-03-02T01:27:51Z,2023-03-02T01:31:58Z,,CONTRIBUTOR,,"By default, [SQLite does not enforce foreign key constraints](https://www.sqlite.org/foreignkeys.html#fk_enable). I typically enable these checks by running:
```sql
PRAGMA foreign_keys = ON;
```
inside of a `prepare_connection` hook.
If a plugin causes the schema to change (eg datasette-scraper creating a new table, or datasette-edit-schema changing a column), then https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/internal_db.py#L71-L77 will fail with:
```
FOREIGN KEY constraint failed
```
This could be resolved by either:
- deleting from the `tables` column last
- changing the schema so that the foreign keys have [ON DELETE CASCADE](https://www.sqlite.org/foreignkeys.html#fk_actions)
Let me know if you'd be open to a PR that addresses this -- since foreign key constraints aren't enabled by default, I guess it's questionable whether this is a bug. I think I can workaround this by inspecting the database parameter in `prepare_connection` and trying not to enable fkey checks on the `_internal` database.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2032/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1594383280,I_kwDOBm6k_c5fCFuw,2030,How to use Datasette with apache webserver on GCP?,19700859,gk7279,closed,0,,,,,2,2023-02-22T03:08:49Z,2023-02-22T21:54:39Z,2023-02-22T21:54:39Z,NONE,,"Hi Simon and Datasette team-
I have installed apache2 webserver inside GCP VM using apt.
I can see my ""Hello World"" index.html if I use the external IP of this GCP in a browser.
However, when I try to run datasette with different combinations of -h and -p, I am still unable to access the webpage.
I cannot invest Docker on this VM.
Any pointers to use datasette with already existing apache2 webserver on GCP is appreciated.
Thanks.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2030/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
828858421,MDU6SXNzdWU4Mjg4NTg0MjE=,1258,Allow canned query params to specify default values,1385831,wdccdw,open,0,,,,,5,2021-03-11T07:19:02Z,2023-02-20T23:39:58Z,,NONE,,"If I call a canned query that includes named parameters, without passing any parameters, datasette runs the query anyway, resulting in an HTTP status code 400, and a visible error in the browser, with only a link back to home. This means that one of the default links on https://site/database/ will lead to a broken page with no apparent way out.
![image](https://user-images.githubusercontent.com/1385831/110748683-13e72300-820e-11eb-855c-32e03dfef5bf.png)
Is there any way to skip performing the query when parameters aren't supplied, but otherwise render the usual canned query page? Alternatively, can I supply default values for my parameters, either when defining my canned queries or when linking to the canned query page from the default database template.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1258/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1592327343,I_kwDOBm6k_c5e6Pyv,2029,"Sorry Simon, didn't know how else to contact you",5804626,llchristopherson,open,0,,,,,0,2023-02-20T19:02:53Z,2023-02-20T19:02:53Z,,NONE,,"Hi Simon,
Would you be willing to chat with me about Datasette? I have some questions. I am working on a project to evaluate data ingestion tools for a research organization and I ran across Datasette. I have looked through a lot of your documentation, but still have some questions, which are very specific. If you would be willing to write me back about this, my email is laura@renci.org.
Thanks,
Laura",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2029/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1586980089,PR_kwDOBm6k_c5KF-by,2026,Avoid repeating primary key columns if included in _col args,8513,runderwood,open,0,,,,,0,2023-02-16T04:16:25Z,2023-02-16T04:16:41Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2026,"...while maintaining given order.
Fixes #1975 (if I'm understanding correctly).
----
:books: Documentation preview :books:: https://datasette--2026.org.readthedocs.build/en/2026/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2026/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1581218043,PR_kwDOBm6k_c5JyqPy,2025,Add database metadata to index.html template context,9993,palewire,open,0,,,,,0,2023-02-12T11:16:58Z,2023-02-12T11:17:14Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/2025,"Fixes #2016
----
:books: Documentation preview :books:: https://datasette--2025.org.readthedocs.build/en/2025/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2025/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1323346408,I_kwDOBm6k_c5O4Kno,1775,i18n support,428820,johnfelipe,open,0,,,,,9,2022-07-31T02:51:04Z,2023-02-10T18:04:40Z,,NONE,,"I want contribute for translate UI to es, de, de and it if you share strings",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1775/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1579973223,I_kwDOBm6k_c5eLHpn,2024,Mention WAL mode in documentation,9599,simonw,open,0,,,,,1,2023-02-10T16:11:10Z,2023-02-10T16:11:53Z,,OWNER,,It's not currently obvious from the docs how you can ensure that Datasette runs well in situations where other processes may update the underlying SQLite files.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2024/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1579695809,I_kwDOBm6k_c5eKD7B,2023,Error: Invalid setting 'hash_urls' in settings.json in 0.64.1,80409402,mlaparie,closed,0,,,,,2,2023-02-10T13:35:01Z,2023-02-10T15:40:00Z,2023-02-10T15:39:59Z,NONE,,"On a Debian machine, using datasette 0.64.1 installed with `pip3`, I am getting a `datasette[114272]: Error: Invalid setting 'hash_urls' in settings.json` in `journalctl -xe`. The same settings work on 0.54.1 on another Debian server.
This is my `settings.json`:
```json
{
""default_page_size"": 200,
""max_returned_rows"": 8000,
""num_sql_threads"": 3,
""sql_time_limit_ms"": 1000,
""default_facet_size"": 30,
""facet_time_limit_ms"": 200,
""facet_suggest_time_limit_ms"": 50,
""hash_urls"": false,
""allow_facet"": true,
""allow_download"": true,
""suggest_facets"": true,
""default_cache_ttl"": 5,
""default_cache_ttl_hashed"": 31536000,
""cache_size_kb"": 0,
""allow_csv_stream"": true,
""max_csv_mb"": 100,
""truncate_cells_html"": 2048,
""force_https_urls"": false,
""template_debug"": false,
""base_url"": ""/pclim/db/""
}
```
This looks ok to me. Would you have any ideas?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2023/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1578609658,I_kwDOBm6k_c5eF6v6,2022,Error 500 - not clear the cause,1667631,DavidPratten,closed,0,,,,,1,2023-02-09T20:57:17Z,2023-02-09T21:13:50Z,2023-02-09T21:13:50Z,NONE,,"On the database that I have sent via linkedIn, datasette works great, but the following URL gives a 500 error.
http://127.0.0.1:8001/literature/authors_papers?authorId=100550354
The cause of the error is not apparent.
Is this expected behaviour?
David",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2022/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1577548579,I_kwDOBm6k_c5eB3sj,2021,Docker images for 1.0 alphas?,1563881,meowcat,open,0,,,,,0,2023-02-09T09:35:52Z,2023-02-09T09:35:52Z,,NONE,,"Hi,
would you consider putting 1.0alpha images on Dockerhub?
(Also, how usable are the alphas?)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2021/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1575880841,I_kwDOBm6k_c5d7giJ,2020,"Documentation refers to ""off"" setting; doesn't seem to work, ""false"" does",1350673,dmick,open,0,,,,,0,2023-02-08T10:38:10Z,2023-02-08T10:38:10Z,,NONE,,"https://docs.datasette.io/en/stable/settings.html#suggest-facets, among others, suggests using ""off"" to disable the setting; however, this doesn't appear to work in the JSON config files, where it apparently needs to be a ""JSON boolean"" and have the values ""true"" or ""false"". Perhaps the Python code is more flexible?...but either way, the documentation probably should mention it.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2020/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1573424830,I_kwDOBm6k_c5dyI6-,2019,Refactor out the keyset pagination code,9599,simonw,open,0,,,,,14,2023-02-06T23:04:00Z,2023-02-08T01:40:46Z,,OWNER,,"While working on:
- #1999
I noticed that some of the most complex code in the existing table view is the code that implements keyset pagination:
https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/views/table.py#L417-L493
Extracting that into a utility function would simplify that code a lot.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2019/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
473288428,MDExOlB1bGxSZXF1ZXN0MzAxNDgzNjEz,564,First proof-of-concept of Datasette Library,9599,simonw,open,0,,,,,1,2019-07-26T10:22:26Z,2023-02-07T15:14:11Z,,OWNER,simonw/datasette/pulls/564,"Refs #417. Run it like this:
datasette -d ~/Library
Uses a new plugin hook - available_databases()
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/564/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1,
1571711808,I_kwDOBm6k_c5drmtA,2018,`check_visibility` gives confusing (wrong?) results if permission is `None`,193185,cldellow,open,0,,,,,0,2023-02-06T01:03:08Z,2023-02-06T01:03:46Z,,CONTRIBUTOR,,"I'm trying to gate access to an edit UI on the user having `update-row` on the underlying view or table.
I expected [datasette.check_visibility](https://docs.datasette.io/en/latest/internals.html#await-check-visibility-actor-action-none-resource-none-permissions-none) to be a good way to do this:
```python
visible, private = await datasette.check_visibility(
request.actor,
permissions=[
(""update-row"", (database, table)),
],
)
if not visible:
return None
```
But `visible` is returning true, even when there is no explicit `update-row` permission. (In this case, `request.actor` is `None`.)
Based on [the update-row permissions docs](https://docs.datasette.io/en/latest/authentication.html#update-row), I expected this to be default deny, and so no explicit permission would result in false.
I think the root cause is that `check_visibility` calls `ensure_permissions` and expects it to throw if the permission is not available.
But `ensure_permissions` does not throw when `permission_allowed` returns None: https://github.com/simonw/datasette/blob/1.0a2/datasette/app.py#L825-L829",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2018/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1571207083,I_kwDOBm6k_c5dprer,2016,Database metadata fields like description are not available in the index page template's context,9993,palewire,open,0,,,3268330,Datasette 1.0,1,2023-02-05T02:25:53Z,2023-02-05T22:56:43Z,,NONE,,"When looping through `databases` in the index.html template, I'd like to print the description of each database alongside its name. But it appears that isn't passed in from the view, unless I'm missing it. It would be great to have that.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2016/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1556065335,PR_kwDOBm6k_c5Ie5nA,2004,"use single quotes for string literals, fixes #2001",193185,cldellow,open,0,,,,,1,2023-01-25T05:08:46Z,2023-02-01T06:37:18Z,,CONTRIBUTOR,simonw/datasette/pulls/2004,"This modernizes some uses of double quotes for string literals to use only single quotes, fixes simonw/datasette#2001
While developing it, I manually enabled the stricter mode by using the code snippet at https://gist.github.com/cldellow/85bba507c314b127f85563869cd94820
I think that code snippet isn't generally safe/portable, so I haven't tried to automate it in the tests.
----
:books: Documentation preview :books:: https://datasette--2004.org.readthedocs.build/en/2004/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2004/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1565179870,I_kwDOBm6k_c5dSr_e,2013,Datasette uses non-standard quoting for identifiers,193185,cldellow,open,0,,,,,0,2023-02-01T00:05:39Z,2023-02-01T00:06:30Z,,CONTRIBUTOR,,"Related to #2001, but where #2001 was about literals, this is about identifiers
From https://www.sqlite.org/lang_keywords.html:
> ""keyword"" A keyword in double-quotes is an identifier.
> [keyword] A keyword enclosed in square brackets is an identifier. This is not standard SQL. This quoting mechanism is used by MS Access and SQL Server and is included in SQLite for compatibility.
Datasette uses this quoting here -- https://github.com/simonw/datasette/blob/0b4a28691468b5c758df74fa1d72a823813c96bf/datasette/utils/__init__.py#L345-L349, in some of the other DB access code, and in some of the test fixtures.
Migrating to standard double quote identifiers would make it easier to get Datasette working with alternative backends",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2013/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1564774831,I_kwDOBm6k_c5dRJGv,2012,Missing space in database summary,9599,simonw,open,0,,,,,0,2023-01-31T18:01:13Z,2023-01-31T18:01:13Z,,OWNER,,"Spotted this on an instance index page:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2012/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1564769997,I_kwDOBm6k_c5dRH7N,2011,"Applied facet did not result in an ""x"" icon to dismiss it",9599,simonw,open,0,,,,,1,2023-01-31T17:57:44Z,2023-01-31T17:58:54Z,,OWNER,,"![CleanShot 2023-01-31 at 09 55 56@2x](https://user-images.githubusercontent.com/9599/215843684-1761a230-d490-4f87-be6d-186319366794.png)
That's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata
It's for `Contract Type` of `Non-Purchasing Contract (Rents, etc.)` - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2011/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1563264257,I_kwDOBm6k_c5dLYUB,2010,Row page should default to card view,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2023-01-30T21:49:37Z,2023-01-30T21:52:06Z,,OWNER,,"Datasette currently uses the same table layout on the row pages as it does on the table pages:
https://datasette.io/content/pypi_packages?_sort=name&name__exact=datasette-column-inspect
https://datasette.io/content/pypi_packages/datasette-column-inspect
If you shrink down to mobile width you get this instead, on both of those pages:
I think that view, which I think of as the ""card view"", is plain better if you're looking at just a single row - and it (or a variant of it) should be the default presentation on the row page.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2010/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1186696202,I_kwDOBm6k_c5Gu4wK,1696,Show foreign key label when filtering,9599,simonw,open,0,,,,,2,2022-03-30T16:18:54Z,2023-01-29T20:56:20Z,,OWNER,,"For example here:
3 corresponds to ""Human Related: Other"" - it would be neat to display this in this area of the page somehow.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1696/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1560982210,PR_kwDOBm6k_c5IvYKw,2008,array facet: don't materialize unnecessary columns,193185,cldellow,open,0,,,,,8,2023-01-28T19:33:40Z,2023-01-29T18:17:40Z,,CONTRIBUTOR,simonw/datasette/pulls/2008,"The presence of `inner.*` causes SQLite to materialize a row with all the columns. Those columns will be discarded later.
Instead, we can select only the column we'll use. This lets SQLite's optimizer realize that the other columns in the CTE definition aren't needed.
On a test table with 278K rows, 98K of which had an array, this speeds up the facet calculation from 4 sec to 1 sec.
----
:books: Documentation preview :books:: https://datasette--2008.org.readthedocs.build/en/2008/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2008/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1515815014,I_kwDOBm6k_c5aWYBm,1973,render_cell plugin hook's row object is not a sqlite.Row,193185,cldellow,open,0,,,,,4,2023-01-01T20:27:46Z,2023-01-29T00:40:31Z,,CONTRIBUTOR,,"From https://docs.datasette.io/en/stable/plugin_hooks.html#render-cell-row-value-column-table-database-datasette:
> row - sqlite.Row
> The SQLite row object that the value being rendered is part of
This appears to actually be a [CustomRow](https://github.com/simonw/datasette/blob/f0fadc28ddb9f82e5cc1ecaa51e8a342eb6dc528/datasette/utils/__init__.py#L773-L789), but I think that's unrelated to my issue.
I have a table:
```sql
CREATE TABLE IF NOT EXISTS ""dss_job_stats""(
job_id integer not null references dss_job(id) on delete cascade,
host text not null,
// other columns elided as irrelevant
primary key (job_id, host)
);
```
On datasette 0.63.2, the `render_cell` hook receives a `row` value that looks like:
```
CustomRow([('job_id', {'value': 2, 'label': '2'}), ('host', 'cldellow.com')])
```
I expected the `job_id` value to be `2`, but it's actually `{'value': 2, 'label': '2'}`.
I can work around this, but was wondering if this was intended behaviour?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1973/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1558644003,I_kwDOBm6k_c5c5wUj,2006,Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release,9599,simonw,open,0,,,3268330,Datasette 1.0,2,2023-01-26T19:17:40Z,2023-01-26T19:20:53Z,,OWNER,,"I just realized that when I ship Datasette 1.0 there may be automated deployments out there which could deploy the 1.0 version by accident, potentially breaking any customizations that aren't compatible with the 1.0 changes.
I can hopefully help avoid that by shipping one last entry in the `0.x` series that ensures `datasette publish` pins to `<1.0` when it installs Datasette itself.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2006/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1557507274,I_kwDOBm6k_c5c1azK,2005,`extra_template_vars` should be OK to return `None`,9599,simonw,open,0,,,,,1,2023-01-26T01:40:45Z,2023-01-26T01:41:50Z,,OWNER,,"Got this exception and had to make sure it always returned `{}`:
```
File "".../python3.11/site-packages/datasette/app.py"", line 1049, in render_template
assert isinstance(extra_vars, dict), ""extra_vars is of type {}"".format(
AssertionError: extra_vars is of type
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2005/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1555701851,PR_kwDOBm6k_c5IdsD7,2003,Show referring tables and rows when the referring foreign key is compound,536941,fgregg,open,0,,,,,3,2023-01-24T21:31:31Z,2023-01-25T18:44:42Z,,CONTRIBUTOR,simonw/datasette/pulls/2003,"sqlite foreign keys can be compound, but that is not as well supported by datasette as single column foreign keys.
in particular,
1. in a table view, there is not a link from the row to the referenced row if the foreign key is compound
2. in a row view, there is no listing of tables and rows that refer to the focal row if those referencing foreign keys are compound.
Both of these issues are discussed in #1099.
This PR only fixes the second one, because it's not clear what the right UX is for the first issue.
![Screenshot 2023-01-24 at 19-47-40 nlrb bargaining_unit](https://user-images.githubusercontent.com/536941/214454749-d53deead-4151-4329-a5d4-8a7a454de7d3.png)
Some things that might not be desirable about this approach.
1. it changes the external API, by changing `column` => `columns` and `other_column` => `other_columns` (see inline comment for more discussion.
2. There are various places where the plural foreign keys have to be checked for length and discarded or transformed to singular.
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2003/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1553615704,I_kwDOBm6k_c5cmktY,2001,Datasette is not compatible with SQLite's strict quoting compilation option,406380,gwk,open,0,,,,,4,2023-01-23T19:10:07Z,2023-01-25T04:59:58Z,,NONE,,"I have linked Python3.11 on macOS against recent SQLite that was compiled using `-DSQLITE_DQS=0`. This option disables interpretation of double-quoted identifiers as string literals, described in the SQLite docs as a ""MySQL 3.x misfeature"". See https://www.sqlite.org/quirks.html#dblquote for background.
Datasette uses the double-quote syntax in a number of key places, and is thus completely broken in this environment.
My experience was to `pip install datasette`, then run `datasette serve -I my-data.db`. When I visit `http://127.0.0.1:8001` I get a 500 response.
The error: `sqlite3.OperationalError: no such column: geometry_columns`
The responsible SQL: `'select 1 from sqlite_master where tbl_name = ""geometry_columns""'`
I then installed datasette from GitHub master in development mode and changed the offending SQL to use correct quotes: `""select 1 from sqlite_master where tbl_name = 'geometry_columns'""`.
With this change, I get a little further, but have the same problem with the first table name in my database (in my case, ""Meta""):
```
OperationalError: no such column: Meta
Traceback (most recent call last):
File ""/Users/gwk/external/datasette/datasette/app.py"", line 1522, in route_path
response = await view(request, send)
^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/views/base.py"", line 151, in view
return await self.dispatch_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/views/base.py"", line 105, in dispatch_request
response = await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/views/index.py"", line 70, in get
""fts_table"": await db.fts_table(table),
^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 363, in fts_table
return await self.execute_fn(lambda conn: detect_fts(conn, table))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 213, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/usr/local/py/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py"", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 211, in in_thread
return fn(conn)
^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/database.py"", line 363, in
return await self.execute_fn(lambda conn: detect_fts(conn, table))
^^^^^^^^^^^^^^^^^^^^^^^
File ""/Users/gwk/external/datasette/datasette/utils/__init__.py"", line 588, in detect_fts
rows = conn.execute(detect_fts_sql(table)).fetchall()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: no such column: Meta
INFO: 127.0.0.1:50258 - ""GET / HTTP/1.1"" 500 Internal Server Error
```
I will try to continue playing with this, but I also hope that the datasette developers will enable this mode in a test environment as I am unlikely to be able to exercise all of the SQL in the codebase, or make a pull request very soon.
Note that the DQS setting compile-time option can be overridden at runtime with calls to the C API:
```
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DDL, 0, (void*)0);
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DML, 0, (void*)0);
```
As far as I can tell, `sqlite3_db_config` is not exposed in Python, but perhaps we could figure out how to invoke it using `ctypes`.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2001/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
743371103,MDU6SXNzdWU3NDMzNzExMDM=,1099,Support linking to compound foreign keys,9599,simonw,open,0,,,,,6,2020-11-15T23:23:17Z,2023-01-25T00:58:26Z,,OWNER,,Reported as a bug in #1098 because they caused 500 errors - but it would be even better if Datasette could hyperlink to related rows via compound foreign keys.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1099/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1554032168,I_kwDOBm6k_c5coKYo,2002,Document how actors are displayed,9599,simonw,open,0,,,,,0,2023-01-24T00:08:49Z,2023-01-24T00:08:49Z,,OWNER,,"https://github.com/simonw/datasette/blob/e4ebef082de90db4e1b8527abc0d582b7ae0bc9d/datasette/utils/__init__.py#L1052-L1056
This logic should be reflected in the documentation on https://docs.datasette.io/en/stable/authentication.html#actors",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2002/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1536851861,I_kwDOBm6k_c5bmn-V,1994,Stuck on loading screen,10913053,jackhagley,open,0,,,,,1,2023-01-17T18:33:49Z,2023-01-23T08:21:08Z,,NONE,,"Can’t actually open it!
Downloaded today from the releases tab
Running macOS13.1
```
bin/python3.9 --version
Python 3.9.6
Took 83ms
bin/python3.9 --version
Python 3.9.6
Took 113ms
bin/pip install datasette>=0.59 datasette-app-support>=0.11.6 datasette-vega>=0.6.2 datasette-cluster-map>=0.17.1 datasette-pretty-json>=0.2.1 datasette-edit-schema>=0.4 datasette-configure-fts>=1.1 datasette-leaflet>=0.2.2 --disable-pip-version-check
Requirement already satisfied: datasette>=0.59 in lib/python3.9/site-packages (0.63)
Requirement already satisfied: datasette-app-support>=0.11.6 in lib/python3.9/site-packages (0.11.6)
Requirement already satisfied: datasette-vega>=0.6.2 in lib/python3.9/site-packages (0.6.2)
Requirement already satisfied: datasette-cluster-map>=0.17.1 in lib/python3.9/site-packages (0.17.2)
Requirement already satisfied: datasette-pretty-json>=0.2.1 in lib/python3.9/site-packages (0.2.2)
Requirement already satisfied: datasette-edit-schema>=0.4 in lib/python3.9/site-packages (0.5.1)
Requirement already satisfied: datasette-configure-fts>=1.1 in lib/python3.9/site-packages (1.1)
Requirement already satisfied: datasette-leaflet>=0.2.2 in lib/python3.9/site-packages (0.2.2)
Requirement already satisfied: click>=7.1.1 in lib/python3.9/site-packages (from datasette>=0.59) (8.1.3)
Requirement already satisfied: hupper>=1.9 in lib/python3.9/site-packages (from datasette>=0.59) (1.10.3)
Requirement already satisfied: pint>=0.9 in lib/python3.9/site-packages (from datasette>=0.59) (0.20.1)
Requirement already satisfied: PyYAML>=5.3 in lib/python3.9/site-packages (from datasette>=0.59) (6.0)
Requirement already satisfied: httpx>=0.20 in lib/python3.9/site-packages (from datasette>=0.59) (0.23.0)
Requirement already satisfied: aiofiles>=0.4 in lib/python3.9/site-packages (from datasette>=0.59) (22.1.0)
Requirement already satisfied: asgi-csrf>=0.9 in lib/python3.9/site-packages (from datasette>=0.59) (0.9)
Requirement already satisfied: asgiref>=3.2.10 in lib/python3.9/site-packages (from datasette>=0.59) (3.5.2)
Requirement already satisfied: uvicorn>=0.11 in lib/python3.9/site-packages (from datasette>=0.59) (0.19.0)
Requirement already satisfied: itsdangerous>=1.1 in lib/python3.9/site-packages (from datasette>=0.59) (2.1.2)
Requirement already satisfied: click-default-group-wheel>=1.2.2 in lib/python3.9/site-packages (from datasette>=0.59) (1.2.2)
Requirement already satisfied: janus>=0.6.2 in lib/python3.9/site-packages (from datasette>=0.59) (1.0.0)
Requirement already satisfied: pluggy>=1.0 in lib/python3.9/site-packages (from datasette>=0.59) (1.0.0)
Requirement already satisfied: Jinja2>=2.10.3 in lib/python3.9/site-packages (from datasette>=0.59) (3.1.2)
Requirement already satisfied: mergedeep>=1.1.1 in lib/python3.9/site-packages (from datasette>=0.59) (1.3.4)
Requirement already satisfied: sqlite-utils in lib/python3.9/site-packages (from datasette-app-support>=0.11.6) (3.30)
Requirement already satisfied: packaging in lib/python3.9/site-packages (from datasette-app-support>=0.11.6) (21.3)
Requirement already satisfied: python-multipart in lib/python3.9/site-packages (from asgi-csrf>=0.9->datasette>=0.59) (0.0.5)
Requirement already satisfied: httpcore<0.16.0,>=0.15.0 in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (0.15.0)
Requirement already satisfied: certifi in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (2022.9.24)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (1.5.0)
Requirement already satisfied: sniffio in lib/python3.9/site-packages (from httpx>=0.20->datasette>=0.59) (1.3.0)
Requirement already satisfied: h11<0.13,>=0.11 in lib/python3.9/site-packages (from httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (0.12.0)
Requirement already satisfied: anyio==3.* in lib/python3.9/site-packages (from httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (3.6.2)
Requirement already satisfied: idna>=2.8 in lib/python3.9/site-packages (from anyio==3.*->httpcore<0.16.0,>=0.15.0->httpx>=0.20->datasette>=0.59) (3.4)
Requirement already satisfied: typing-extensions>=3.7.4.3 in lib/python3.9/site-packages (from janus>=0.6.2->datasette>=0.59) (4.4.0)
Requirement already satisfied: MarkupSafe>=2.0 in lib/python3.9/site-packages (from Jinja2>=2.10.3->datasette>=0.59) (2.1.1)
Requirement already satisfied: tabulate in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (0.9.0)
Requirement already satisfied: python-dateutil in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (2.8.2)
Requirement already satisfied: sqlite-fts4 in lib/python3.9/site-packages (from sqlite-utils->datasette-app-support>=0.11.6) (1.0.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in lib/python3.9/site-packages (from packaging->datasette-app-support>=0.11.6) (3.0.9)
Requirement already satisfied: six>=1.5 in lib/python3.9/site-packages (from python-dateutil->sqlite-utils->datasette-app-support>=0.11.6) (1.16.0)
Took 784ms
```
STUCK",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1994/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1552368054,I_kwDOBm6k_c5ch0G2,2000,rewrite_sql hook,193185,cldellow,open,0,,,,,1,2023-01-23T01:02:52Z,2023-01-23T06:08:01Z,,CONTRIBUTOR,,"I'm not sold that this is a good idea, but thought it'd be worth writing up a ticket. Proposal: add a hook like
```python
def rewrite_sql(datasette, database, request, fn, sql, params)
```
It would be called from Database.execute, Database.execute_write, Database.execute_write_script, Database.execute_write_many before running the user's SQL. `fn` would indicate which method was being used, in case that's relevant for the SQL inspection -- for example `execute` only permits a single statement.
The hook could return a SQL statement to be executed instead, or an async function to be awaited on that returned the SQL to be executed.
Plugins that could be written with this hook:
- https://github.com/cldellow/datasette-ersatz-table-valued-functions would use this to avoid monkey-patching
- a plugin to inspect and reject unsafe Spatialite function calls (reported by [Simon in Discord](https://discord.com/channels/823971286308356157/823971286941302908/1066438832293159004))
- a plugin to do more general rewrites of queries to enforce table or row-level security, for example, based on the currently logged in actor's ID
- a plugin to maintain audit tables when users write to a table
- a plugin to cache expensive queries (eg the queries that drive facets) - these could allow stale reads if previously cached, then refresh them in an offline queue
Flaws with this idea:
`execute_fn` and `execute_write_fn` would not go through this hook, which limits the guarantees you can make about it for security purposes.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2000/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
749283032,MDU6SXNzdWU3NDkyODMwMzI=,1101,register_output_renderer() should support streaming data,9599,simonw,open,0,,,3268330,Datasette 1.0,13,2020-11-24T02:17:09Z,2023-01-21T22:07:19Z,,OWNER,,"> I'd like to implement this by first extending the `register_output_renderer()` hook to support streaming huge responses, then switching CSV to use the plugin hook in addition to TSV using it.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1096#issuecomment-732542285_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1101/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1551113681,I_kwDOBm6k_c5cdB3R,1998,`datasette --version` should also show the SQLite version,9599,simonw,open,0,,,,,2,2023-01-20T16:11:30Z,2023-01-20T18:19:06Z,,OWNER,,Idea came up here: https://discord.com/channels/823971286308356157/823971286941302908/1066026473003159783,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1998/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1533673397,I_kwDOBm6k_c5baf-1,1991,fts5 tables are not auto-detected and hidden,83819,keturn,open,0,,,,,0,2023-01-15T06:00:42Z,2023-01-20T04:54:24Z,,NONE,,"I set up a [Datasette instance](https://huggingface.co/spaces/Sygil/INE-dataset-explorer/tree/main) and was following the docs on full-text search.
When I used fts4, datasette automatically hid the FTS tables and added the FTS search box where appropriate, but when I changed to fts5 it no longer does either.
If I [manually set](https://huggingface.co/spaces/keturn/INED-datasette/blob/main/metadata.json#L9) `fts_table` for a view, then search does work as expected.
My table and view creation code looks like this:
```py
connection.execute(""""""CREATE TABLE IF NOT EXISTS
captions(image_key text PRIMARY KEY, caption text NOT NULL)
"""""")
connection.execute(""""""CREATE VIRTUAL TABLE
captions_fts USING
fts5(caption, image_key UNINDEXED, content=captions)
"""""")
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1991/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1538342965,PR_kwDOBm6k_c5HpNYo,1996,Document custom json encoder,25778,eyeseast,open,0,,,,,1,2023-01-18T16:54:14Z,2023-01-19T12:55:57Z,,CONTRIBUTOR,simonw/datasette/pulls/1996,"Closes #1983
All documentation here. Edits welcome.
----
:books: Documentation preview :books:: https://datasette--1996.org.readthedocs.build/en/1996/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1996/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1034535001,I_kwDOBm6k_c49qcBZ,1497,"Publish to Docker Hub failing with ""libcrypt.so.1: cannot open shared object file""",9599,simonw,closed,0,,,,,18,2021-10-24T22:57:07Z,2023-01-18T17:13:45Z,2021-10-24T23:36:55Z,OWNER,,"This means the Datasette 0.59.1 release has not been published to Docker Hub.
Here's where that failed: https://github.com/simonw/datasette/runs/3991043374?check_suite_focus=true
```
Preparing to unpack .../libc6_2.32-4_amd64.deb ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Checking for services that may need to be restarted...
Checking init scripts...
Unpacking libc6:amd64 (2.32-4) over (2.28-10) ...
Setting up libc6:amd64 (2.32-4) ...
/usr/bin/perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
dpkg: error processing package libc6:amd64 (--configure):
installed libc6:amd64 package post-installation script subprocess returned error exit status 127
Errors were encountered while processing:
libc6:amd64
E: Sub-process /usr/bin/dpkg returned an error code (1)
The command '/bin/sh -c apt-get update && apt-get -y --no-install-recommends install software-properties-common && add-apt-repository ""deb http://httpredir.debian.org/debian sid main"" && apt-get update && apt-get -t sid install -y --no-install-recommends libsqlite3-mod-spatialite && apt-get remove -y software-properties-common && apt clean && rm -rf /var/lib/apt && rm -rf /var/lib/dpkg/info/*' returned a non-zero code: 100
```
Same problem when I attempted to publish using the ""Push specific Docker tag"" workflow: https://github.com/simonw/datasette/runs/3991059912?check_suite_focus=true",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1497/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1538197093,I_kwDOBm6k_c5brwZl,1995,foreign_keys error 500,137183,jonschoning,open,0,,,,,0,2023-01-18T15:27:36Z,2023-01-18T16:44:01Z,,NONE,,"**Error 500 expected string or bytes-like object**
[espial-new.sqlite3.zip](https://github.com/simonw/datasette/files/10447965/espial-new.sqlite3.zip)
run `datasette espial-new.sqlite3` & click on any table other than `User`
```
/home/jon/.local/lib/python3.10/site-packages/datasette/app.py:814 in │
│ expand_foreign_keys │
│ │
│ 811 │ │ │ from {other_table} │
│ 812 │ │ │ where {other_column} in ({placeholders}) │
│ 813 │ │ """""".format( │
│ ❱ 814 │ │ │ other_column=escape_sqlite(fk[""other_column""]), │
│ 815 │ │ │ label_column=escape_sqlite(label_column), │
│ 816 │ │ │ other_table=escape_sqlite(fk[""other_table""]), │
│ 817 │ │ │ placeholders="", "".join([""?""] * len(set(values))), │
│ │
│ ╭───────────────────────────── locals ──────────────────────────────╮ │
│ │ column = 'user_id' │ │
│ │ database = 'espial-new' │ │
│ │ db = │ │
│ │ fk = { │ │
│ │ │ 'column': 'user_id', │ │
│ │ │ 'other_table': 'user', │ │
│ │ │ 'other_column': None │ │
│ │ } │ │
│ │ foreign_keys = [ │ │
│ │ │ { │ │
│ │ │ │ 'column': 'user_id', │ │
│ │ │ │ 'other_table': 'user', │ │
│ │ │ │ 'other_column': None │ │
│ │ │ } │ │
│ │ ] │ │
│ │ label_column = 'name' │ │
│ │ labeled_fks = {} │ │
│ │ self = │ │
│ │ table = 'bookmark' │ │
│ │ values = [] │ │
│ ╰───────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/jon/.local/lib/python3.10/site-packages/datasette/utils/__init__.py:346 │
│ in escape_sqlite │
│ │
│ 343 │
│ 344 │
│ 345 def escape_sqlite(s): │
│ ❱ 346 │ if _boring_keyword_re.match(s) and (s.lower() not in reserved_words) │
│ 347 │ │ return s │
│ 348 │ else: │
│ 349 │ │ return f""[{s}]"" │
│ │
│ ╭─ locals ─╮ │
│ │ s = None │ │
│ ╰──────────╯ │
╰─────────────────────────────────────────────────────────────────────────────────╯
TypeError: expected string or bytes-like object
Traceback (most recent call last):
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/app.py"", line 1354, in route_path
response = await view(request, send)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 134, in view
return await self.dispatch_request(request)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 91, in dispatch_request
return await handler(request)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/base.py"", line 361, in get
response_or_template_contexts = await self.data(request, **data_kwargs)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/table.py"", line 158, in data
return await self._data_traced(request, default_labels, _next, _size)
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/views/table.py"", line 603, in _data_traced
await self.ds.expand_foreign_keys(
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/app.py"", line 814, in expand_foreign_keys
other_column=escape_sqlite(fk[""other_column""]),
File ""/home/jon/.local/lib/python3.10/site-packages/datasette/utils/__init__.py"", line 346, in escape_sqlite
if _boring_keyword_re.match(s) and (s.lower() not in reserved_words):
TypeError: expected string or bytes-like object
INFO: 127.0.0.1:38574 - ""GET /espial-new/bookmark HTTP/1.1"" 500 Internal Server Error
INFO: 127.0.0.1:38574 - ""GET /-/static/app.css?d59929 HTTP/1.1"" 200 OK
```
Schema:
```
CREATE TABLE IF NOT EXISTS ""user""
(
""id"" INTEGER PRIMARY KEY,
""name"" VARCHAR NOT NULL,
""password_hash"" VARCHAR NOT NULL,
""api_token"" VARCHAR NULL,
""private_default"" BOOLEAN NOT NULL,
""archive_default"" BOOLEAN NOT NULL,
""privacy_lock"" BOOLEAN NOT NULL,
CONSTRAINT ""unique_user_name"" UNIQUE (""name"")
);
CREATE TABLE IF NOT EXISTS ""bookmark""
(
""id"" INTEGER PRIMARY KEY,
""user_id"" INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""slug"" VARCHAR NOT NULL DEFAULT (Lower(Hex(Randomblob(6)))),
""href"" VARCHAR NOT NULL,
""description"" VARCHAR NOT NULL,
""extended"" VARCHAR NOT NULL,
""time"" TIMESTAMP NOT NULL,
""shared"" BOOLEAN NOT NULL,
""to_read"" BOOLEAN NOT NULL,
""selected"" BOOLEAN NOT NULL,
""archive_href"" VARCHAR NULL,
CONSTRAINT ""unique_user_href"" UNIQUE (""user_id"", ""href""),
CONSTRAINT ""unique_user_slug"" UNIQUE (""user_id"", ""slug"")
);
CREATE TABLE IF NOT EXISTS ""bookmark_tag""
(
""id"" INTEGER PRIMARY KEY,
""user_id"" INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""tag"" VARCHAR NOT NULL,
""bookmark_id"" INTEGER NOT NULL REFERENCES ""bookmark"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""seq"" INTEGER NOT NULL,
CONSTRAINT ""unique_user_tag_bookmark_id"" UNIQUE (""user_id"", ""tag"", ""bookmark_id""),
CONSTRAINT ""unique_user_bookmark_id_tag_seq"" UNIQUE (""user_id"", ""bookmark_id"", ""tag"", ""seq"")
);
CREATE TABLE IF NOT EXISTS ""note""
(
""id"" INTEGER PRIMARY KEY,
""user_id"" INTEGER NOT NULL REFERENCES ""user"" ON DELETE RESTRICT ON UPDATE RESTRICT,
""slug"" VARCHAR NOT NULL DEFAULT (Lower(Hex(Randomblob(10)))),
""length"" INTEGER NOT NULL,
""title"" VARCHAR NOT NULL,
""text"" VARCHAR NOT NULL,
""is_markdown"" BOOLEAN NOT NULL,
""shared"" BOOLEAN NOT NULL DEFAULT false,
""created"" TIMESTAMP NOT NULL,
""updated"" TIMESTAMP NOT NULL
);
CREATE INDEX idx_bookmark_time ON bookmark (user_id, time DESC);
CREATE INDEX idx_bookmark_tag_bookmark_id ON bookmark_tag (bookmark_id, id, tag, seq);
CREATE INDEX idx_note_user_created ON note (user_id, created DESC);
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1995/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1532000914,I_kwDOBm6k_c5bUHqS,1990,Suggestion: Highlight error messages ('These facets timed out'),116795,pax,open,0,,,,,0,2023-01-13T09:40:58Z,2023-01-13T09:40:58Z,,NONE,,"I had trouble figuring out why faceting didn't work in some instances, it took a while before I noticed the _These facets timed out_ notice.
It might help if that would be highlighted, or fading out highlight - if one might think it would be too visually disturbing.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1990/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1529707837,I_kwDOBm6k_c5bLX09,1988,Reconsider pattern where plugins could break existing template context,9599,simonw,open,0,,,3268330,Datasette 1.0,4,2023-01-11T21:13:43Z,2023-01-11T21:25:05Z,,OWNER,,"> I hadn't run into an issue with plugins like `datasette-template-sql` interfering with the existing context for other features before! Definitely not a good thing.
_Originally posted by @simonw in https://github.com/simonw/datasette-write/issues/6#issuecomment-1379490596_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1988/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1529452371,I_kwDOBm6k_c5bKZdT,1987,installpython3.com is now a spam website,9599,simonw,closed,0,,,,,4,2023-01-11T17:55:12Z,2023-01-11T18:29:26Z,2023-01-11T18:29:25Z,OWNER,,"Need to stop linking to it from the docs.
I'll link to https://www.python.org/about/gettingstarted/ instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1987/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1528448642,I_kwDOBm6k_c5bGkaC,1985,Don't let Datasette(path) without a list cause weird errors,9599,simonw,closed,0,,,,,1,2023-01-11T05:17:44Z,2023-01-11T18:25:04Z,2023-01-11T18:25:04Z,OWNER,,"I got a confusing `sqlite3.OperationalError: disk I/O error` error in my tests, it turned out it was because this:
```python
ds = Datasette(path)
```
Should have been this:
```python
ds = Datasette([path])
```
_Originally posted by @simonw in https://github.com/simonw/datasette-faiss/issues/1#issuecomment-1378252673_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1985/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1525560504,PR_kwDOBm6k_c5G-ZsQ,1982,Bump sphinx from 5.3.0 to 6.1.2,49699333,dependabot[bot],closed,0,,,,,1,2023-01-09T13:06:11Z,2023-01-10T02:03:21Z,2023-01-10T02:03:19Z,CONTRIBUTOR,simonw/datasette/pulls/1982,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.3.0 to 6.1.2.
Release notes
Sourced from sphinx's releases .
v6.1.2
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.1.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.1.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b2
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 6.1.2 (released Jan 07, 2023)
Bugs fixed
#11101 : LaTeX: div.topic_padding
key of sphinxsetup documented at 5.1.0 was
implemented with name topic_padding
#11099 : LaTeX: shadowrule
key of sphinxsetup causes PDF build to crash
since Sphinx 5.1.0
#11096 : LaTeX: shadowsize
key of sphinxsetup causes PDF build to crash
since Sphinx 5.1.0
#11095 : LaTeX: shadow of :dudir:topic
and contents_ boxes not in page
margin since Sphinx 5.1.0
.. _contents: https://docutils.sourceforge.io/docs/ref/rst/directives.html#table-of-contents
#11100 : Fix copying images when running under parallel mode.
Release 6.1.1 (released Jan 05, 2023)
Bugs fixed
#11091 : Fix util.nodes.apply_source_workaround
for literal_block
nodes
with no source information in the node or the node's parents.
Release 6.1.0 (released Jan 05, 2023)
Dependencies
Incompatible changes
#10979 : gettext: Removed support for pluralisation in get_translation
.
This was unused and complicated other changes to sphinx.locale
.
Deprecated
sphinx.util
functions:
Renamed sphinx.util.typing.stringify()
to sphinx.util.typing.stringify_annotation()
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=5.3.0&new-version=6.1.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1982.org.readthedocs.build/en/1982/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1982/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1515185383,I_kwDOBm6k_c5aT-Tn,1971,Upgrade for Sphinx 6.0 (once Furo has support for it),9599,simonw,closed,0,,,,,3,2022-12-31T19:04:35Z,2023-01-10T02:02:34Z,2023-01-10T02:02:34Z,OWNER,,"A deployment of #1967 to ReadTheDocs just failed like this: https://readthedocs.org/projects/datasette/builds/19045460/
```
Running Sphinx v6.0.0
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 28 source files that are out of date
updating environment: [new config] 28 added, 0 changed, 0 removed
reading sources... [ 3%] authentication
reading sources... [ 7%] binary_data
reading sources... [ 10%] changelog
Traceback (most recent call last):
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 299, in next_line
self.line = self.input_lines[self.line_offset]
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 1136, in __getitem__
return self.data[i]
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 226, in run
self.next_line()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 302, in next_line
raise EOFError
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/cmd/build.py"", line 281, in build_main
app.build(args.force_all, args.filenames)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/application.py"", line 344, in build
self.builder.build_update()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 310, in build_update
self.build(to_build,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 326, in build
updated_docnames = set(self.read())
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 433, in read
self._read_serial(docnames)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 454, in _read_serial
self.read_doc(docname)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/builders/__init__.py"", line 510, in read_doc
publisher.publish()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/core.py"", line 224, in publish
self.document = self.reader.read(self.source, self.parser,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/io.py"", line 103, in read
self.parse()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/readers/__init__.py"", line 76, in parse
self.parser.parse(self.input, document)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/parsers.py"", line 78, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 169, in run
results = StateMachineWS.run(self, input_lines, input_offset,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 233, in run
context, next_state, result = self.check_line(
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 445, in check_line
return method(match, context, next_state)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 3024, in text
self.section(title.lstrip(), source, style, lineno + 1, messages)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 325, in section
self.new_subsection(title, lineno, messages)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 391, in new_subsection
newabsoffset = self.nested_parse(
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 279, in nested_parse
state_machine.run(block, input_offset, memo=self.memo,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 233, in run
context, next_state, result = self.check_line(
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 445, in check_line
return method(match, context, next_state)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 2785, in underline
self.section(title, source, style, lineno - 1, messages)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 325, in section
self.new_subsection(title, lineno, messages)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 391, in new_subsection
newabsoffset = self.nested_parse(
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 279, in nested_parse
state_machine.run(block, input_offset, memo=self.memo,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 233, in run
context, next_state, result = self.check_line(
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 445, in check_line
return method(match, context, next_state)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 1273, in bullet
i, blank_finish = self.list_item(match.end())
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 1295, in list_item
self.nested_parse(indented, input_offset=line_offset,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 279, in nested_parse
state_machine.run(block, input_offset, memo=self.memo,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/statemachine.py"", line 239, in run
result = state.eof(context)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 2725, in eof
self.blank(None, context, None)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 2716, in blank
paragraph, literalnext = self.paragraph(
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 416, in paragraph
textnodes, messages = self.inline_text(text, lineno)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 425, in inline_text
nodes, messages = self.inliner.parse(text, lineno,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 649, in parse
before, inlines, remaining, sysmessages = method(self, match,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 792, in interpreted_or_phrase_ref
nodelist, messages = self.interpreted(rawsource, escaped, role,
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/docutils/parsers/rst/states.py"", line 889, in interpreted
nodes, messages2 = role_fn(role, rawsource, text, lineno, self)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/ext/extlinks.py"", line 101, in role
title = caption % part
TypeError: not all arguments converted during string formatting
Exception occurred:
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/latest/lib/python3.9/site-packages/sphinx/ext/extlinks.py"", line 101, in role
title = caption % part
TypeError: not all arguments converted during string formatting
The full traceback has been saved in /tmp/sphinx-err-kq7ylgqo.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at . Thanks!
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1971/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1526635374,PR_kwDOBm6k_c5HCCY2,1984,Upgrade Sphinx,9599,simonw,closed,0,,,,,1,2023-01-10T02:00:40Z,2023-01-10T02:02:33Z,2023-01-10T02:02:33Z,OWNER,simonw/datasette/pulls/1984,"Refs #1971
----
:books: Documentation preview :books:: https://datasette--1984.org.readthedocs.build/en/1984/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1984/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1509783085,I_kwDOBm6k_c5Z_XYt,1969,sql-formatter javascript is not now working with CloudFlare rocketloader,536941,fgregg,open,0,,,,,0,2022-12-23T21:14:06Z,2023-01-10T01:56:33Z,,CONTRIBUTOR,,"This is probably not a bug with datasette, but I thought you might want to know, @simonw.
I noticed today that my CloudFlare proxied datasette instance lost the ""Format SQL"" option. I'm pretty sure it was there last week.
In the CloudFlare settings, if I turn off [Rocket Loader](https://developers.cloudflare.com/fundamentals/speed/rocket-loader/), I get the ""Format SQL"" option back.
Rocket Loader works by asynchronously loading the javascript, so maybe there was a recent change that doesn't play well with the asynch loading?
I'm up to date with https://github.com/simonw/datasette/commit/e03aed00026cc2e59c09ca41f69a247e1a85cc89",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1969/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1525815985,I_kwDOBm6k_c5a8hqx,1983,Make CustomJSONEncoder a documented public API,9599,simonw,open,0,,,,,3,2023-01-09T15:27:05Z,2023-01-09T15:35:58Z,,OWNER,,It's used by `datasette-geojson` here: https://github.com/eyeseast/datasette-geojson/commit/902bf135a5a33a0dc8264673d00a59a67cb05152,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1983/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1426080014,I_kwDOBm6k_c5VAEEO,1867,/db/table/-/rename API (also allows atomic replace),9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-10-27T18:13:23Z,2023-01-09T15:34:12Z,,OWNER,,"> There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first.
>
> ...
>
> If people care about that kind of thing they could always push all of their inserts to a table called `_tablename` and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism).
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1867/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
710650633,MDU6SXNzdWU3MTA2NTA2MzM=,979,Default table view JSON should include CREATE TABLE,9599,simonw,closed,0,,,,,3,2020-09-28T23:54:58Z,2023-01-09T15:32:39Z,2023-01-09T15:32:22Z,OWNER,,"https://latest.datasette.io/fixtures/facetable.json doesn't currently include the CREATE TABLE statement for the page, even though it's available on the HTML version at https://latest.datasette.io/fixtures/facetable",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/979/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1082584499,I_kwDOBm6k_c5Ahu2z,1558,Redesign `facet_results` JSON structure prior to Datasette 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,3,2021-12-16T19:45:10Z,2023-01-09T15:31:17Z,,OWNER,,"> Decision: as an initial fix I'm going to de-duplicate those keys by using `tags__array` etc - with a `_2` on the end if that key is already used.
>
> I'll open a separate issue to redesign this better for Datasette 1.0.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/625#issuecomment-996130862_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1558/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1522778923,I_kwDOBm6k_c5aw8Mr,1978,Document datasette.urls.row and row_blob,25778,eyeseast,closed,0,,,,,2,2023-01-06T15:45:51Z,2023-01-09T14:30:00Z,2023-01-09T14:30:00Z,CONTRIBUTOR,,"These are in the codebase but not in documentation. I think everything else in this class is documented.
```python
class Urls:
...
def row(self, database, table, row_path, format=None):
path = f""{self.table(database, table)}/{row_path}""
if format is not None:
path = path_with_format(path=path, format=format)
return PrefixedUrlString(path)
def row_blob(self, database, table, row_path, column):
return self.table(database, table) + ""/{}.blob?_blob_column={}"".format(
row_path, urllib.parse.quote_plus(column)
)
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1978/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,not_planned
1522552817,PR_kwDOBm6k_c5G0XxH,1977,Bump sphinx from 5.3.0 to 6.1.1,49699333,dependabot[bot],closed,0,,,,,2,2023-01-06T13:02:12Z,2023-01-09T13:06:17Z,2023-01-09T13:06:14Z,CONTRIBUTOR,simonw/datasette/pulls/1977,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.3.0 to 6.1.1.
Release notes
Sourced from sphinx's releases .
v6.1.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.1.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b2
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 6.1.1 (released Jan 05, 2023)
Bugs fixed
#11091 : Fix util.nodes.apply_source_workaround
for literal_block
nodes
with no source information in the node or the node's parents.
Release 6.1.0 (released Jan 05, 2023)
Dependencies
Incompatible changes
#10979 : gettext: Removed support for pluralisation in get_translation
.
This was unused and complicated other changes to sphinx.locale
.
Deprecated
sphinx.util
functions:
Renamed sphinx.util.typing.stringify()
to sphinx.util.typing.stringify_annotation()
Moved sphinx.util.xmlname_checker()
to sphinx.builders.epub3._XML_NAME_PATTERN
Moved to sphinx.util.display
:
sphinx.util.status_iterator
sphinx.util.display_chunk
sphinx.util.SkipProgressMessage
sphinx.util.progress_message
Moved to sphinx.util.http_date
:
sphinx.util.epoch_to_rfc1123
sphinx.util.rfc1123_to_epoch
Moved to sphinx.util.exceptions
:
sphinx.util.save_traceback
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=5.3.0&new-version=6.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1977.org.readthedocs.build/en/1977/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1977/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1524983536,I_kwDOBm6k_c5a5Wbw,1981,Canned query field labels truncated,9599,simonw,open,0,,,,,1,2023-01-09T06:04:24Z,2023-01-09T06:05:44Z,,OWNER,,"Eg here on mobile: https://timezones.datasette.io/timezones/by_point?longitude=-0.1406632&latitude=50.8246776
![107A1894-D1DA-4158-9EA3-40C840DD10E3](https://user-images.githubusercontent.com/9599/211248895-c922ce61-95d3-47ca-9314-dcff7c86afab.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1981/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1524867951,I_kwDOBm6k_c5a46Nv,1980,"""Cannot sort table by id"" when sortable_columns is used",9599,simonw,open,0,,,,,2,2023-01-09T03:21:33Z,2023-01-09T03:23:53Z,,OWNER,,"I had an instance with this in `metadata.yml`:
```yaml
databases:
timezones:
tables:
timezones:
sortable_columns:
- tzid
```
When I clicked on the ""Apply"" button here:
It sent me to `/timezones/timezones?_sort=id&id__exact=133` with the error message:
> 500: Cannot sort table by id",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1980/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1524076587,I_kwDOBm6k_c5a15Ar,1979,More useful error message if enable_load_extension is not available,9599,simonw,closed,0,,,,,5,2023-01-07T19:13:19Z,2023-01-08T00:21:23Z,2023-01-08T00:21:23Z,OWNER,,"I get this from:
datasette --load-extension spatialite --get /-/versions.json
```
File ""/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/datasette/app.py"", line 614, in _prepare_connection
conn.enable_load_extension(True)
AttributeError: 'sqlite3.Connection' object has no attribute 'enable_load_extension'
```
It would be useful if Datasette caught this error and output something more friendly.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1979/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957310278,MDU6SXNzdWU5NTczMTAyNzg=,1409,`default_allow_sql` setting (a re-imagining of the old `allow_sql` setting),9599,simonw,closed,0,,,3268330,Datasette 1.0,10,2021-07-31T19:48:56Z,2023-01-07T18:06:01Z,2023-01-05T00:51:31Z,OWNER,,"In 49d6d2f7b0f6cb02e25022e1c9403811f1fa0a7c as part of #813 I removed the `allow_sql` setting - on the basis that users could disable the ability to execute custom SQL queries using the new permission system instead.
I don't think this was the right decision. Disabling custom SQL is an important security capability, and explaining how to do it using permissions is significantly more complex than letting people know they can add `--setting allow_sql off`.
So I want to bring that setting back - maybe with a different, better name - and have it modify the default for that option if the permissions system doesn't have an opinion.
That way people can still use the setting but then use permissions to allow specific signed-in users access to execute SQL.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1409/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1520712722,PR_kwDOBm6k_c5GuDBN,1976,Bump sphinx from 5.3.0 to 6.1.0,49699333,dependabot[bot],closed,0,,,,,2,2023-01-05T13:02:37Z,2023-01-06T13:02:17Z,2023-01-06T13:02:15Z,CONTRIBUTOR,simonw/datasette/pulls/1976,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.3.0 to 6.1.0.
Release notes
Sourced from sphinx's releases .
v6.1.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b2
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 6.1.0 (released Jan 05, 2023)
Dependencies
Incompatible changes
#10979 : gettext: Removed support for pluralisation in get_translation
.
This was unused and complicated other changes to sphinx.locale
.
Deprecated
sphinx.util
functions:
Renamed sphinx.util.typing.stringify()
to sphinx.util.typing.stringify_annotation()
Moved sphinx.util.xmlname_checker()
to sphinx.builders.epub3._XML_NAME_PATTERN
Moved to sphinx.util.display
:
sphinx.util.status_iterator
sphinx.util.display_chunk
sphinx.util.SkipProgressMessage
sphinx.util.progress_message
Moved to sphinx.util.http_date
:
sphinx.util.epoch_to_rfc1123
sphinx.util.rfc1123_to_epoch
Moved to sphinx.util.exceptions
:
sphinx.util.save_traceback
sphinx.util.format_exception_cut_frames
Features added
Cache doctrees in the build environment during the writing phase.
Make all writing phase tasks support parallel execution.
#11072 : Use PEP 604 (X | Y
) display conventions for typing.Optional
and typing.Optional
types within the Python domain and autodoc.
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=5.3.0&new-version=6.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1976.org.readthedocs.build/en/1976/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1976/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1516376583,PR_kwDOBm6k_c5GfPJL,1974,Bump sphinx from 5.3.0 to 6.0.0,49699333,dependabot[bot],closed,0,,,,,2,2023-01-02T13:04:26Z,2023-01-05T13:02:42Z,2023-01-05T13:02:40Z,CONTRIBUTOR,simonw/datasette/pulls/1974,"Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.3.0 to 6.0.0.
Release notes
Sourced from sphinx's releases .
v6.0.0
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b2
Changelog: https://www.sphinx-doc.org/en/master/changes.html
v6.0.0b1
Changelog: https://www.sphinx-doc.org/en/master/changes.html
Changelog
Sourced from sphinx's changelog .
Release 6.0.0 (released Dec 29, 2022)
Dependencies
#10468 : Drop Python 3.6 support
#10470 : Drop Python 3.7, Docutils 0.14, Docutils 0.15, Docutils 0.16, and
Docutils 0.17 support. Patch by Adam Turner
Incompatible changes
#7405 : Removed the jQuery and underscore.js JavaScript frameworks.
These frameworks are no longer be automatically injected into themes from
Sphinx 6.0. If you develop a theme or extension that uses the
jQuery
, $
, or $u
global objects, you need to update your
JavaScript to modern standards, or use the mitigation below.
The first option is to use the sphinxcontrib.jquery_ extension, which has been
developed by the Sphinx team and contributors. To use this, add
sphinxcontrib.jquery
to the extensions
list in conf.py
, or call
app.setup_extension("sphinxcontrib.jquery")
if you develop a Sphinx theme
or extension.
The second option is to manually ensure that the frameworks are present.
To re-add jQuery and underscore.js, you will need to copy jquery.js
and
underscore.js
from the Sphinx repository
_ to your static
directory,
and add the following to your layout.html
:
.. code-block:: html+jinja
{%- block scripts %}
{{ super() }}
{%- endblock %}
.. _sphinxcontrib.jquery: https://github.com/sphinx-contrib/jquery/
Patch by Adam Turner.
#10471 , #10565 : Removed deprecated APIs scheduled for removal in Sphinx 6.0. See
:ref:dev-deprecated-apis
for details. Patch by Adam Turner.
#10901 : C Domain: Remove support for parsing pre-v3 style type directives and
roles. Also remove associated configuration variables c_allow_pre_v3
and
c_warn_on_allowed_pre_v3
. Patch by Adam Turner.
Features added
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx&package-manager=pip&previous-version=5.3.0&new-version=6.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1974.org.readthedocs.build/en/1974/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1974/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1516815571,I_kwDOBm6k_c5aaMTT,1975,_col=id can cause id column to export twice in CSV export,9599,simonw,open,0,,,,,0,2023-01-03T00:25:15Z,2023-01-03T00:25:21Z,,OWNER,,"https://datasette.simonwillison.net/simonwillisonblog/blog_entry.csv?_col=id&_col=title&_col=body&_labels=on&_size=1
```csv
id,id,title,body
1,1,WaSP Phase II,""The Web Standards project has launched Phase II.
""
```
That should not have two `id` columns.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1975/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1515182998,I_kwDOBm6k_c5aT9uW,1970,"Path ""None"" in _internal database table",9599,simonw,closed,0,,,,,2,2022-12-31T18:51:05Z,2022-12-31T19:22:58Z,2022-12-31T18:52:49Z,OWNER,,"See https://latest.datasette.io/_internal/databases (after https://latest.datasette.io/login-as-root)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1970/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1503010009,PR_kwDOBm6k_c5FyT3c,1967,Add favicon to documentation,1839645,choldgraf,closed,0,,,,,2,2022-12-19T14:01:04Z,2022-12-31T19:15:51Z,2022-12-31T19:00:31Z,CONTRIBUTOR,simonw/datasette/pulls/1967,"I've been browsing the datasette documentation and found it hard to quickly locate tabs with many of them open, because it does not ship a favicon. So this PR:
- Grabs the favicon `.png` from datasette itself[^1]
- Adds it to the `_static/` folder
- Sets `html_favicon` to load it in the docs
[^1]: I also learned that Chrome can fetch favicons as an internal service! See `chrome://favicon/https://datasette.io/tools/github-to-sqlite`.
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1967/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1501900064,I_kwDOBm6k_c5ZhS0g,1966,Broken link to live demo in Getting started docs,7551922,lbellomo,closed,0,,,,,1,2022-12-18T13:17:00Z,2022-12-31T19:15:19Z,2022-12-31T19:15:10Z,NONE,,The link in [Play with a live demo in Getting started](https://github.com/simonw/datasette/blob/main/docs/getting_started.rst#play-with-a-live-demo) to [https://fivethirtyeight.datasettes.com/fivethirtyeight](https://fivethirtyeight.datasettes.com/fivethirtyeight) is broken and the datasette is no longer working (maybe due to the end of the free tier).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1966/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1515186569,I_kwDOBm6k_c5aT-mJ,1972,Fix Sphinx warning about extlink extension,9599,simonw,closed,0,,,,,0,2022-12-31T19:12:04Z,2022-12-31T19:13:26Z,2022-12-31T19:13:26Z,OWNER,,"```
[sphinx-autobuild] > sphinx-build -b html /Users/simon/Dropbox/Development/datasette/docs /Users/simon/Dropbox/Development/datasette/docs/_build
Running Sphinx v5.3.0
loading pickled environment... done
WARNING: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'.
```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1971#issuecomment-1368266904_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1972/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1115435536,I_kwDOBm6k_c5CfDIQ,1614,Try again with SQLite codemirror support,9599,simonw,open,0,,,,,1,2022-01-26T20:05:20Z,2022-12-23T21:27:10Z,,OWNER,,"I tried and failed to implement autocomplete a while ago. Relevant code:
https://github.com/codemirror/legacy-modes/blob/8f36abca5f55024258cd23d9cfb0203d8d244f0d/mode/sql.js#L335
Sounds like upgrading to CodeMirror 6 ASAP would be worthwhile since it has better accessibility and touch screen support: https://codemirror.net/6/",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1614/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1501843596,PR_kwDOBm6k_c5FuaJm,1965,Detect server start/stop more reliably.,11321,janl,closed,0,,,,,2,2022-12-18T10:03:42Z,2022-12-20T19:08:26Z,2022-12-18T16:01:51Z,CONTRIBUTOR,simonw/datasette/pulls/1965,"This is useful, especially in testing, since your test hosts might not reliabliy start the server within two seconds, so we do a definite check before progressing.
By the same token, after `kill $server_pid` wait for the pid to be gone from the process list.
Since now the script can end prematurely, I also added a cleanup function to make sure the temporary certs are removed in any case.
n.b. this could also be done with the use of `trap 'fn' ERR` but that felt like a bit too much magic for this short a script.
----
:books: Documentation preview :books:: https://datasette--1965.org.readthedocs.build/en/1965/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1965/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1504352503,I_kwDOBm6k_c5Zqpj3,1968,Allow to hide some queries in metadata.yml,562352,CharlesNepote,open,0,,,,,0,2022-12-20T10:45:41Z,2022-12-20T10:45:41Z,,NONE,,"By default all queries are displayed.
But there are many cases where it would be interesting to hide the queries by default:
* the website is targeting non-tech people
* the query is veeeeeery long ([eg.](https://mirabelle.openfoodfacts.org/products/energy_calculator))
* reading the query is not important for the users, they only want to see the result
Of course, the user still could have the option to see the query.
It could be an option in the metadata file:
```yml
databases:
awesome_db:
tables:
products:
hide_sql: true
queries:
great_query:
hide_sql: true
sql: select * from products where code = :barcode
```
The priority could be:
* no option in the metadata and nothing in the URL: query displayed
* hide_sql in the metadata and nothing in the URL: query displayed as asked in the metadata
* hide_sql in the metadata and &_hide_sql= in the URL: query as asked in the URL
See also: #1824
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1968/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1496652622,I_kwDOBm6k_c5ZNRtO,1955,"invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things",32839123,Rik-de-Kort,closed,0,,,,,36,2022-12-14T13:39:56Z,2022-12-19T04:34:16Z,2022-12-18T02:45:18Z,NONE,,"In the past (pre-september 14, #1809) I had a running deployment of Datasette on Azure WebApps by emulating the call in cli.py to Gunicorn: `gunicorn -w 2 -k uvicorn.workers.UvicornWorker app:app`.
My most recent deployment, however, fails loudly by shouting that `Datasette.invoke_startup()` was not called. It does not seem to be possible to call `invoke_startup` when running using a uvicorn command directly like this (I've reproduced this locally using `uvicorn`). Two candidates that I have tried:
* Uvicorn has a `--factory` option, but the app factory has to be synchronous, so no `await invoke_startup` there
* `asyncio.get_event_loop().run_until_complete` is also not an option because `uvicorn` already has the event loop running.
One additional option is:
* Use Gunicorn's [server hooks](https://docs.gunicorn.org/en/stable/settings.html#server-hooks) to call `invoke_startup`. These are also synchronous, but I might be able to get ahead of the event loop starting here.
In my current deployment setup, it does not appear to be possible to use `datasette serve` directly, so I'm stuck either
* Trying to rework my complete deployment setup, for instance, using Azure functions as described [here](https://github.com/simonw/azure-functions-datasette))
* Or dig into the ASGI spec and write a wrapper for the sole purpose of launching Datasette using a direct Uvicorn invocation.
Questions for the maintainers:
* Is this intended behaviour/will not support/etc.? If so, I'd be happy to add a PR with a couple lines in the documentation.
* if this is not intended behaviour, what is a good way to fix it? I could have a go at the ASGI spec thing (I think the Azure Functions thing is related) and provide a PR with the wrapper here, but I'm all ears!
Almost forgot, minimal reproducer:
```python
from datasette import Datasette
ds = Datasette(files=['./global-power-plants.db'])]
app = ds.app()
```
Save as app.py in the same folder as global-power-plants.db, and then try running
`uvicorn app:app`.
Opening the resulting Datasette instance in the browser will show the error message.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1955/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1447050738,I_kwDOBm6k_c5WQD3y,1886,"Call for birthday presents: if you're using Datasette, let us know how you're using it here",9599,simonw,open,0,,,,,13,2022-11-13T19:25:51Z,2022-12-18T17:34:20Z,,OWNER,,"Datasette is 5 years old today. To celebrate, I'm asking the community for birthday presents:
https://simonwillison.net/2022/Nov/13/datasette-birthday/
> To celebrate this open source project’s birthday, I’ve decided to try something new: I’m going to ask for birthday presents.
>
> An aspect of Datastte’s marketing that I’ve so far neglected is social proof. I think it’s time to change that: I know people are using the software to do cool things, but this often happens behind closed doors.
>
> For Datastte’s birthday, I’m looking for endorsements and case studies and just general demonstrations that show how people are using it do so cool stuff.
>
> So: if you’ve used Datasette to solve a problem, and you’re willing to publicize it, please give us the gift of your endorsement!
>
> [...]
>
> Add a comment to [this issue thread](https://github.com/simonw/datasette/issues/1886) describing what you’re doing. Just a few sentences is fine—though a screenshot or even a link to a live instance would be even better",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1886/reactions"", ""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1501778647,I_kwDOBm6k_c5Zg1LX,1964,Cog menu is not keyboard accessible (also no ARIA),9599,simonw,open,0,,,,,1,2022-12-18T06:36:28Z,2022-12-18T06:37:28Z,,OWNER,,"This menu here: https://latest.datasette.io/fixtures/attraction_characteristic
You can tab to it (see the outline) and hit space or enter to open it, but you can't then navigate the items in the open menu using the keyboard.
![cog-menu](https://user-images.githubusercontent.com/9599/208284973-2a04cdab-ed95-4316-979c-67fe5f7787db.gif)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1964/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1306984363,I_kwDOBm6k_c5N5v-r,1771,minor a11y: has no visual indicator when tabbed to,1473102,mustafa0x,closed,0,,,,,5,2022-07-17T04:30:14Z,2022-12-18T06:34:20Z,2022-12-18T06:28:12Z,NONE,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1771/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1501713288,I_kwDOBm6k_c5ZglOI,1963,0.63.3 bugfix release,9599,simonw,closed,0,,,,,2,2022-12-18T02:48:15Z,2022-12-18T03:26:55Z,2022-12-18T03:26:55Z,OWNER,,"I'm going to ship a release which back-ports these two fixes:
- https://github.com/simonw/datasette/issues/1958
- https://github.com/simonw/datasette/issues/1955",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1963/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
806849424,MDU6SXNzdWU4MDY4NDk0MjQ=,1221,Support SSL/TLS directly,9599,simonw,closed,0,,,,,4,2021-02-12T00:18:29Z,2022-12-18T02:39:04Z,2021-02-12T00:52:18Z,OWNER,,This should be pretty easy because Uvicorn supports them already. Need a good mechanism for testing it - https://pypi.org/project/trustme/ looks ideal.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1221/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1408757705,I_kwDOBm6k_c5T9-_J,1843,"Intermittent ""Too many open files"" error running tests",9599,simonw,open,0,,,,,16,2022-10-14T04:45:01Z,2022-12-17T22:02:41Z,,OWNER,,"Partial stack trace from one of them:
```
/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/jinja2/loaders.py:200: in get_source
f = open_if_exists(filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
filename = '/Users/simon/Dropbox/Development/datasette/datasette/templates/error.html', mode = 'rb'
def open_if_exists(filename: str, mode: str = ""rb"") -> t.Optional[t.IO]:
""""""Returns a file descriptor for the filename if that file exists,
otherwise ``None``.
""""""
if not os.path.isfile(filename):
return None
> return open(filename, mode)
E OSError: [Errno 24] Too many open files: '/Users/simon/Dropbox/Development/datasette/datasette/templates/error.html'
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1843/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened
1499081664,I_kwDOBm6k_c5ZWivA,1959,Refactor test suite to use mostly `async def` tests,9599,simonw,closed,0,,,,,9,2022-12-15T21:02:54Z,2022-12-17T21:49:37Z,2022-12-17T21:49:36Z,OWNER,,"I got blocked working on this issue due to weird and hard-to-debug test suite problems:
- #1955
The test suite has needed a major upgrade for several years now. It has a LOT of `def test_...` synchronous functions that could be upgraded to `async def` for better performance and less test complexity - I've used the new `async def` pattern in plugins and new tests for a couple of years now.
Hopefully I can get more of the tests to use in-memory named databases too, ideally so I can fix this consistent problem:
- #1843",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1959/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1499150951,PR_kwDOBm6k_c5FlZmG,1960,Port as many tests as possible to async def tests against ds_client,9599,simonw,closed,0,,,,,29,2022-12-15T21:45:53Z,2022-12-17T21:47:56Z,2022-12-17T21:47:55Z,OWNER,simonw/datasette/pulls/1960,"Refs:
- #1959
----
:books: Documentation preview :books:: https://datasette--1960.org.readthedocs.build/en/1960/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1960/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1500636982,I_kwDOBm6k_c5Zcec2,1962,"Alternative, async-friendly pattern for `make_app_client()` and similar - fully retire `TestClient`",9599,simonw,open,0,,,,,1,2022-12-16T17:56:51Z,2022-12-16T21:55:29Z,,OWNER,,"In this issue I replaced a whole bunch of places that used the non-async `app_client` fixture with an async `ds_client` fixture instead:
- #1959
But I didn't get everything, and a lot of tests are still using the old `TestClient` mechanism as a result.
The main work here is replacing all of the `app_client_...` fixtures which use variants on the default client - and changing the tests that call `make_app_client()` to do something else instead.
This requires some careful thought. I need to come up with a really nice pattern for creating variants on the `ds_client` default fixture - and do so in a way that minimizes the number of open files, refs:
- #1843",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1962/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1497909798,I_kwDOBm6k_c5ZSEom,1958,datasette --root running in Docker doesn't reliably show the magic URL,11729897,davidhaley,closed,0,,,,,11,2022-12-13T16:29:13Z,2022-12-16T00:59:12Z,2022-12-16T00:55:19Z,NONE,,"I followed these steps:
`docker run datasetteproject/datasette pip install datasette-upload-csvs`
`docker commit $(docker ps -lq) datasette-with-plugins`
`docker run -p 8001:8001 -v $(pwd):/mnt datasette-with-plugins datasette --root -p 8001 -h 0.0.0.0`
Visited: http://127.0.0.1:8001/-/plugins
![image](https://user-images.githubusercontent.com/11729897/207392071-d939cd5e-1d96-4e11-b0be-dc06dd207866.png)
Visited: http://localhost:8001/-/upload-csvs
![image](https://user-images.githubusercontent.com/11729897/207389241-3e96ca66-ca74-4a16-8b7d-4427ee862c5e.png)
I may have missed a step?
Thank you.
---
Ubuntu 22.04.1 LTS",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1958/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1499387139,PR_kwDOBm6k_c5FmNXs,1961,Use click.echo() instead of print() for --root option,9599,simonw,closed,0,,,,,0,2022-12-16T00:54:56Z,2022-12-16T00:55:19Z,2022-12-16T00:55:18Z,OWNER,simonw/datasette/pulls/1961,"This ensures the URL is output correctly when running under Docker.
Closes #1958
----
:books: Documentation preview :books:: https://datasette--1961.org.readthedocs.build/en/1961/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1961/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1495821607,I_kwDOBm6k_c5ZKG0n,1953,Release notes for Datasette 1.0a2,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,2,2022-12-14T06:26:40Z,2022-12-15T02:02:15Z,2022-12-15T02:01:08Z,OWNER,,"https://github.com/simonw/datasette/milestone/27?closed=1
https://github.com/simonw/datasette/compare/1.0a1...9ad76d279e2c3874ca5070626a25458ce129f126",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1953/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1473411197,I_kwDOBm6k_c5X0nh9,1927,ignore:true/replace:true options for /db/-/create API,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,5,2022-12-02T20:32:30Z,2022-12-15T01:47:01Z,2022-12-08T01:43:01Z,OWNER,,"See also:
- #1924
It turns out I want to be able to call `/db/-/create` multiple times with the `rows` argument, so that I don't have to worry about creating the table first.
As such I find myself wanting support for the `""insert"": true` and `""replace"": true` options as well.
Still TODO:
- [x] A test for the case where you call `/-/create` twice with `rows` without using these options
- [x] `pk` should be required if you are using these options
- [x] Error if you pass `pk` and the table exists already but has a different `pk`
- [x] Documentation for `insert` and `replace` - and what happens if you repeat a `/-/create` with rows generally
- [x] Documentation should explain that you are allowed to call `/-/create` more than once using `rows`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1927/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1493471221,I_kwDOBm6k_c5ZBI_1,1949,`.json` errors should be returned as JSON,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,10,2022-12-13T06:14:12Z,2022-12-15T00:46:27Z,,OWNER,,"Eg the error in this issue:
- #1945 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1949/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1490576818,I_kwDOBm6k_c5Y2GWy,1943,`/-/permissions` should list available permissions,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-12-11T23:38:03Z,2022-12-15T00:41:37Z,,OWNER,,"> Idea: a `/-/permissions` introspection endpoint for listing registered permissions
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1939#issuecomment-1345691103_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1943/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1497577017,I_kwDOBm6k_c5ZQzY5,1957,Reconsider row value truncation on query page,9599,simonw,open,0,,,,,1,2022-12-14T23:49:47Z,2022-12-14T23:50:50Z,,OWNER,,"Consider this example: https://ripgrep.datasette.io/repos?sql=select+json_group_array%28full_name%29+from+repos
```sql
select json_group_array(full_name) from repos
```
![CleanShot 2022-12-14 at 15 48 32@2x](https://user-images.githubusercontent.com/9599/207739709-8177f683-f938-49a1-8225-42791fad88fe.png)
My intention here was to get a string of JSON I can copy and paste elsewhere - see: https://til.simonwillison.net/sqlite/compare-before-after-json
The truncation isn't helping here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1957/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
855296937,MDU6SXNzdWU4NTUyOTY5Mzc=,1295,Errors should have links to further information,9599,simonw,open,0,,,,,2,2021-04-11T12:39:12Z,2022-12-14T23:28:49Z,,OWNER,,"Inspired by this tweet:
https://twitter.com/willmcgugan/status/1381186384510255104
> While I am thinking about faqs. I’d also like to add short URLs to Rich exceptions.
>
> I loath cryptic error messages, and I’ve created a fair few myself. In Rich I’ve tried to make them as plain English as possible. But...
>
> would be great if every error message linked to a page that explains the error in detail and offers fixes.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1295/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1483320357,I_kwDOBm6k_c5Yaawl,1937,/db/-/create API should require insert-rows permission to use row: or rows: option,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,2,2022-12-08T01:33:09Z,2022-12-14T20:21:26Z,2022-12-14T20:21:26Z,OWNER,,Otherwise someone with `create-table` but no` insert-rows` permission could abuse it to insert data.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1937/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1497288666,I_kwDOBm6k_c5ZPs_a,1956,Handle abbreviations properly in permission_allowed_actor_restrictions,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,2,2022-12-14T19:54:21Z,2022-12-14T20:04:29Z,2022-12-14T20:04:28Z,OWNER,,"This code currently assumes abbreviations are:
```pyton
action_initials = """".join([word[0] for word in action.split(""-"")])
```
https://github.com/simonw/datasette/blob/1a3dcf494376e32f7cff110c86a88e5b0a3f3924/datasette/default_permissions.py#L182-L208
That's no longer correct, they are now registered by the new plugin hook:
- #1939 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1956/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1495716243,I_kwDOBm6k_c5ZJtGT,1952,Improvements to /-/create-token restrictions interface,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-12-14T05:22:39Z,2022-12-14T05:23:13Z,,OWNER,,"> It would be neat not to show write permissions against immutable databases too - and not hard from a performance perspective since it doesn't involve hundreds more permission checks.
>
> That will need permissions to grow a flag for if they need a mutable database though, which is a bigger job.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1947#issuecomment-1350414402_
Also, DO show the `_memory` database there if Datasette was started in `--crossdb` mode.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1952/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1493390939,I_kwDOBm6k_c5ZA1Zb,1947,UI to create reduced scope tokens from the `/-/create-token` page,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,22,2022-12-13T05:10:48Z,2022-12-14T05:22:00Z,2022-12-14T05:13:24Z,OWNER,,"Split from:
- #1855",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1947/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1495431932,I_kwDOBm6k_c5ZInr8,1951,`datasette.create_token(...)` method for creating signed API tokens,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,6,2022-12-14T01:25:34Z,2022-12-14T02:43:45Z,2022-12-14T02:42:05Z,OWNER,,"I need this for:
- #1947
And I can refactor this to use it too:
- #1855
By making this a documented internal API it can be used by other plugins too. It's also going to be really useful for writing tests.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1951/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1423336089,I_kwDOBm6k_c5U1mKZ,1855,`datasette create-token` ability to create tokens with a reduced set of permissions,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,19,2022-10-26T02:20:52Z,2022-12-14T01:24:49Z,2022-12-13T05:20:24Z,OWNER,,"Initial design ideas: https://github.com/simonw/datasette/issues/1852#issuecomment-1289733483
> Token design concept:
>
> ```json
> {
> ""t"": {
> ""a"": [""ir"", ""ur"", ""dr""],
> ""d"": {
> ""fixtures"": [""ir"", ""ur"", ""dr""]
> },
> ""t"": {
> ""fixtures"": {
> ""searchable"": [""ir""]
> }
> }
> }
> }
> ```
>
> That JSON would be minified and signed.
>
> Minified version of the above looks like this (101 characters):
>
> `{""t"":{""a"":[""ir"",""ur"",""dr""],""d"":{""fixtures"":[""ir"",""ur"",""dr""]},""t"":{""fixtures"":{""searchable"":[""ir""]}}}}`
>
> The `""t""` key shows this is a token that as a default API key.
>
> `""a""` means ""all"" - these are permissions that have been granted on all tables and databases.
>
> `""d""` means ""databases"" - this is a way to set permissions for all tables in a specific database.
>
> `""t""` means ""tables"" - this lets you set permissions at a finely grained table level.
>
> Then the permissions themselves are two character codes which are shortened versions - so:
>
> * `ir` = `insert-row`
> * `ur` = `update-row`
> * `dr` = `delete-row`
## Remaining tasks
- [x] Add these options to the `datasette create-token` command
- [x] Tests for `datasette create-token` options
- [x] Documentation for those options at https://docs.datasette.io/en/latest/authentication.html#datasette-create-token
- [x] A way to handle permissions that don't have known abbreviations (permissions added by plugins). Probably need to solve the plugin permission registration problem as part of that
- [x] Stop hard-coding names of actions in the `permission_allowed_actor_restrictions` function",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1855/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1495241162,I_kwDOBm6k_c5ZH5HK,1950,"Bad ?_sort returns a 500 error, should be a 400",9599,simonw,closed,0,,,,,2,2022-12-13T22:08:16Z,2022-12-13T22:23:22Z,2022-12-13T22:23:22Z,OWNER,,"https://latest.datasette.io/fixtures/facetable?_sort=bad
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1950/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1468689139,I_kwDOBm6k_c5Ximrz,1914,Finalize design of JSON for Datasette 1.0,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-11-29T20:59:10Z,2022-12-13T06:15:54Z,,OWNER,,"Tracking issue.
- [ ] #1709
- [ ] #1729
- [ ] #1875",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1914/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1468495358,I_kwDOBm6k_c5Xh3X-,1910,Check incoming column types on various write APIs,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,0,2022-11-29T18:09:10Z,2022-12-13T05:29:09Z,,OWNER,,"> I do think this needs type checking - I just tried and you really can send a string to an integer column and have it work, which feels bad.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1331089156_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1910/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1428630253,I_kwDOBm6k_c5VJyrt,1873,Ensure insert API has good tests for rowid and compound primark key tables,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,11,2022-10-30T06:22:17Z,2022-12-13T05:29:08Z,,OWNER,,"Following:
- #1866
I need to design and implement various edge-cases or primary keys:
- Table without an auto-incrementing primary key
- Table with compound primary keys
- Table with just a `rowid`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1873/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened
1430797211,I_kwDOBm6k_c5VSDub,1875,Figure out design for JSON errors (consider RFC 7807),9599,simonw,open,0,,,8755003,Datasette 1.0a-next,7,2022-11-01T03:14:15Z,2022-12-13T05:29:08Z,,OWNER,,"https://datatracker.ietf.org/doc/draft-ietf-httpapi-rfc7807bis/ is a brand new standard.
Since I need a neat, predictable format for my JSON errors, maybe I should use this one?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1875/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1447465004,I_kwDOBm6k_c5WRpAs,1889,Ability to create new tokens via the API,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,0,2022-11-14T06:21:36Z,2022-12-13T05:29:08Z,,OWNER,,"Refs:
- #1850
Initially I decided that the API shouldn't be able to create new tokens at all - I don't like the idea of an API token holder creating themselves additional tokens.
Then I realized that two of the API features are specifically more useful if you can generate fresh tokens via the API:
- Tokes that expire after a time limit are MUCH more useful if they can be automatically generated
- Likewise, tokens that are restricted to a subset of permissions (see #1855) make more sense to be generated like this, especially in conjunction with expiry times",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1889/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1216436131,I_kwDOBm6k_c5IgVej,1721,"Implement plugin hooks: `register_table_extras`, `register_row_extras`, `register_query_extras`",9599,simonw,open,0,,,8755003,Datasette 1.0a-next,0,2022-04-26T20:21:49Z,2022-12-13T05:29:07Z,,OWNER,,"Designed in:
- #1720
Part of:
- #262
- #1709",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1721/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1219385669,I_kwDOBm6k_c5IrllF,1729,Implement ?_extra and new API design for TableView,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,12,2022-04-28T22:28:14Z,2022-12-13T05:29:07Z,,OWNER,,"Part of:
- #262
- #1518",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1729/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1200649124,I_kwDOBm6k_c5HkHOk,1708,Datasette 1.0 alpha upcoming release notes,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,2,2022-04-11T22:57:12Z,2022-12-13T05:29:06Z,,OWNER,,"I'm going to try writing the release notes first, to see if that helps unblock me.
# ⚠️ Any release notes in this issue are a draft, and should not be treated as the real thing ⚠️ ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1708/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1200649502,I_kwDOBm6k_c5HkHUe,1709,Redesigned JSON API with ?_extra= parameters,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-04-11T22:57:49Z,2022-12-13T05:29:06Z,,OWNER,,This will be the single biggest breaking change for the 1.0 release.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1709/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1200650491,I_kwDOBm6k_c5HkHj7,1711,Template context powered entirely by the JSON API format,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-04-11T22:59:27Z,2022-12-13T05:29:06Z,,OWNER,,Datasette 1.0 will have a stable template context. I'm going to achieve this by refactoring the templates to work only with keys returned by the API (or some of its extras) - then the API documentation will double up as template documentation.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1711/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1197926598,I_kwDOBm6k_c5HZujG,1705,How to upgrade your plugin for 1.0 documentation,9599,simonw,open,0,,,8755003,Datasette 1.0a-next,1,2022-04-08T23:16:47Z,2022-12-13T05:29:05Z,,OWNER,,"Among other things, needed by:
- #1704",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1705/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1491840863,PR_kwDOBm6k_c5FMKSG,1944,Bump black from 22.10.0 to 22.12.0,49699333,dependabot[bot],closed,0,,,,,0,2022-12-12T13:05:11Z,2022-12-13T05:23:31Z,2022-12-13T05:23:30Z,CONTRIBUTOR,simonw/datasette/pulls/1944,"Bumps [black](https://github.com/psf/black) from 22.10.0 to 22.12.0.
Release notes
Sourced from black's releases .
22.12.0
Preview style
Enforce empty lines before classes and functions with sticky leading comments (#3302 )
Reformat empty and whitespace-only files as either an empty file (if no newline is
present) or as a single newline character (if a newline is present) (#3348 )
Implicitly concatenated strings used as function args are now wrapped inside
parentheses (#3307 )
Correctly handle trailing commas that are inside a line's leading non-nested parens
(#3370 )
Configuration
Fix incorrectly applied .gitignore
rules by considering the .gitignore
location
and the relative path to the target file (#3338 )
Fix incorrectly ignoring .gitignore
presence when more than one source directory is
specified (#3336 )
Parser
Parsing support has been added for walruses inside generator expression that are
passed as function args (for example,
any(match := my_re.match(text) for text in texts)
) (#3327 ).
Integrations
Vim plugin: Optionally allow using the system installation of Black via
let g:black_use_virtualenv = 0
(#3309 )
Changelog
Sourced from black's changelog .
22.12.0
Preview style
Enforce empty lines before classes and functions with sticky leading comments (#3302 )
Reformat empty and whitespace-only files as either an empty file (if no newline is
present) or as a single newline character (if a newline is present) (#3348 )
Implicitly concatenated strings used as function args are now wrapped inside
parentheses (#3307 )
Correctly handle trailing commas that are inside a line's leading non-nested parens
(#3370 )
Configuration
Fix incorrectly applied .gitignore
rules by considering the .gitignore
location
and the relative path to the target file (#3338 )
Fix incorrectly ignoring .gitignore
presence when more than one source directory is
specified (#3336 )
Parser
Parsing support has been added for walruses inside generator expression that are
passed as function args (for example,
any(match := my_re.match(text) for text in texts)
) (#3327 ).
Integrations
Vim plugin: Optionally allow using the system installation of Black via
let g:black_use_virtualenv = 0
(#3309 )
Commits
2ddea29
Prepare release 22.12.0 (#3413 )
5b1443a
release: skip bad macos wheels for now (#3411 )
9ace064
Bump peter-evans/find-comment from 2.0.1 to 2.1.0 (#3404 )
19c5fe4
Fix CI with latest flake8-bugbear (#3412 )
d4a8564
Bump sphinx-copybutton from 0.5.0 to 0.5.1 in /docs (#3390 )
2793249
Wordsmith current_style.md (#3383 )
d97b789
Remove whitespaces of whitespace-only files (#3348 )
c23a5c1
Clarify that Black runs with --safe by default (#3378 )
8091b25
Correctly handle trailing commas that are inside a line's leading non-nested ...
ffaaf48
Compare each .gitignore found with an appropiate relative path (#3338 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.10.0&new-version=22.12.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1944/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1485488236,PR_kwDOBm6k_c5E1iJG,1938,"""permissions"" blocks in metadata.json/yaml",9599,simonw,closed,0,,,8711695, Datasette 1.0a2,3,2022-12-08T22:07:36Z,2022-12-13T05:23:18Z,2022-12-13T05:23:18Z,OWNER,simonw/datasette/pulls/1938,"Refs #1636
- [x] Documentation
- [ ] Implementation
- [ ] Validate metadata to check there are no nonsensical permissions (like `debug-menu` set at the table level)
- [ ] Tests
----
:books: Documentation preview :books:: https://datasette--1938.org.readthedocs.build/en/1938/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1938/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1493404423,I_kwDOBm6k_c5ZA4sH,1948,500 error on permission debug page when testing actors with _r,9599,simonw,open,0,,,,,1,2022-12-13T05:22:03Z,2022-12-13T05:22:19Z,,OWNER,,"
The 500 error is silent unless you are looking at the DevTools network pane.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1948/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1493339206,I_kwDOBm6k_c5ZAoxG,1946,`datasette --get` mechanism for sending tokens,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,2,2022-12-13T04:25:05Z,2022-12-13T04:36:57Z,2022-12-13T04:36:57Z,OWNER,,"> For the tests for `datasette create-token` it would be useful if `datasette --get` had a mechanism for sending an `Authorization: Bearer X` header.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1855#issuecomment-1347731288_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1946/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1493306655,I_kwDOBm6k_c5ZAg0f,1945,`view-instance` should not be checked for /-/actor.json,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,0,2022-12-13T04:01:46Z,2022-12-13T04:11:56Z,2022-12-13T04:11:56Z,OWNER,,"Spotted this while testing:
- #1855
```
export TOKEN=$(datasette create-token root --secret s -a foo)
curl -H ""Authorization: Bearer $TOKEN"" http://localhost:8002/-/actor.json
```
Returned a Forbidden error (and not in JSON either).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1945/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1138008042,I_kwDOBm6k_c5D1J_q,1636,"""permissions"" propery in metadata for configuring arbitrary permissions",9599,simonw,closed,0,,,8711695, Datasette 1.0a2,14,2022-02-15T00:25:59Z,2022-12-13T02:40:50Z,2022-12-13T02:40:50Z,OWNER,,"The `""allow""` block mechanism can already be used to configure various default permissions. When adding permissions to `datasette-tiddlywiki` I realized it would be good to be able to configure arbitrary permissions such as `edit-tiddlywiki` there too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1636/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1485757511,I_kwDOBm6k_c5YjtxH,1939,register_permissions(datasette) plugin hook,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,20,2022-12-09T01:33:25Z,2022-12-13T02:07:50Z,2022-12-13T02:05:56Z,OWNER,,"A plugin hook that adds more named permissions to the list which is initially populated here:
https://github.com/simonw/datasette/blob/e539c1c024bc62d88df91d9107cbe37e7f0fe55f/datasette/permissions.py#L1-L19
Originally imagined this hook in this comment:
- https://github.com/simonw/datasette/issues/1881#issuecomment-1301639370
I need this for a few reasons:
- https://github.com/simonw/datasette/issues/1636
- Needs it in order to validate that permissions defined in `metadata.json` are set in the right place (don't set an instance permissions at table level for example)
- https://github.com/simonw/datasette/issues/1855
- Needs it to be able to register additional abbreviations for use in signed cookies
- And for validation when you use `datasette create-token` and pass in extra permissions
- The https://latest.datasette.io/-/permissions debug interface needs it to add extra debug options to the ``",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1939/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1486011362,PR_kwDOBm6k_c5E3XqB,1940,register_permissions() plugin hook,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,6,2022-12-09T05:09:28Z,2022-12-13T02:05:55Z,2022-12-13T02:05:54Z,OWNER,simonw/datasette/pulls/1940,"Refs #1939
From this comment: https://github.com/simonw/datasette/issues/1939#issuecomment-1343872168
- [x] Unit test for the registration plugin hook itself
- [x] Use them in `check_permission_actions_are_documented` test in `conftest.py`
- [x] Add description field to `Permissions` (and update tests and docs)
- [x] Documentation for `datasette.permissions` dictionary
- [x] If no `default=` provided in call to `permission_allowed()` then use default from `datasette.permissions` list
- [x] Remove `default=` from a bunch of places
- [x] Throw an error if two permissions are registered with the same name or abbreviation (but other attributes differ)
- [x] Update authentication and permissions documentation to explain that permissions are now registered and have a registered default
----
:books: Documentation preview :books:: https://datasette--1940.org.readthedocs.build/en/1940/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1940/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1384549993,I_kwDOBm6k_c5Sho5p,1818,Setting to turn off table row counts entirely,9599,simonw,open,0,,,,,4,2022-09-24T06:39:22Z,2022-12-11T02:03:09Z,,OWNER,,"There are situations - such as loading SQLite files remotely using HTTP range headers - where counting all of the rows in a table should be avoided entirely.
> > Also, this chunked inefficiency means that I have to hack the URL to not load tables of a database as it seems to try to load the whole database when I click on a database.
>
> I bet that's because Datasette tries to show a count of all of the rows in each table when it shows the list on that page, which triggers a full table scan.
>
> Would be great to have a setting that turns that feature off, which could then be exposed as a query string option for Datasette Lite.
_Originally posted by @simonw in https://github.com/simonw/datasette-lite/issues/49#issuecomment-1256880715_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1818/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1487738738,I_kwDOBm6k_c5YrRdy,1942,Option for plugins to request that JSON be served on the page,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2022-12-10T01:08:53Z,2022-12-10T01:11:30Z,,OWNER,,"Idea came from a conversation with @hydrosquall - what if a Datasette plugin could say ""I'd like the JSON for a page to be included in a variable on the HTML page""?
`datasette-cluster-map` already needs this - the first thing it does when the page loads is `fetch()` a JSON representation of that same data.
This idea fits with my overall goals to unify the JSON and HTML context too.
Refs:
- #1711",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1942/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1198822563,I_kwDOBm6k_c5HdJSj,1706,"[feature] immutable mode for a directory, not just individual sqlite file",9020979,hydrosquall,open,0,,,,,4,2022-04-10T00:50:57Z,2022-12-09T19:11:40Z,,CONTRIBUTOR,,"## Motivation
- I have a directory of sqlite databases
- I'd like to use immutable mode when opening them for better performance [docs](https://docs.datasette.io/en/0.54/performance.html#immutable-mode)
- Currently using this flag throws the following error
IsADirectoryError: [Errno 21] Is a directory: '/name-of-directory'
## Proposal
Immutable flag works for both single files and directories
datasette -i /folder-of-sqlite-files",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1706/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1486036269,I_kwDOBm6k_c5Ykx0t,1941,Mechanism for supporting key rotation for DATASETTE_SECRET,9599,simonw,open,0,,,,,1,2022-12-09T05:24:53Z,2022-12-09T05:25:20Z,,OWNER,,"Currently if you change `DATASETTE_SECRET` all existing signed tokens - both cookies and API tokens and potentially other things too - will instantly expire.
Adding support for key rotation would allow keys to be rotated on a semi-regular basis without logging everyone out / invalidating every API token instantly.
Can model this on how Django does it: https://github.com/django/django/commit/0dcd549bbe36c060f536ec270d34d9e7d4b8e6c7",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1941/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1434094365,I_kwDOBm6k_c5Veosd,1881,Tool for simulating permission checks against actors,9599,simonw,closed,0,,,,,9,2022-11-03T04:43:20Z,2022-12-09T01:38:21Z,2022-11-04T00:13:05Z,OWNER,,"In working on this issue:
- #1855
I realized that if I'm going to make actors more complicated (the proposed `_r` key for additional restricted permissions) I really need an interactive tool for simulating these checks, similar to the https://latest.datasette.io/-/allow-debug tool.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1881/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1473664029,PR_kwDOBm6k_c5ELz0u,1930,Typo in JSON API `Updating a row` documentation,3556,davidbgk,closed,0,,,,,2,2022-12-03T02:22:31Z,2022-12-08T21:12:35Z,2022-12-08T21:12:35Z,CONTRIBUTOR,simonw/datasette/pulls/1930,"
----
:books: Documentation preview :books:: https://datasette--1930.org.readthedocs.build/en/1930/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1930/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1481875485,PR_kwDOBm6k_c5EouZs,1935,Bump furo from 2022.9.29 to 2022.12.7,49699333,dependabot[bot],closed,0,,,,,1,2022-12-07T13:02:57Z,2022-12-08T21:12:08Z,2022-12-08T21:12:07Z,CONTRIBUTOR,simonw/datasette/pulls/1935,"Bumps [furo](https://github.com/pradyunsg/furo) from 2022.9.29 to 2022.12.7.
Changelog
Sourced from furo's changelog .
Changelog
2022.12.07 -- Reverent Raspberry
✨ Add support for Sphinx 6.
✨ Improve footnote presentation with docutils 0.18+.
Drop support for Sphinx 4.
Improve documentation about what the edit button does.
Improve handling of empty-flexboxes for better print experience on Chrome.
Improve styling for inline signatures.
Replace the meta
generator tag with a comment.
Tweak labels with icons to prevent users selecting icons as text on touch.
2022.09.29 -- Quaint Quartz
Add ability to set arbitrary URLs for edit button.
Add support for aligning text in MyST-parser generated tables.
2022.09.15 -- Pragmatic Pistachio
Add a minimum version constraint on pygments.
Add an explicit dependency on sass
.
Change right sidebar title from "Contents" to "On this page".
Correctly position sidebars on small screens.
Correctly select only Furo's own svg
in related pages nav
.
Make numpy-style documentation headers consistent.
Retitle the reference section.
Update npm dependencies.
2022.06.21 -- Opulent Opal
Fix docutils <= 0.17.x
compatibility.
Bump to the latest Node.js LTS.
2022.06.04.1 -- Naughty Nickel bugfix
Fix the URL used in the "Edit this page" for Read the Docs builds.
2022.06.04 -- Naughty Nickel
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.9.29&new-version=2022.12.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1935.org.readthedocs.build/en/1935/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1935/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1483250004,I_kwDOBm6k_c5YaJlU,1936,Fix /db/table/-/upsert in the API explorer,9599,simonw,open,0,,,3268330,Datasette 1.0,2,2022-12-08T00:59:34Z,2022-12-08T01:36:02Z,,OWNER,,"Split from:
- #1931
- #1878
This is a bit tricky because the code needs to figure out what the primary keys are for an item, and whether or not `rowid` should be included.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1936/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1432013704,I_kwDOBm6k_c5VWsuI,1878,/db/table/-/upsert API,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,8,2022-11-01T20:01:18Z,2022-12-08T01:12:18Z,2022-12-08T01:12:17Z,OWNER,,Equivalent to `sqlite-utils upsert`: https://sqlite-utils.datasette.io/en/stable/python-api.html#upserting-data,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1878/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1473814539,PR_kwDOBm6k_c5EMVug,1931,/db/table/-/upsert,9599,simonw,closed,0,,,8711695, Datasette 1.0a2,8,2022-12-03T07:01:44Z,2022-12-08T01:12:17Z,2022-12-08T01:12:16Z,OWNER,simonw/datasette/pulls/1931,"Refs #1878
Still todo:
- [x] Support `""return"": true` properly for upserts (with tests)
- [x] Require both `insert-row` and `update-row` permissions
- [x] Tests are going to need to cover both rowid-only and compound primary key tables, including all of the error states
- [x] Documentation
----
:books: Documentation preview :books:: https://datasette--1931.org.readthedocs.build/en/1931/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1931/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1473659191,I_kwDOBm6k_c5X1kE3,1929,Incorrect link from the API explorer to the JSON API documentation,3556,davidbgk,closed,0,,,,,4,2022-12-03T02:08:58Z,2022-12-06T19:36:23Z,2022-12-06T19:34:20Z,CONTRIBUTOR,,"I installed `datasette==1.0a1`.
When I go to http://127.0.0.1:8001/-/api I have a link: `Use this tool to try out the [Datasette API](https://docs.datasette.io/en/1.0a1/json_api.html).` but that documentation page does not exist.
I'm not sure where it has to be fixed, should it link to the stable page https://docs.datasette.io/en/stable/json_api.html , the latest one https://docs.datasette.io/en/latest/json_api.html#the-json-write-api or would it be more appropriated to deploy documentation for the `1.0a1` version?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1929/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1479920517,I_kwDOBm6k_c5YNcuF,1934,Return number of ignored/replaced items from /-/insert,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-12-06T19:01:58Z,2022-12-06T19:02:03Z,,OWNER,,"Idea from here:
- https://github.com/simonw/sqlite-utils/issues/516",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1934/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1473481262,I_kwDOBm6k_c5X04ou,1928,Hacker News Datasette write demo,9599,simonw,closed,0,,,,,7,2022-12-02T21:17:41Z,2022-12-02T23:47:11Z,2022-12-02T21:43:19Z,OWNER,,"Idea is to have my existing scraper at https://github.com/simonw/scrape-hacker-news-by-domain also write to my private Datasette Cloud account, then create an atom feed from it.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1928/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175690070,I_kwDOBm6k_c5GE5tW,1676,"Reconsider ensure_permissions() logic, can it be less confusing?",9599,simonw,open,0,,,3268330,Datasette 1.0,3,2022-03-21T17:14:57Z,2022-12-02T01:23:40Z,,OWNER,,"> Updated documentation: https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/docs/internals.rst#await-ensure_permissionsactor-permissions
>
>> This method allows multiple permissions to be checked at onced. It raises a `datasette.Forbidden` exception if any of the checks are denied before one of them is explicitly granted.
>>
>> This is useful when you need to check multiple permissions at once. For example, an actor should be able to view a table if either one of the following checks returns `True` or not a single one of them returns `False`:
>
> That's pretty hard to understand! I'm going to open a separate issue to reconsider if this is a useful enough abstraction given how confusing it is.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1675#issuecomment-1074177827_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1676/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1471969984,I_kwDOBm6k_c5XvHrA,1926,Release notes for 1.0a1 (and release it),9599,simonw,closed,0,,,7867486,Datasette 1.0a1,1,2022-12-01T21:18:12Z,2022-12-01T22:06:13Z,2022-12-01T22:06:12Z,OWNER,,"Mainly CORS support and a few small bug fixes.
Changes: https://github.com/simonw/datasette/compare/1.0a0...99da46f7258225fc6fd8e94ddc20859ccccc4109",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1926/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1214859703,I_kwDOBm6k_c5IaUm3,1719,Refactor `RowView` and remove `RowTableShared`,9599,simonw,closed,0,,,,,3,2022-04-25T18:06:24Z,2022-12-01T21:15:19Z,2022-04-25T18:33:44Z,OWNER,,"> The `RowTableShared` class is making this a whole lot more complicated.
>
> I'm going to split the `RowView` view out into an entirely separate `views/row.py` module.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1715#issuecomment-1108875068_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1719/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1215174094,I_kwDOBm6k_c5IbhXO,1720,Design plugin hook for extras,9599,simonw,closed,0,,,,,14,2022-04-26T00:08:10Z,2022-12-01T21:15:19Z,2022-04-26T20:20:27Z,OWNER,,"Refs:
- #262
- #1709
I realized that this is a really natural plugin hook - and if I design it as a hook I can implement Datasette's core extras as default plugins.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1720/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1212823665,I_kwDOBm6k_c5ISjhx,1715,Refactor TableView to use asyncinject,9599,simonw,closed,0,,,,,13,2022-04-22T21:43:39Z,2022-12-01T21:15:18Z,2022-04-28T22:26:56Z,OWNER,,"I've been working on a dependency injection mechanism in a separate library:
- https://github.com/simonw/asyncinject
I think it's ready to try out with Datasette to see if it's a pattern that will work here.
I'm going to attempt to refactor `TableView` to use it. There are two overall goals here:
- Use `asyncinject` to add parallel execution of some aspects of the table page - most notably I want to be able to execute the `count(*)` query, the `select ...` query, the various faceting queries and the facet suggestion queries in parallel - and measure if doing so is good for performance.
- Use it to execute different output formats (possibly with some changes to the existing `register_output_renderer()` plugin hook). I want CSV and JSON to use the same mechanism that plugins use.
Stretch goal is to get this working with streaming data too, see:
- #1101",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1715/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1469973742,I_kwDOBm6k_c5XngTu,1922,Make sure CORS works for write APIs,9599,simonw,closed,0,,,7867486,Datasette 1.0a1,13,2022-11-30T17:15:55Z,2022-12-01T18:47:00Z,2022-12-01T18:47:00Z,OWNER,,"Split from:
- #1850",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1922/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1470509936,I_kwDOBm6k_c5XpjNw,1924,Docs for replace:true and ignore:true options for insert API,9599,simonw,closed,0,,,7867486,Datasette 1.0a1,4,2022-12-01T01:33:25Z,2022-12-01T18:15:15Z,2022-12-01T02:08:02Z,OWNER,,Equivalent to https://sqlite-utils.datasette.io/en/stable/cli.html#insert-replacing-data,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1924/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1469821027,I_kwDOBm6k_c5Xm7Bj,1921,Document methods to get canned queries,25778,eyeseast,open,0,,,,,0,2022-11-30T15:26:33Z,2022-11-30T23:34:21Z,,CONTRIBUTOR,,"Two methods will get canned queries for a Datasette instance:
[`Datasette.get_canned_queries`](https://github.com/simonw/datasette/blob/main/datasette/app.py#L575) will return all canned queries for a database that an `actor` can see.
[`Datasette.get_canned_query`](https://github.com/simonw/datasette/blob/main/datasette/app.py#L592) will return a single canned query by name.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1921/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1470320227,I_kwDOBm6k_c5Xo05j,1923,latest.datasette.io Cloud Run deploys failing,9599,simonw,closed,0,,,,,3,2022-11-30T22:49:34Z,2022-11-30T23:04:56Z,2022-11-30T23:04:56Z,OWNER,,"https://github.com/simonw/datasette/actions/runs/3587402085/jobs/6038106719v
```
Warning: ""service_account_key"" has been deprecated. Please switch to using google-github-actions/auth which supports both Workload Identity Federation and Service Account Key JSON authentication. For more details, see https://github.com/google-github-actions/setup-gcloud#authorization
Error: google-github-actions/setup-gcloud failed with: failed to execute command `gcloud --quiet auth activate-service-account *** --key-file -`: /opt/hostedtoolcache/gcloud/275.0.0/x64/lib/googlecloudsdk/core/console/console_io.py:544: SyntaxWarning: ""is"" with a literal. Did you mean ""==""?
if answer is None or (answer is '' and default is not None):
ERROR: gcloud failed to load: module 'collections' has no attribute 'MutableMapping'
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1923/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1469796454,I_kwDOBm6k_c5Xm1Bm,1920,Document Datasette.metadata() method,25778,eyeseast,open,0,,,,,0,2022-11-30T15:10:36Z,2022-11-30T15:10:36Z,,CONTRIBUTOR,,"Code is here: https://github.com/simonw/datasette/blob/main/datasette/app.py#L503
This will be the official way to access metadata from plugins.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1920/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1108671952,I_kwDOBm6k_c5CFP3Q,1605,Scripted exports,25778,eyeseast,open,0,,,,,10,2022-01-19T23:45:55Z,2022-11-30T15:06:38Z,,CONTRIBUTOR,,"Posting this while I'm thinking about it: I mentioned at the end of [this thread](https://twitter.com/eyeseast/status/1483893011658551299) that I'm usually doing `datasette --get` to export canned queries.
I used to use a tool called [datafreeze](https://github.com/pudo/datafreeze) to do scripted exports, but that project looks dead now. The ergonomics of it are pretty nice, though, and the `Freezefile.yml` structure is actually not too far from Datasette's canned queries.
This is related to the idea for `datasette query` (#1356) but I think it's a distinct feature. It's most likely a plugin, but I want to raise it here because it's probably something other people have thought about.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1605/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1469044738,I_kwDOBm6k_c5Xj9gC,1918,API explorer should list mutable databases first,9599,simonw,closed,0,,,7867486,Datasette 1.0a1,1,2022-11-30T04:53:33Z,2022-11-30T05:22:07Z,2022-11-30T05:07:56Z,OWNER,,"https://latest.datasette.io/-/api hides `ephemeral` down at the bottom, would be more interesting if it was at the top.
Related:
- #1915 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1918/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1469062686,I_kwDOBm6k_c5XkB4e,1919,Intermittent `test_delete_row` test failure ,9599,simonw,open,0,,,,,1,2022-11-30T05:18:46Z,2022-11-30T05:20:56Z,,OWNER,,"https://github.com/simonw/datasette/actions/runs/3580503393/jobs/6022689591
```
delete_response = await ds_write.client.post(
""/data/{}/{}/-/delete"".format(table, delete_path),
headers={
""Authorization"": ""***"".format(write_token(ds_write)),
},
)
> assert delete_response.status_code == 200
E assert 404 == 200
E + where 404 = .status_code
/home/runner/work/datasette/datasette/tests/test_api_write.py:396: AssertionError
=========================== short test summary info ============================
FAILED tests/test_api_write.py::test_delete_row[compound_pk_table-row_for_create2-pks2-article,k] - assert 404 == 200
+ where 404 = .status_code
```
This passes most of the time, but very occasionally fails - in this case in Python 3.7
It seems to only fail for the `article,k` compound primary key test.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1919/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1469015001,I_kwDOBm6k_c5Xj2PZ,1916,GET requests against POST endpoints should not 500 error,9599,simonw,closed,0,,,7867486,Datasette 1.0a1,1,2022-11-30T04:04:43Z,2022-11-30T05:15:19Z,2022-11-30T05:15:19Z,OWNER,,"![CF37BA4D-0677-4DDD-A339-EAF163BB63B7](https://user-images.githubusercontent.com/9599/204705025-6f88e9f7-757d-45e8-a89c-ab97e84781e8.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1916/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1469043836,I_kwDOBm6k_c5Xj9R8,1917,Don't allow writable API to edit the `_memory` database,9599,simonw,closed,0,,,7867486,Datasette 1.0a1,2,2022-11-30T04:51:59Z,2022-11-30T05:07:56Z,2022-11-30T05:07:55Z,OWNER,,"It shows up on https://latest.datasette.io/-/api (once you are signed in as root) - but there's no point in creating tables in it because they likely won't persist from one request to the next, as it's not a shared named database.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1917/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1468709531,I_kwDOBm6k_c5Xirqb,1915,Interactive demo of Datasette 1.0 write APIs,9599,simonw,closed,0,,,,,6,2022-11-29T21:16:03Z,2022-11-30T04:05:46Z,2022-11-30T04:05:46Z,OWNER,,"I'm going to try to get this working on https://latest.datasette.io/ - it already has a way for people to sign in as root, but none of the databases there are writable.
So I'm going to build a plugin which adds a writable named in-memory database.
And some kind of mechanism for clearing out that database on a regular basis - maybe tables in that database get deleted automatically an hour after they are created?
(Would be neat to display their time-left-until-deleted too)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1915/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1421529723,I_kwDOBm6k_c5UutJ7,1850,Write API in Datasette core,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,13,2022-10-24T22:13:24Z,2022-11-29T20:11:20Z,2022-11-29T20:11:20Z,OWNER,,"I need this for Datasette Cloud, and in thinking it through I realized that it's really time Datasette grew a default write API as well.
I'm going to mostly model this off `sqlite-utils`, since I've spent a bunch of time iterating on a pseudo-JSON API for that over the past few years (piping JSON to stdin etc).
I want this for Datasette 1.0. I'm going to be building it in the new [1.0-dev](https://github.com/simonw/datasette/tree/1.0-dev) branch, which is automatically deployed to https://latest-1-0-dev.datasette.io/ running on Cloud Run.
API features to build:
- [x] #1852
- [x] #1856
- [x] #1857
- [x] #1858
- [x] #1859
- [x] #1871
- [x] #1888
- [x] #1868
- [x] #1851
- [x] #1863
- [x] #1864
- [x] #1866
- [x] https://github.com/simonw/datasette/issues/1882
- [x] #1862
- [x] #1874
- [x] https://github.com/simonw/datasette/issues/1887
- [x] #1877
Bumped to later on:
- #1855
- #1878
- #1873
- #1875
- Make sure CORS works
- https://github.com/simonw/datasette/issues/1889
- Alter a table - `sqlite-utils transform` style (more powerful than straight ALTER)
- Execute SQL against a write connection
- Maybe even multiple write SQL statements bundled in a single transaction
- https://github.com/simonw/datasette/issues/1867
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1850/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1468603401,I_kwDOBm6k_c5XiRwJ,1913,Release Datasette 1.0a0,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,9,2022-11-29T19:41:42Z,2022-11-29T20:10:35Z,2022-11-29T20:10:35Z,OWNER,,"I attempted the release just now - https://github.com/simonw/datasette/releases/tag/1.0a0 - and got an unexpected test failure:
https://github.com/simonw/datasette/actions/runs/3577355358/attempts/1
```
> assert delete_response.status_code == 200
E assert 404 == 200
E + where 404 = .status_code
/home/runner/work/datasette/datasette/tests/test_api_write.py:396: AssertionError
=========================== short test summary info ============================
FAILED tests/test_api_write.py::test_delete_row[compound_pk_table-row_for_create2-pks2-article,k] - assert 404 == 200
+ where 404 = .status_code
```
I hit ""retry"" on that test but I expect it to fail again.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1913/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1432012302,I_kwDOBm6k_c5VWsYO,1877,Refactor and tidy up final write API code,9599,simonw,closed,0,,,,,1,2022-11-01T20:00:11Z,2022-11-29T19:44:16Z,2022-11-29T19:44:07Z,OWNER,,"- `views/table.py` has got a bit too big - I think the write classes should be pulled out into a separate module.
- [x] There's duplicate logic for deciding if the table and database exist and checking permissions",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1877/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1450312343,I_kwDOBm6k_c5WcgKX,1892,Merge 1.0-dev branch back to main,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,3,2022-11-15T20:04:25Z,2022-11-29T19:40:23Z,2022-11-29T19:40:23Z,OWNER,,"I'm committed enough to the 1.0 work now that I'm ready for the `main` branch to reflect that instead.
If I need to make any dot-releases against 0.63 I can do those from a branch.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1892/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1468592292,PR_kwDOBm6k_c5D6nzE,1912,Merge 1.0-dev (with initial write API) back into main,9599,simonw,closed,0,,,,,1,2022-11-29T19:31:21Z,2022-11-29T19:39:37Z,2022-11-29T19:39:36Z,OWNER,simonw/datasette/pulls/1912,"See:
- #1892
----
:books: Documentation preview :books:: https://datasette--1912.org.readthedocs.build/en/1912/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1912/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1450303205,I_kwDOBm6k_c5Wcd7l,1891,1.0a0 release notes,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,4,2022-11-15T19:58:20Z,2022-11-29T19:23:41Z,2022-11-29T19:23:41Z,OWNER,,"This release will mainly help preview the new Datasette write API:
- #1850",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1891/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1425029275,I_kwDOBm6k_c5U8Dib,1864,Delete a single record from an existing table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,4,2022-10-27T04:53:22Z,2022-11-29T18:54:04Z,2022-11-29T18:54:04Z,OWNER,,"API design:
```
POST /db/table/row-pks/-/delete
Or...
DELETE /db/table/row-pks/-/delete
```
I'm just going to do `POST` for the moment, like I did here:
- #1874
Permission: `delete-row`
Still needed:
- [ ] Tests for rowid tables
- [ ] Tests for compound primary keys",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1864/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1468519699,I_kwDOBm6k_c5Xh9UT,1911,`/db/-/create` should support creating tables with compound primary keys,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,2,2022-11-29T18:30:47Z,2022-11-29T18:50:58Z,2022-11-29T18:48:05Z,OWNER,,"Found myself needing this to write the tests for:
- #1864 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1911/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1425029242,I_kwDOBm6k_c5U8Dh6,1863,Update a single record in an existing table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,16,2022-10-27T04:53:17Z,2022-11-29T18:08:53Z,2022-11-29T18:06:37Z,OWNER,,"API design:
```
POST /db/table/row-pks/-/update
{
""field"": ""updated_value""
}
```
Only the fields that you pass will be updated.
Maybe this is the wrong design though? The design for insert currently looks like this:
- https://github.com/simonw/datasette/issues/1851#issuecomment-1294224185
```
POST /db/table/-/insert
Authorization: Bearer xxx
Content-Type: application/json
{
""row"": {
""id"": 1,
""name"": ""New name""
}
}
```
I could use the same format for `/-/update`, but in this case the API doesn't require you to pass every field so `""row""` doesn't seem like the right key.
I think I'll go with this:
```
POST /db/table/1/-/update
Authorization: Bearer xxx
Content-Type: application/json
{
""update"": {
""name"": ""New name""
}
}
```
The benefit of having an `""update""` key is that it allows me to use other keys in the future. Maybe a `""alter"": true` key to indicate that new columns should be added if they are missing.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1863/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1466952626,I_kwDOBm6k_c5Xb-uy,1909,Option to sort facets alphabetically,9599,simonw,open,0,,,,,1,2022-11-28T19:18:14Z,2022-11-28T19:19:26Z,,OWNER,,"Suggested here:
- https://github.com/simonw/datasette/discussions/1908",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1909/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1439009231,I_kwDOBm6k_c5VxYnP,1884,Exclude virtual tables from datasette inspect,25778,eyeseast,open,0,,,,,6,2022-11-07T21:26:01Z,2022-11-21T04:40:56Z,,CONTRIBUTOR,,"Ran `inspect` on a spatialite database and got these warnings:
```
ERROR: conn=, sql = 'select count(*) from [SpatialIndex]', params = None: no such module: VirtualSpatialIndex
ERROR: conn=, sql = 'select count(*) from [ElementaryGeometries]', params = None: no such module: VirtualElementary
ERROR: conn=, sql = 'select count(*) from [KNN]', params = None: no such module: VirtualKNN
```
It still worked, but probably want to catch this.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1884/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1455928469,I_kwDOBm6k_c5Wx7SV,1903,Refactor all error classes into a datasette.exceptions module,9599,simonw,open,0,,,3268330,Datasette 1.0,2,2022-11-18T22:44:45Z,2022-11-20T22:35:01Z,,OWNER,,"While working on this issue:
- #1896
I realized that Datasette has error classes scattered around a fair bit, including some in the `datasette.utils.asgi` module for some reason.
I should clean these up.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1903/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1456012874,I_kwDOBm6k_c5WyP5K,1905,`publish heroku` failing due to old Python version,9599,simonw,closed,0,,,,,4,2022-11-19T00:01:45Z,2022-11-19T01:12:05Z,2022-11-19T00:52:29Z,OWNER,,"Reported on Discord: https://discord.com/channels/823971286308356157/823971286941302908/1042814317118115901
```
-----> Building on the Heroku-22 stack
-----> Determining which buildpack to use for this app
-----> Python app detected
-----> Using Python version specified in runtime.txt
! Requested runtime 'python-3.8.10' is not available for this stack (heroku-22).
! For supported versions, see: https://devcenter.heroku.com/articles/python-support
! Push rejected, failed to compile Python app.
! Push failed
▸ Build failed
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1905/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1456013930,I_kwDOBm6k_c5WyQJq,1906,Extract publish Heroku support to a plugin,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-11-19T00:02:51Z,2022-11-19T00:03:10Z,,OWNER,,"> This is a strong argument for extracting the Heroku support out to a plugin - it would allow this to be fixed with a plugin release without needing to push a full release of Datasette itself.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1905#issuecomment-1320678715_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1906/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1455932972,I_kwDOBm6k_c5Wx8Ys,1904,Datasette Lite tests failing due to httpx upgrade,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,0,2022-11-18T22:49:31Z,2022-11-18T22:57:48Z,2022-11-18T22:52:22Z,OWNER,,"Same problem as this one:
- https://github.com/simonw/datasette-lite/issues/56
Caused this failure: https://github.com/simonw/datasette/actions/runs/3500765964",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1904/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1452364777,I_kwDOBm6k_c5WkVPp,1896,Extract logic for resolving a URL to a database / table / row,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,4,2022-11-16T22:25:20Z,2022-11-18T22:57:47Z,2022-11-18T22:56:55Z,OWNER,,"> In trying to write this I realize that there's a lot of duplicated code with delete row, specifically around resolving the incoming URL into a row (or a database or a table).
>
> Since this is so common, I think it's worth extracting the logic out first.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1863#issuecomment-1317755263_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1896/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1452572348,I_kwDOBm6k_c5WlH68,1900,datasette package --spatialite throws error during build,419145,rdmurphy,open,0,,,,,11,2022-11-17T02:03:28Z,2022-11-18T08:00:38Z,,NONE,,"Hello! Attempting to use `datasette package` to bundle up a SpatiaLite DB and I'm getting this error during the `docker build`:
```
sqlite3.OperationalError: /usr/lib/x86_64-linux-gnu/mod_spatialite.so.so: cannot open shared object file: No such file or directory
```
Seems to be throwing when this step is ran:
```
ERROR [6/6] RUN datasette inspect results.db --inspect-file inspect-data.json
```
This is with `v0.63.1`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1900/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1450796965,I_kwDOBm6k_c5WeWel,1894,Initialize CodeMirror during DOMContentLoaded instead of onload,95570,bgrins,closed,0,,,,,0,2022-11-16T03:52:19Z,2022-11-18T07:29:02Z,2022-11-18T07:29:02Z,CONTRIBUTOR,,As per https://github.com/simonw/datasette/pull/1893/files#r1023248927 this should prevent a flash between the textarea being replaced by CodeMirror.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1894/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1452485922,PR_kwDOBm6k_c5DEh-E,1898,Use DOMContentLoaded instead of load event for CodeMirror initialization,95570,bgrins,closed,0,,,,,2,2022-11-17T00:19:21Z,2022-11-18T07:29:01Z,2022-11-18T07:29:01Z,CONTRIBUTOR,simonw/datasette/pulls/1898," Closes #1894
----
:books: Documentation preview :books:: https://datasette--1898.org.readthedocs.build/en/1898/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1898/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1452495049,I_kwDOBm6k_c5Wk1DJ,1899,Clicking within the CodeMirror area below the SQL (i.e. when there's only a single line) doesn't cause the editor to get focused ,95570,bgrins,closed,0,,,,,4,2022-11-17T00:29:52Z,2022-11-18T07:28:28Z,2022-11-18T07:20:53Z,CONTRIBUTOR,,"After the upgrade to 6 (#1893) I noticed this. I think it's because we're doing overflow:hidden to accomplish the CSS resizer.
When there's a single line of SQL there's a gap below that line where clicking doesn't do anything. It should focus at the end of the line.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1899/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1453813400,I_kwDOBm6k_c5Wp26Y,1901,"Some plugins show ""home"" breadcrumbs twice in the top left",95570,bgrins,closed,0,,,,,8,2022-11-17T18:44:58Z,2022-11-18T07:22:37Z,2022-11-18T07:02:56Z,CONTRIBUTOR,,"
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1901/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1454532488,I_kwDOBm6k_c5WsmeI,1902,Document {% block crumbs %} for plugin authors,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-11-18T06:16:30Z,2022-11-18T06:16:39Z,,OWNER,,"> For `datasette-copyable` I want to show breadcrumbs that take database/instance permissions into account, so I'm removing `{% block nav %}` entirely and replacing it with this:
>
> ```html+jinja
> {% block crumbs %}
> {{ crumbs.nav(request=request, database=database, table=table) }}
> {% endblock %}
> ```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1901#issuecomment-1319588163_
I should document this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1902/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1452457263,I_kwDOBm6k_c5Wkr0v,1897,Serve schema JSON to the SQL editor to enable autocomplete,9599,simonw,closed,0,,,,,9,2022-11-16T23:47:45Z,2022-11-18T05:33:20Z,2022-11-18T02:54:43Z,OWNER,,"See:
- https://github.com/simonw/datasette/issues/1893#issuecomment-1317831555
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1897/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1448143294,I_kwDOBm6k_c5WUOm-,1890,Autocomplete text entry for filter values that correspond to facets,536941,fgregg,closed,0,,,,,16,2022-11-14T14:11:31Z,2022-11-17T00:47:36Z,2022-11-16T03:23:01Z,CONTRIBUTOR,,"datasette allows users to enter in the value for named parameters into a free-text form field.
I think it would add a lot of usability, if the form field could be a drop down of options when query value is already a faceted column.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1890/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1450363982,PR_kwDOBm6k_c5C9ZuP,1893,"Upgrade to CodeMirror 6, add SQL autocomplete",95570,bgrins,closed,0,,,,,48,2022-11-15T20:52:35Z,2022-11-16T23:54:02Z,2022-11-16T23:49:06Z,CONTRIBUTOR,simonw/datasette/pulls/1893,"In an effort to get closer to table / column autocomplete I took a shot at https://github.com/simonw/datasette/issues/1796. I haven't done a lot of testing but would be curious if this fixes some of the concerns raised in https://github.com/simonw/datasette/issues/1796#issue-1355148385 for example.
Done:
* Changed to bundling using rollup as per https://codemirror.net/examples/bundle/
* Restored a fromTextArea-like function from https://codemirror.net/docs/migration/
* Removed old JS and CSS files (no external CSS needed anymore as per https://codemirror.net/examples/styling/)
* Updated instructions for building the bundle
Not done:
* cmResize had an error, so commented out the resize handle
* Add extraKeys option for shift+enter and tab
----
:books: Documentation preview :books:: https://datasette--1893.org.readthedocs.build/en/1893/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1893/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1452360613,I_kwDOBm6k_c5WkUOl,1895,Avoid using host name when building absolute URLs?,14294,hubgit,open,0,,,,,0,2022-11-16T22:21:27Z,2022-11-16T22:21:27Z,,NONE,,"When deploying Datasette to Cloud Run and rewriting certain routes from a Firebase app to the Cloud Run service, some of the URLs in the page start with `https://[service].run.app` rather than the (custom) domain of the Firebase app.
I guess this is because a) the custom domain of the Firebase app isn't being passed through in the `host` header of the request to the Cloud Run instance and b) the `absolute_url` function in Datasette is using information from the request to build the URL.
Would it be possible to not use the host name when building the absolute URLs, i.e. only include the path in the URL?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1895/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1433576351,I_kwDOBm6k_c5VcqOf,1880,Datasette with many and large databases > Memory use,525934,amitkoth,open,0,,,,,4,2022-11-02T18:10:27Z,2022-11-16T17:50:29Z,,NONE,,"> Datasette maintains an in-memory SQLite database with details of the the databases, tables and columns for all of the attached databases.
The above is from the docs ^. There's two problems here - the number of datasette ""instances"" in a single server/VM and the size of the database itself. We want the **opposite** of in-memory, including what happens on SQLlite - documented in https://www.sqlite.org/inmemorydb.html
From the context in https://github.com/simonw/datasette/issues/1150 - does it mean datasette is memory-bound to the size of the dataset - which might be a deal-breaker for many large-scale use cases?
In an extreme case - let's say a single server had 100 SQLlite databases, which would enable 100 ""instances"" of datasette to run, one per client (e.g. in a SaaS multi-tenant environment). How could we achieve all these goals:
1. Allow any _one_ of these 100 databases to grow to say 2Tb in size
2. Have one datasette instance, which connects to 1 of the 100 instances, based on incoming credentials/tenant ID
3. Minimize memory use entirely - both by datasette and SQLlite, such that almost all operations are executed in real-time on-disk with little to no memory consumption per-tenant, or per-database.
Any ideas appreciated - we're looking to use this in a SaaS type of setting - many instances, single server.
@simonw great work on datasette, in general! Possibly related to https://github.com/simonw/datasette/issues/1480 but we don't want use any kind of serverless infra - this is a long-running VM/server.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1880/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1447388809,I_kwDOBm6k_c5WRWaJ,1887,Add a confirm step to the drop table API,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,2,2022-11-14T04:59:53Z,2022-11-15T19:59:59Z,2022-11-14T05:18:51Z,OWNER,,"> In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057_
Added drop table API in:
- #1874",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1887/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1429030341,I_kwDOBm6k_c5VLUXF,1874,API to drop a table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,4,2022-10-30T21:55:11Z,2022-11-15T19:59:53Z,2022-11-14T05:45:06Z,OWNER,,"`POST /db/table/-/drop`
Require `drop-table` permission.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1874/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1425011030,I_kwDOBm6k_c5U7_FW,1862,"Create a new table from one or more records, `sqlite-utils` style",9599,simonw,closed,0,,,8658075,Datasette 1.0a0,5,2022-10-27T04:25:02Z,2022-11-15T19:59:47Z,2022-11-15T06:42:09Z,OWNER,,"It's interesting to also think about what the form-based UI for this could look like - since that would involve users creating new columns of different types on the fly.
Will need the `create-table` permission.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1862/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1435294468,I_kwDOBm6k_c5VjNsE,1882,`/db/-/create` API for creating tables,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,12,2022-11-03T21:44:32Z,2022-11-15T19:59:43Z,2022-11-15T06:00:41Z,OWNER,,"> It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document):
>
> https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1882/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1426001541,I_kwDOBm6k_c5U_w6F,1866,API for bulk inserting records into a table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,12,2022-10-27T17:19:25Z,2022-11-15T19:59:34Z,2022-10-30T06:04:07Z,OWNER,,"Similar to https://github.com/simonw/datasette-insert/blob/0.8/README.md#inserting-data-and-creating-tables
I expect this to become by far the most common way that data gets into a Datasette instance - more so than the individual row API in:
- #1851 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1866/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1421544654,I_kwDOBm6k_c5UuwzO,1851,API to insert a single record into an existing table,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,22,2022-10-24T22:24:21Z,2022-11-15T19:59:18Z,2022-10-28T00:59:25Z,OWNER,,Controlled by a new `insert-row` permission.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1851/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1426195437,I_kwDOBm6k_c5VAgPt,1868,Design URLs for the write API,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,5,2022-10-27T19:55:30Z,2022-11-15T19:59:14Z,2022-10-27T20:07:01Z,OWNER,,"My original design for this issue:
- #1851
Was `POST /db/table` with JSON of `{""insert"": {...}}`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1868/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1447439985,I_kwDOBm6k_c5WRi5x,1888,API explorer should take immutability into account,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,1,2022-11-14T06:00:14Z,2022-11-15T19:59:10Z,2022-11-14T06:04:48Z,OWNER,,"Refs:
- #1871
I noticed the API explorer doesn't show any links on https://latest-1-0-dev.datasette.io/-/api because the `fixtures` database is immutable.
It should still show read examples there.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1888/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1427293909,I_kwDOBm6k_c5VEsbV,1871,API explorer tool,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,24,2022-10-28T13:49:11Z,2022-11-15T19:59:05Z,2022-11-14T04:59:59Z,OWNER,,The API will be much easier to develop if there's a page that helps you execute JSON POSTs against it.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1871/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1423369494,I_kwDOBm6k_c5U1uUW,1859,datasette create-token CLI command,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,3,2022-10-26T03:12:59Z,2022-11-15T19:59:00Z,2022-10-26T04:31:39Z,OWNER,,The CLI equivalent of the `/-/create-token` page.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1859/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1423364990,I_kwDOBm6k_c5U1tN-,1858,`max_signed_tokens_ttl` setting for a maximum duration on API tokens,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,4,2022-10-26T03:05:53Z,2022-11-15T19:58:52Z,2022-10-27T03:15:05Z,OWNER,,"It's currently possible to use `/-/create-token` to create a token that lasts forever.
Some administrators may wish to have a maximum expiry instead. I should support that with a setting.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1858/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1423347412,I_kwDOBm6k_c5U1o7U,1857,Prevent API tokens from using /-/create-token to create more tokens,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,1,2022-10-26T02:38:09Z,2022-11-15T19:57:11Z,2022-10-26T02:57:26Z,OWNER,,"> It strikes me that users should NOT be able to use a token to create additional tokens.
>
> The current design actually does allow that, since the `dstok_` Bearer token can be used to authenticate calls to the `/-/create-token` page.
>
> So I think I need a mechanism whereby that page can only allow access to users authenticated by cookie.
>
> Not obvious how to do that though, since Datasette's authentication actor system is designed to abstract that detail away!
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1291417100_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1857/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1423336122,I_kwDOBm6k_c5U1mK6,1856,allow_signed_tokens setting for disabling API signed token mechanism,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,3,2022-10-26T02:20:55Z,2022-11-15T19:57:05Z,2022-10-26T02:58:35Z,OWNER,,"Had some design thoughts here: https://github.com/simonw/datasette/issues/1852#issuecomment-1291272280
I liked this option the most:
--setting allow_create_tokens off",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1856/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1421552095,I_kwDOBm6k_c5Uuynf,1852,Default API token authentication mechanism,9599,simonw,closed,0,,,8658075,Datasette 1.0a0,30,2022-10-24T22:31:07Z,2022-11-15T19:57:00Z,2022-10-26T02:19:54Z,OWNER,,"API authentication will be via `Authorization: Bearer XXX` request headers.
I'm inclined to add a default token mechanism to Datasette based on tokens that are signed with the `DATASETTE_SECRET`. Maybe the root user can access `/-/create-token` which provides a UI for generating a time-limited signed token? Could also have a `datasette token` command for creating such tokens at the command-line.
Plugins can then define alternative ways of creating tokens, such as the existing https://datasette.io/plugins/datasette-auth-tokens plugin.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1289706439_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1852/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1446657889,I_kwDOBm6k_c5WOj9h,1885,Integrate inside GUI app (tkinter),5115787,dmalves,open,0,,,,,0,2022-11-13T00:10:43Z,2022-11-13T00:11:09Z,,NONE,,"Hi, I'd like to integrate datasette inside a tkinter app. The app should be able to start/stop datasette server. How could I integrate datasette inside my app, so it can start and stop datasette server?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1885/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1435917503,I_kwDOBm6k_c5Vlly_,1883,Errors when using table filters behind a proxy,31312775,mattmalcher,closed,0,,,,,13,2022-11-04T11:18:47Z,2022-11-11T09:20:22Z,2022-11-11T06:54:58Z,NONE,,"Using datasette==0.63 table filters do not respect the `base_url` setting as described [here](https://docs.datasette.io/en/stable/deploying.html#running-datasette-behind-a-proxy)
To reproduce, go to:
https://datasette-apache-proxy-demo.datasette.io/prefix/fixtures/binary_data
Then use the table filter buttons.
The `/prefix/` is dropped, resulting in URL not found:
https://datasette-apache-proxy-demo.datasette.io/fixtures/binary_data?_sort=rowid&rowid__exact=1
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1883/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 1}",,completed
802513359,MDU6SXNzdWU4MDI1MTMzNTk=,1217,Possible to deploy as a python app (for Rstudio connect server)?,6165713,plpxsk,open,0,,,,,4,2021-02-05T22:21:24Z,2022-11-04T11:37:52Z,,NONE,,"Is it possible to deploy a `datasette` application as a python web app?
In my enterprise, I have option to deploy python apps via [Rstudio Connect](https://github.com/rstudio/rsconnect-python), and I would like to publish a `datasette` dashboard for sharing.
I welcome any pointers to converting `datasette serve` into a python app that can be run as something like `python datasette.py --my_data.db`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1217/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1432037325,I_kwDOBm6k_c5VWyfN,1879,Make it easier to fix URL proxy problems,9599,simonw,open,0,,,,,5,2022-11-01T20:19:23Z,2022-11-01T20:33:52Z,,OWNER,,"This came up on Discord again today: figuring out how to run Datasette behind a proxy that might hide the incoming Host: header (and strip HTTPS) is really hard!
https://discord.com/channels/823971286308356157/823971286941302908/1037012475322847263",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1879/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1431786951,I_kwDOBm6k_c5VV1XH,1876,SQL query should wrap on SQL interrupted screen,9599,simonw,closed,0,,,,,2,2022-11-01T17:14:01Z,2022-11-01T17:22:33Z,2022-11-01T17:22:33Z,OWNER,,"Just saw this:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1876/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1428560020,I_kwDOBm6k_c5VJhiU,1872,"SITE-BUSTING ERROR: ""render_template() called before await ds.invoke_startup()""",192568,mroswell,closed,0,,,,,3,2022-10-30T02:28:39Z,2022-10-30T06:26:01Z,2022-10-30T06:26:01Z,CONTRIBUTOR,,"1. My https://list.saferdisinfectants.org/disinfectants/listN page (linked from https://SaferDisinfectants.org ) has been running beautifully for a year and a half, including a GitHub Actions workflow that's been routinely updating the database.
2. I received a recent report that the list page is down. I don't know when it went down, but the content is replaced with: ""render_template() called before await ds.invoke_startup()""
3. The local datasette repo runs without incident.
4. The site is hosted on vercel, linked to my github repo. Perhaps some vercel changes were made, but not by anyone on our side. Here is a screenshot of the current project settings:
Here a screenshot of the latest deployment status:
This is my repository:
https://github.com/mroswell/list-N
(I notice: datasette==0.59 in my requirements.txt file)
Because it's been long while since I actively worked on this or any other datasette project, I forget a lot of what I knew at one point. Perhaps some configuration file could be missing? Or perhaps I just need to know the right incantation to add to that vercel settings page.
Help is welcome as the nonprofit org is soon hosting its annual conference, and we'd love to have the page working again.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1872/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1426253476,I_kwDOBm6k_c5VAuak,1869,Release 0.63,9599,simonw,closed,0,,,,,3,2022-10-27T20:53:01Z,2022-10-27T22:24:38Z,2022-10-27T22:11:33Z,OWNER,,"Most of the release notes are already written:
- https://github.com/simonw/datasette/releases/tag/0.63a0
- https://github.com/simonw/datasette/releases/tag/0.63a1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1869/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1342430983,I_kwDOBm6k_c5QA98H,1786,Adjust height of textarea for no JS case,9599,simonw,closed,0,,,,,4,2022-08-18T01:15:15Z,2022-10-27T21:50:12Z,2022-08-18T16:06:09Z,OWNER,,"Datasette Lite: https://lite.datasette.io/?sql=https://gist.githubusercontent.com/simonw/1f8a91123ccefd8844187225b1832d7a/raw/5069075b86aa79358fbab3d4482d1d269077d632/recipes.sql#/data?sql=select+id%2C+name%2C+ingredients%2C+%28%0A++select+json_group_array%28value%29+from+json_each%28ingredients%29%0A++where+value+in+%28select+value+from+json_each%28%3Ap0%29%29%0A%29+as+matching_ingredients%0Afrom+recipes%0Awhere+json_array_length%28matching_ingredients%29+%3E+0%0Aorder+by+json_array_length%28matching_ingredients%29+desc&p0=%5B%22sugar%22%2C+%22cheese%22%5D
![46F8101E-8CE3-4F61-B200-F865E6B5DBCC](https://user-images.githubusercontent.com/9599/185270723-f55513b0-b561-434d-9d7c-4fe5be9756e0.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1786/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1401155623,PR_kwDOBm6k_c5AZLzm,1839,Bump black from 22.8.0 to 22.10.0,49699333,dependabot[bot],closed,0,,,,,1,2022-10-07T13:13:41Z,2022-10-27T20:51:46Z,2022-10-27T20:51:45Z,CONTRIBUTOR,simonw/datasette/pulls/1839,"Bumps [black](https://github.com/psf/black) from 22.8.0 to 22.10.0.
Release notes
Sourced from black's releases .
22.10.0
Highlights
Runtime support for Python 3.6 has been removed. Formatting 3.6 code will still be
supported until further notice.
Stable style
Fix a crash when # fmt: on
is used on a different block level than # fmt: off
(#3281 )
Preview style
Fix a crash when formatting some dicts with parenthesis-wrapped long string keys
(#3262 )
Configuration
.ipynb_checkpoints
directories are now excluded by default (#3293 )
Add --skip-source-first-line
/ -x
option to ignore the first line of source code
while formatting (#3299 )
Packaging
Executables made with PyInstaller will no longer crash when formatting several files
at once on macOS. Native x86-64 executables for macOS are available once again.
(#3275 )
Hatchling is now used as the build backend. This will not have any effect for users
who install Black with its wheels from PyPI. (#3233 )
Faster compiled wheels are now available for CPython 3.11 (#3276 )
Blackd
Windows style (CRLF) newlines will be preserved (#3257 ).
Integrations
Vim plugin: add flag (g:black_preview
) to enable/disable the preview style (#3246 )
Update GitHub Action to support formatting of Jupyter Notebook files via a jupyter
option (#3282 )
Update GitHub Action to support use of version specifiers (e.g. <23
) for Black
version (#3265 )
Changelog
Sourced from black's changelog .
22.10.0
Highlights
Runtime support for Python 3.6 has been removed. Formatting 3.6 code will still be
supported until further notice.
Stable style
Fix a crash when # fmt: on
is used on a different block level than # fmt: off
(#3281 )
Preview style
Fix a crash when formatting some dicts with parenthesis-wrapped long string keys
(#3262 )
Configuration
.ipynb_checkpoints
directories are now excluded by default (#3293 )
Add --skip-source-first-line
/ -x
option to ignore the first line of source code
while formatting (#3299 )
Packaging
Executables made with PyInstaller will no longer crash when formatting several files
at once on macOS. Native x86-64 executables for macOS are available once again.
(#3275 )
Hatchling is now used as the build backend. This will not have any effect for users
who install Black with its wheels from PyPI. (#3233 )
Faster compiled wheels are now available for CPython 3.11 (#3276 )
Blackd
Windows style (CRLF) newlines will be preserved (#3257 ).
Integrations
Vim plugin: add flag (g:black_preview
) to enable/disable the preview style (#3246 )
Update GitHub Action to support formatting of Jupyter Notebook files via a jupyter
option (#3282 )
Update GitHub Action to support use of version specifiers (e.g. <23
) for Black
version (#3265 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.8.0&new-version=22.10.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1839.org.readthedocs.build/en/1839/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1839/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1400121355,PR_kwDOBm6k_c5AVujU,1835,use inspect data for hash and file size,536941,fgregg,closed,0,,,,,3,2022-10-06T18:25:24Z,2022-10-27T20:51:30Z,2022-10-06T20:06:07Z,CONTRIBUTOR,simonw/datasette/pulls/1835,"`inspect_data` should already include the hash and the db file size, so this PR takes advantage of using those instead of always recalculating. should help a lot on startup with large DBs.
closes #1834 ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1835/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1400431789,PR_kwDOBm6k_c5AWyQK,1837,Make hash and size a lazy property,536941,fgregg,closed,0,,,,,2,2022-10-06T23:51:22Z,2022-10-27T20:51:21Z,2022-10-27T20:51:20Z,CONTRIBUTOR,simonw/datasette/pulls/1837,"Many apologies, @simonw. My previous PR #1835 did not really solve the problem because the name of the database is often not known to database object in the init method.
I took a cue from how you dealt with this issue and made hash a lazy property and did something similar with size.
----
:books: Documentation preview :books:: https://datasette--1837.org.readthedocs.build/en/1837/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1837/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1424378012,I_kwDOBm6k_c5U5kic,1860,SQL query field can't begin by a comment,562352,CharlesNepote,closed,0,,,,,12,2022-10-26T16:55:31Z,2022-10-27T18:57:37Z,2022-10-27T04:21:40Z,NONE,,"![image](https://user-images.githubusercontent.com/562352/198085197-f26fcd61-4dac-4ca4-a346-e70f88a30ecc.png)
SQL comments are **very** useful to explain the meaning of the query. It's currently impossible to put it at the beginning of the field as seen on the screen capture: it leads to an error: `Statement must be a SELECT`.
It would be great to make it possible because:
* as the request is the title of the page:
* it eases the search with search engines
* it eases the search in the browsers' url field
* it acts as a kind of title: the global meaning of the query is immediately understandable
* some tools, such as Slack, are shortening long URLs and displaying the beginning of the URLs (eg. `https://example.org/products?sql=select+%28length%28data_quality_errors_ta[...]+%21%3D+%22%22+group+by+NB_of_issues+order+by+NB_of_issues+desc+limit+200`)
Beginning a query with a comment is possible with SQLite.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1860/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1425682079,I_kwDOBm6k_c5U-i6f,1865,Stop syncing main to master,9599,simonw,closed,0,,,,,1,2022-10-27T13:55:38Z,2022-10-27T13:58:27Z,2022-10-27T13:56:13Z,OWNER,,"I think it's been long enough now that I can drop the code that syncs the main branch to master.
I originally added this for people who might be using `datasette publish ... --branch master` - which might only have been me anyway!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1865/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
639072811,MDU6SXNzdWU2MzkwNzI4MTE=,849,Rename master branch to main,9599,simonw,closed,0,,,3268330,Datasette 1.0,10,2020-06-15T19:05:54Z,2022-10-27T13:57:08Z,2020-09-15T20:37:14Z,OWNER,,"I was waiting for consensus to form around this (and kind-of hoping for `trunk` since I like the tree metaphor) and it looks like `main` is it.
I've seen convincing arguments against `trunk` too - it indicates that the branch has some special significance like in Subversion (where all branches come from trunk) when it doesn't. So `main` is better anyway.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/849/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1424980545,I_kwDOBm6k_c5U73pB,1861,"request.headers.get(""Content-Type"") fails",9599,simonw,open,0,,,,,0,2022-10-27T03:39:12Z,2022-10-27T03:39:12Z,,OWNER,,"Turns out this is case-sensitive, needs to be:
request.headers.get(""content-type"") != ""application/json""
That's not great usability. It should be case insensitive.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1861/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1420174670,I_kwDOBm6k_c5UpiVO,1849,NoneType' object has no attribute 'actor',9599,simonw,closed,0,,,,,5,2022-10-24T04:02:15Z,2022-10-26T21:13:40Z,2022-10-26T21:13:40Z,OWNER,,"```
File ""/usr/local/lib/python3.10/site-packages/datasette/templates/_crumbs.html"", line 3, in template
{% set items=crumb_items(request=request, database=database, table=table) %}
File ""jinja2/async_utils.py"", line 65, in auto_await
return await t.cast(""t.Awaitable[V]"", value)
File ""datasette/app.py"", line 638, in _crumb_items
actor=request.actor, action=""view-instance"", default=True
```
From Sentry.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1849/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
642297505,MDU6SXNzdWU2NDIyOTc1MDU=,857,Comprehensive documentation for variables made available to templates,9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2020-06-20T03:19:43Z,2022-10-26T02:58:17Z,2022-10-26T02:58:17Z,OWNER,,"Needed for the Datasette 1.0 release, so template authors can trust that Datasette is unlikely to break their templates.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/857/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1422973111,I_kwDOBm6k_c5U0Ni3,1854,Flaky test: test_serve_localhost_http,9599,simonw,closed,0,,,,,3,2022-10-25T19:37:35Z,2022-10-25T19:53:02Z,2022-10-25T19:53:02Z,OWNER,,Failing on Python 3.10 at the moment: https://github.com/simonw/datasette/actions/runs/3323629947/jobs/5494340302,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1854/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1422915587,I_kwDOBm6k_c5Uz_gD,1853,Upgrade Datasette Docker to Python 3.11,9599,simonw,closed,0,,,,,7,2022-10-25T18:44:31Z,2022-10-25T19:28:56Z,2022-10-25T19:05:16Z,OWNER,,"Related:
- #1768
I think this base image looks right: [3.11.0-slim-bullseye](https://hub.docker.com/layers/library/python/3.11.0-slim-bullseye/images/sha256-244c0b0e6e7608a16f87382fc8a5ef3c330d042113a9a7b6fc15a95360181651?context=explore)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1853/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1420090659,I_kwDOBm6k_c5UpN0j,1848,Private database page should show padlock on every table,9599,simonw,closed,0,,,,,3,2022-10-24T02:28:38Z,2022-10-24T02:50:29Z,2022-10-24T02:42:34Z,OWNER,,"Following:
- #1829
https://latest.datasette.io/_internal looks like this:
But those queries and tables are private too, and should also show the padlock icon.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1848/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1396948693,I_kwDOBm6k_c5TQ77V,1829,Table/database that is private due to inherited permissions does not show padlock,9599,simonw,closed,0,,,,,8,2022-10-04T23:14:16Z,2022-10-24T02:23:46Z,2022-10-24T02:11:37Z,OWNER,,"I noticed that a table page that is private because the database or instance is private, e.g. this one:
Is not displaying the padlock icon that indicates the table is not visible to the public.
Same issue for the database page too, which in this case is private due to `view-instance`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1829/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1408561039,PR_kwDOBm6k_c5Axrpb,1842,check_visibility can now take multiple permissions into account,9599,simonw,closed,0,,,,,3,2022-10-14T00:06:04Z,2022-10-24T02:11:36Z,2022-10-24T02:11:36Z,OWNER,simonw/datasette/pulls/1842,"Refs #1829
- [x] Fix table page
- [x] Fix database page
- [x] Fix query page
- [x] Fix row page
- [x] Tests
- [x] Updated documentation for `check_visibility` method, to cover the new `permissions=` keyword argument
Also this fix is currently only applied on the table page - needs to be applied on database, row and query pages too.
----
:books: Documentation preview :books:: https://datasette--1842.org.readthedocs.build/en/1842/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1842/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1420055377,I_kwDOBm6k_c5UpFNR,1847,Both _local_metadata and _metadata_local?,9599,simonw,closed,0,,,,,2,2022-10-24T01:43:08Z,2022-10-24T01:53:13Z,2022-10-24T01:53:13Z,OWNER,,"Spotted this in the debugger against the `datasette` object while running tests (`pytest -k test_permissions_cascade` to be exact):
```
(Pdb) [p for p in dir(self) if p.startswith('_') and '__' not in p]
['_actor', '_asset_urls', '_connected_databases', '_crumb_items', '_local_metadata', '_metadata', '_metadata_local', '_metadata_recursive_update', '_permission_checks', '_plugins', '_prepare_connection', '_refresh_schemas', '_refresh_schemas_lock', '_register_custom_units', '_register_renderers', '_root_token', '_routes', '_secret', '_settings', '_show_messages', '_startup_hook_calculation', '_startup_hook_fired', '_startup_invoked', '_threads', '_versions', '_write_messages_to_response']
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1847/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1410305897,I_kwDOBm6k_c5UD49p,1845,Reconsider the Datasette first-run experience,9599,simonw,open,0,,,,,3,2022-10-15T22:21:31Z,2022-10-16T08:54:53Z,,OWNER,,"Had a really interesting conversation today about how hard it is to get from ""I installed Datasette"" to ""I've done something useful with it"": https://news.ycombinator.com/item?id=33216789#33218590
Spending some time focusing on that first-run experience feels very worthwhile.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1845/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1409679008,I_kwDOBm6k_c5UBf6g,1844,Update screenshots in documentation to match latest designs,9599,simonw,closed,0,,,,,18,2022-10-14T18:01:18Z,2022-10-14T23:51:46Z,2022-10-14T19:57:17Z,OWNER,,"https://docs.datasette.io/en/0.62/full_text_search.html has this out-of-date screenshot:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1844/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1397084281,I_kwDOBm6k_c5TRdB5,1831,If user can see table but NOT database/instance nav links should not display,9599,simonw,closed,0,,,,,10,2022-10-05T02:16:31Z,2022-10-13T21:52:04Z,2022-10-13T21:52:04Z,OWNER,,"Spotted this bug while building this plugin:
- https://github.com/simonw/datasette-public
This is a public table, but the two links in the nav go to forbidden pages:
Those nav links shouldn't be shown at all.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1831/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1406860394,I_kwDOBm6k_c5T2vxq,1841,Drop format_bytes for Jinja filesizeformat filter,9599,simonw,open,0,,,,,0,2022-10-12T22:06:34Z,2022-10-12T22:06:34Z,,OWNER,,"Turns out this isn't necessary:
https://github.com/simonw/datasette/blob/5aa359b86907d11b3ee601510775a85a90224da8/datasette/utils/__init__.py#L849-L858
I can use this instead: https://jinja.palletsprojects.com/en/3.1.x/templates/#jinja-filters.filesizeformat",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1841/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1402608214,PR_kwDOBm6k_c5AdyZ4,1840,test commit,102635518,7lingyuan,closed,0,,,,,0,2022-10-10T05:15:26Z,2022-10-10T09:11:50Z,2022-10-10T09:11:50Z,FIRST_TIMER,simonw/datasette/pulls/1840,"lalalalalalala
----
:books: Documentation preview :books:: https://datasette--1840.org.readthedocs.build/en/1840/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1840/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
912864936,MDU6SXNzdWU5MTI4NjQ5MzY=,1362,Consider using CSP to protect against future XSS,9599,simonw,open,0,,,,,17,2021-06-06T15:32:20Z,2022-10-08T18:42:09Z,,OWNER,,The XSS in #1360 would have been a lot less damaging if Datasette used CSP to protect against such vulnerabilities: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1362/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1400374908,I_kwDOBm6k_c5TeAZ8,1836,docker image is duplicating db files somehow,536941,fgregg,open,0,,,,,13,2022-10-06T22:35:54Z,2022-10-08T16:56:51Z,,CONTRIBUTOR,,"if you look into the docker image created by docker publish, the `datasette inspect` line is duplicating the db files.
here's the result of the inspect command:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1836/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1157182254,I_kwDOBm6k_c5E-TMu,1646,Configuration directory mode does not pick up other file extensions than .db,15640196,dnsos,closed,0,,,,,3,2022-03-02T13:15:23Z,2022-10-07T23:06:17Z,2022-10-07T23:03:35Z,NONE,,"Hello, I've been trying to run Datasette with the [configuration directory mode](https://docs.datasette.io/en/stable/settings.html#configuration-directory-mode) with a structure such as this one:
```plain
some-directory/
example.sqlite3
another-example.db
one-more.custom
[...]
```
(In my scenario I can't just change the filename extension without other problems arising)
Now databases with the `.sqlite3` or the custom filename extension are ignored by Datasette in this case. I'm aware that the docs state that a `.db` extension is required, but I was wondering if there is a reason for restricting this or any workaround available? When I run `datasette example.sqlite3` or `datasette one-more.custom` the databases are served by Datasette without a problem.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1646/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1400494162,PR_kwDOBm6k_c5AW_kl,1838,Open Datasette link in new tab,4399499,ocdtrekkie,closed,0,,,,,3,2022-10-07T01:12:20Z,2022-10-07T16:28:41Z,2022-10-07T02:01:07Z,NONE,simonw/datasette/pulls/1838,"This is technically a Sandstorm-specific fix (as external links do not work inside the grain frame), however, I think it is an improvement to the upstream project, so I wanted to propose it here rather than patching it in our package.
There's much opinions on the Internet about whether external links should open in a new tab by default or not, but I'd argue very few people who might click a ""powered by"" link intend to complete their interaction with the source page (a Datasette). And furthermore, users may be working within various queries or loading visualizations (navigating away when trying to plot a million GPS coordinates pretty much just resets your progress!), so linking away within the tab might be a frustrating or destructive act to one's work, even inadvertently.
original report: https://github.com/ocdtrekkie/datasette-sandstorm/issues/1
----
:books: Documentation preview :books:: https://datasette--1838.org.readthedocs.build/en/1838/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1838/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1386456717,PR_kwDOBm6k_c4_oHI4,1820,[SPIKE] Don't truncate query CSVs,536941,fgregg,closed,0,,,,,2,2022-09-26T17:27:01Z,2022-10-07T16:12:17Z,2022-10-07T16:12:17Z,CONTRIBUTOR,simonw/datasette/pulls/1820,"Relates to #526
This is a minimal set of changes needed for having *query* CSVs attempt to download all the rows.
What's good about it is the minimalism.
What's bad about it:
1. We are abusing the `_size` argument to indicate we don't want truncation, which isn't the most obvious thing. Additionally, there are various checks that make sure the ""_size"" URL parameter is a positive integer, which we are relying on to prevent overloading.
2. The default CSV on a table page will use the max_returned_rows argument. Changing this could be a breaking change, since that's currently a place that has some facilities for pagination. Additionally, i think there's a limit under the hood somewhere which if we removed could lead to sql timeouts
3. There are similar reasons for leaving the current streaming method alone, as the current methods could allow for downloading very large files that could have a sql timeout if we tried to get them in one go.
----
:books: Documentation preview :books:: https://datasette--1820.org.readthedocs.build/en/1820/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1820/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1,
1015646369,I_kwDOBm6k_c48iYih,1480,Exceeding Cloud Run memory limits when deploying a 4.8G database,110420,ghing,open,0,,,,,9,2021-10-04T21:20:24Z,2022-10-07T04:39:10Z,,CONTRIBUTOR,,"When I try to deploy a 4.8G SQLite database to Google Cloud Run, I get this error message:
> Memory limit of 8192M exceeded with 8826M used. Consider increasing the memory limit, see https://cloud.google.com/run/docs/configuring/memory-limits
Unfortunately, the maximum amount of memory that can be allocated to an instance is 8192M.
Naively profiling the memory usage of running Datasette with this database locally on my MacBook shows the following memory usage (using Activity Monitor) when I just start up Datasette locally:
- Real Memory Size: 70.6 MB
- Virtual Memory Size: 4.51 GB
- Shared Memory Size: 2.5 MB
- Private Memory Size: 57.4 MB
I'm trying to understand if there's a query or other operation that gets run during container deployment that causes memory use to be so large and if this can be avoided somehow.
This is somewhat related to #1082, but on a different platform, so I decided to open a new issue.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1480/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
860722711,MDU6SXNzdWU4NjA3MjI3MTE=,1301,Publishing to cloudrun with immutable mode?,5413548,louispotok,open,0,,,,,1,2021-04-18T17:51:46Z,2022-10-07T02:38:04Z,,CONTRIBUTOR,,"I'm a bit confused about immutable mode and publishing to cloudrun. (I want to publish with immutable mode so that I can support database downloads.)
Running `datasette publish cloudrun --extra-options=""-i example.db""` leads to an error:
> Error: Invalid value for '-i' / '--immutable': Path 'example.db' does not exist.
However, running `datasette publish cloudrun example.db` not only works but seems to publish in immutable mode anyway! I'm seeing this both with `/-/databases.json` and the fact that downloads are working.
When I just `datasette serve` locally, this succeeds both ways and works as expected.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1301/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1400083043,I_kwDOBm6k_c5Tc5Jj,1834,inspect data is not used for caching database hash,536941,fgregg,closed,0,,,,,0,2022-10-06T17:52:01Z,2022-10-06T20:06:21Z,2022-10-06T20:06:08Z,CONTRIBUTOR,,"When databases are loaded,
https://github.com/simonw/datasette/blob/cb1e093fd361b758120aefc1a444df02462389a3/datasette/app.py#L257-L260
there is nothing preventing the rehashing of the database for immutable databases.
https://github.com/simonw/datasette/blob/cb1e093fd361b758120aefc1a444df02462389a3/datasette/database.py#L50-L53
what i might expect is that relevant values of `inspect_data` get passed to the `Database` class to prevent re-hashing?
With data that is many gigs large, this is a significant start up time.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1834/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1399933513,I_kwDOBm6k_c5TcUpJ,1833,Ability to submit long queries by POST,9599,simonw,open,0,,,,,0,2022-10-06T16:03:26Z,2022-10-06T16:18:00Z,,OWNER,,"Datasette doesn't limit URL lengths but some common web proxies do - the one in front of Google Cloud Run for example limits to 8KB total for incoming request headers: https://cloud.google.com/load-balancing/docs/quotas#https-lb-header-limits
This means longer SQL queries can break!
Need an optional mechanism for submitting queries by POST instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1833/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1387712501,I_kwDOBm6k_c5Sts_1,1824,Convert &_hide_sql=1 to #_hide_sql,562352,CharlesNepote,open,0,,,,,1,2022-09-27T12:53:31Z,2022-10-05T12:56:27Z,,NONE,,"Hiding the SQL textarea with `&_hide_sql=1` enforces a page reload, which can take several seconds and use server resource (which is annoying for big database or complex queries).
It could probably be done with a few lines of Javascript (I'm going to see if I can do that).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1824/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1386917344,PR_kwDOBm6k_c4_prjN,1823,Keyword-only arguments for a bunch of internal methods,9599,simonw,open,0,,,,,3,2022-09-27T00:44:59Z,2022-10-05T04:37:54Z,,OWNER,simonw/datasette/pulls/1823,"Refs #1822
----
:books: Documentation preview :books:: https://datasette--1823.org.readthedocs.build/en/1823/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1823/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1397193691,I_kwDOBm6k_c5TR3vb,1832,__bool__ method on Results,9599,simonw,closed,0,,,,,2,2022-10-05T04:18:12Z,2022-10-05T04:32:33Z,2022-10-05T04:32:33Z,OWNER,,"Wrote this code today: https://github.com/simonw/datasette-public/blob/1401bfae50e71c1dfd2bfb6954f2e86d5a7ab21b/datasette_public/__init__.py#L41
```python
results = await db.execute(
""select 1 from _public_tables where table_name = ?"", [table_name]
)
if len(results):
return True
```
Would be nice if I could use `if results` there instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1832/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1396977994,I_kwDOBm6k_c5TRDFK,1830,Add documentation for writing tests with signed actor cookies,9599,simonw,open,0,,,,,0,2022-10-04T23:51:26Z,2022-10-04T23:51:26Z,,OWNER,,"I use this pattirn in a lot of plugin tests, e.g. https://github.com/simonw/datasette-edit-templates/blob/087f6a6cabc20020f2b0524f11aa3a7836320848/tests/test_edit_templates.py#L55-L58
```python
actor = ds.sign({""a"": {""id"": ""root""}}, ""actor"")
response1 = await ds.client.get(
""/-/edit-templates/_footer.html"", cookies={""ds_actor"": actor}
)
```
I should add this to the documentation on this page: https://docs.datasette.io/en/latest/testing_plugins.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1830/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1363552780,I_kwDOBm6k_c5RRioM,1805,truncate_cells_html does not work for links?,562352,CharlesNepote,open,0,,,,,7,2022-09-06T16:41:29Z,2022-10-03T09:18:06Z,,NONE,,"We have many links inside our dataset (please don't blame us ;-).
When I use `--settings truncate_cells_html 60` it is not working for the links.
Eg. https://images.openfoodfacts.org/images/products/000/000/000/088/nutrition_fr.5.200.jpg (87 chars) is not truncated:
![image](https://user-images.githubusercontent.com/562352/188689045-1946d776-2305-47cf-bfc5-b5685b9206b7.png)
IMHO It would make sense that links should be treated as HTML. The link should work of course, but Datasette could truncate it:
[https://images.openfoodfacts.org/images/products/00[...].jpg](https://images.openfoodfacts.org/images/products/000/000/000/088/nutrition_fr.5.200.jpg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1805/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened
447469253,MDU6SXNzdWU0NDc0NjkyNTM=,485,Improvements to table label detection ,9599,simonw,open,0,9599,simonw,,,10,2019-05-23T06:19:49Z,2022-10-03T00:04:42Z,,OWNER,,"Label detection doesn't work if the primary key is called pk rather than id, so this page doesn't work: https://latest.datasette.io/fixtures/roadside_attraction_characteristics
Code is here:
https://github.com/simonw/datasette/blob/cccea85be6aaaeadb31f3b588ec7f732628815f5/datasette/app.py#L644-L653",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/485/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1393903845,I_kwDOBm6k_c5TFUjl,1828,word-wrap: anywhere resulting in weird display,9599,simonw,closed,0,,,,,2,2022-10-02T21:25:03Z,2022-10-02T23:01:17Z,2022-10-02T23:01:17Z,OWNER,,"e.g. on https://github-to-sqlite.dogsheep.net/github/commits
This is from a change introduced here: https://github.com/simonw/datasette/commit/bf8d84af5422606597be893cedd375020cb2b369 in #1805
https://github.com/simonw/datasette/blob/bf8d84af5422606597be893cedd375020cb2b369/datasette/static/app.css#L447-L450",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1828/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1392426838,PR_kwDOBm6k_c4_8BMC,1827,Bump furo from 2022.9.15 to 2022.9.29,49699333,dependabot[bot],closed,0,,,,,1,2022-09-30T13:15:35Z,2022-09-30T17:55:42Z,2022-09-30T17:55:41Z,CONTRIBUTOR,simonw/datasette/pulls/1827,"Bumps [furo](https://github.com/pradyunsg/furo) from 2022.9.15 to 2022.9.29.
Changelog
Sourced from furo's changelog .
Changelog
2022.09.29 -- Quaint Quartz
Add ability to set arbitrary URLs for edit button.
Add support for aligning text in MyST-parser generated tables.
2022.09.15 -- Pragmatic Pistachio
Add a minimum version constraint on pygments.
Add an explicit dependency on sass
.
Change right sidebar title from "Contents" to "On this page".
Correctly position sidebars on small screens.
Correctly select only Furo's own svg
in related pages nav
.
Make numpy-style documentation headers consistent.
Retitle the reference section.
Update npm dependencies.
2022.06.21 -- Opulent Opal
Fix docutils <= 0.17.x
compatibility.
Bump to the latest Node.js LTS.
2022.06.04.1 -- Naughty Nickel bugfix
Fix the URL used in the "Edit this page" for Read the Docs builds.
2022.06.04 -- Naughty Nickel
✨ Advertise Sphinx 5 compatibility.
✨ Change to basic-ng
as the base theme (from {pypi}sphinx-basic-ng
).
Document site-wide announcement banners.
Drop the pin on pygments.
Improve edit button, using basic-ng
's edit-this-page
component.
Tweak headings to better match what users expect.
Tweak how Sphinx's default HTML is rendered, using docutils post-transforms (this replaces parsing+modifying it with BeautifulSoup).
When built with docutils 0.18, footnotes are rendered differently and stylised differently in Furo.
2022.04.07 -- Magical Mauve
... (truncated)
Commits
1375f9d
Prepare release: 2022.09.29
af43607
Update changelog
bc0fe52
Update user-facing documentation for edit button
509c558
Modernise the edit-this-page.html
template
5a0ceca
Add source_edit_link
as a theme configuration parameter (#510 )
52fc32f
Build documentation in pull requests
149f77b
Fix stylesheet for MyST tables
9af2e44
Support MyST table column alignment (#531 )
82dd61c
Back to development
See full diff in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.9.15&new-version=2022.9.29)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1827.org.readthedocs.build/en/1827/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1827/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
377155320,MDU6SXNzdWUzNzcxNTUzMjA=,370,Integration with JupyterLab,82988,psychemedia,open,0,,,,,4,2018-11-04T13:57:13Z,2022-09-29T08:17:47Z,,CONTRIBUTOR,,"I just watched a demo video for the [JupyterLab Chart Editor](https://www.crowdcast.io/e/introducing-JupyterLab-Chart-Editor/) which wraps the plotly chart editor app in a JupyterLab panel and lets you open a plotly chart JSON file in that editor. Essentially, it pops an HTML app into a panel in JupyterLab, and I think registers the app as a file viewer for a particular file type. (I'm not completely taken by it, tbh, because it means you can do irreproducible things to the chart definition file, but that's another issue).
JupyterLab extensions can also open files from a dialogue as the iframe/html previewer shows: https://github.com/timkpaine/jupyterlab_iframe.
This made me wonder about what `datasette` integration with JupyterLab might do.
For example, by right-clicking on a CSV file (for which there is already a CSV table view) in the file browser, offer a *View / Run as datasette* file viewer option that will:
- run the CSV file through `csvs-to-sqlite`;
- launch the `datasette` server and display the `datasette` view in a JupyterLab panel.
(? Create a new SQLite db for each CSV file and launch each datasette view on a new port? Or have a JupyterLab (session?) SQLite db that stores all `datasette` viewed CSVs and runs on a single port?)
As a freebie, the `datasette` API would allow you to run efficient SQL queries against the file eg using using `pandas.read_sql()` queries in a notebook in the same space.
Related:
- [JupyterLab extensions docs](https://jupyterlab.readthedocs.io/en/stable/user/extensions.html)
- a [cookiecutter for wrting JupyterLab extensions using Javascript](https://github.com/jupyterlab/extension-cookiecutter-js)
- a [cookiecutter for writing JupyterLab extensions using Typescript](https://github.com/jupyterlab/extension-cookiecutter-ts)
- tutorial: [Let’s Make an xkcd JupyterLab Extension](https://jupyterlab.readthedocs.io/en/stable/developer/xkcd_extension_tutorial.html)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/370/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1122427321,I_kwDOBm6k_c5C5uG5,1624,Index page `/` has no CORS headers,9599,simonw,open,0,,,,,2,2022-02-02T21:56:10Z,2022-09-28T16:54:22Z,,OWNER,,"Compare the following:
```
% curl -I 'https://latest.datasette.io/fixtures'
HTTP/1.1 200 OK
link: https://latest.datasette.io/fixtures.json; rel=""alternate""; type=""application/json+datasette""
cache-control: max-age=5
referrer-policy: no-referrer
access-control-allow-origin: *
access-control-allow-headers: Authorization
access-control-expose-headers: Link
content-type: text/html; charset=utf-8
x-databases: _memory, _internal, fixtures, extra_database
Date: Wed, 02 Feb 2022 21:55:49 GMT
Server: Google Frontend
Transfer-Encoding: chunked
% curl -I 'https://latest.datasette.io/'
HTTP/1.1 200 OK
link: https://latest.datasette.io/.json; rel=""alternate""; type=""application/json+datasette""
content-type: text/html; charset=utf-8
x-databases: _memory, _internal, fixtures, extra_database
Date: Wed, 02 Feb 2022 21:55:52 GMT
Server: Google Frontend
Transfer-Encoding: chunked
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1624/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
732674148,MDU6SXNzdWU3MzI2NzQxNDg=,1062,Refactor .csv to be an output renderer - and teach register_output_renderer to stream all rows,9599,simonw,open,0,,,3268330,Datasette 1.0,5,2020-10-29T21:25:02Z,2022-09-28T14:09:54Z,,OWNER,,This can drive the upgrade of the `register_output_renderer` hook to be able to handle streaming all rows in a large query.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1062/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1388631785,I_kwDOBm6k_c5SxNbp,1826,render_cell documentation example doesn't match the method signature,66709385,pjamargh,closed,0,,,,,3,2022-09-28T02:37:59Z,2022-09-28T04:30:28Z,2022-09-28T04:05:16Z,NONE,,"Open Datasette stable doc at https://docs.datasette.io/en/stable/plugin_hooks.html?highlight=render_cell#render-cell-row-value-column-table-database-datasette
render_cell plugin hook method signature is `render_cell(row, value, column, table, database, datasette)`, the example shown inline uses `render_cell(value)`.
![image](https://user-images.githubusercontent.com/66709385/192674691-34265b81-6cdd-41d2-8424-aa12f8bc8c94.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1826/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1388227245,PR_kwDOBm6k_c4_uCkO,1825,Add documentation for serving via OpenRC,1048831,asimpson,closed,0,,,,,2,2022-09-27T19:00:56Z,2022-09-28T04:21:37Z,2022-09-28T04:21:37Z,CONTRIBUTOR,simonw/datasette/pulls/1825,"I also removed a few lines which felt redundant given the following section dedicated to running behind a nginx proxy.
----
:books: Documentation preview :books:: https://datasette--1825.org.readthedocs.build/en/1825/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1825/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
459882902,MDU6SXNzdWU0NTk4ODI5MDI=,526,Stream all results for arbitrary SQL and canned queries,50578294,matej-fr,open,0,,,,,23,2019-06-24T13:09:45Z,2022-09-28T04:01:25Z,,NONE,,"I think that there is a difficulty with canned queries.
When I want to stream all results of a canned query TwoDays I get only first 1.000 records.
Example:
`http://myserver/history_sample/two_days.csv?_stream=on`
returns only first 1.000 records.
If I do the same with the whole database i.e.
`http://myserver/history_sample/database.csv?_stream=on`
I get correctly all records.
Any ideas?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/526/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1386854246,I_kwDOBm6k_c5Sqbdm,1822,Switch to keyword-only arguments for a bunch of internal methods,9599,simonw,open,0,,,3268330,Datasette 1.0,3,2022-09-26T23:20:38Z,2022-09-27T00:44:04Z,,OWNER,,"This is a good idea, and one that needs to happen before Datasette 1.0:
> While you are adding features, would you be future-proofing your APIs if you switched over some arguments over to keyword-only arguments or would that be too disruptive?
>
> Thinking out loud:
>
> ```
> async def render_template(
> self, templates, *, context=None, plugin_context=None, request=None, view_name=None
> ):
> ```
_Originally posted by @jefftriplett in https://github.com/simonw/datasette/issues/1817#issuecomment-1256781274_
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1822/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1384273985,I_kwDOBm6k_c5SglhB,1817,Expose `sql` and `params` arguments to various plugin hooks,9599,simonw,open,0,,,,,7,2022-09-23T20:34:45Z,2022-09-27T00:27:53Z,,OWNER,,"On Discord: https://discord.com/channels/823971286308356157/996877076982415491/1022784534363787305
> Hi! I'm attempting to write a plugin that would provide some statistics on text fields (most common words, etc). I would want this information displayed in the table pages, and (ideally) also updated when users make custom queries from the table pages.
>
> It seems one way to do this would be to use the extra_template_vars hook, and make the appropriate SQL query there. So extra_template_vars would create a variable that is a list of most common words, and this is displayed on the page, possibly above the regular table view.
>
> Is there a way that the plugin code can access the SQL query (or even the data) that was used to produce the table view? I can see that TableView class constructs the SQL query, but I can't seem to find a way to access that information from the objects that are available to extra_template_vars.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1817/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1385026210,I_kwDOBm6k_c5SjdKi,1819,Preserve query on timeout,2182,danp,closed,0,,,,,3,2022-09-25T13:32:31Z,2022-09-26T23:16:15Z,2022-09-26T23:06:06Z,CONTRIBUTOR,,"If a query hits the timeout it shows a message like:
> SQL query took too long. The time limit is controlled by the [sql_time_limit_ms](https://docs.datasette.io/en/stable/settings.html#sql-time-limit-ms) configuration option.
But the query is lost. Hitting the browser back button shows the query _before_ the one that errored.
It would be nice if the query that errored was preserved for more tweaking. This would make it similar to how ""invalid syntax"" works since #1346 / #619.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1819/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1386734383,I_kwDOBm6k_c5Sp-Mv,1821,Release Datasette 0.63a0,9599,simonw,closed,0,,,,,1,2022-09-26T21:15:27Z,2022-09-26T22:06:39Z,2022-09-26T22:06:39Z,OWNER,,"> - The [prepare_jinja2_environment(env, datasette)](https://docs.datasette.io/en/latest/plugin_hooks.html#plugin-hook-prepare-jinja2-environment) plugin hook now accepts an optional `datasette` argument. Hook implementations can also now return an `async` function which will be awaited automatically. ([#1809](https://github.com/simonw/datasette/issues/1809))
> - `--load-extension` option now supports entrypoints. Thanks, Alex Garcia. ([#1789](https://github.com/simonw/datasette/pull/1789))
> - New tutorial: [Cleaning data with sqlite-utils and Datasette](https://datasette.io/tutorials/clean-data).
> - Facet size can now be set per-table with the new `facet_size` table metadata option. ([#1804](https://github.com/simonw/datasette/issues/1804))
> - `truncate_cells_html` setting now also affects long URLs in columns. ([#1805](https://github.com/simonw/datasette/issues/1805))
> - `Database(is_mutable=)` now defaults to `True`. ([#1808](https://github.com/simonw/datasette/issues/1808))
> - Non-JavaScript textarea now increases height to fit the SQL query. ([#1786](https://github.com/simonw/datasette/issues/1786))
> - More detailed command descriptions on the [CLI reference](https://docs.datasette.io/en/latest/cli-reference.html#cli-reference) page. ([#1787](https://github.com/simonw/datasette/issues/1787))
> - Datasette no longer enforces upper bounds on its depenedencies. ([#1800](https://github.com/simonw/datasette/issues/1800))
> - Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ([#1794](https://github.com/simonw/datasette/pull/1794))
> - The `settings.json` file used in [Configuration directory mode](https://docs.datasette.io/en/latest/settings.html#config-dir) is now validated on startup. ([#1816](https://github.com/simonw/datasette/issues/1816))",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1821/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1217759117,I_kwDOBm6k_c5IlYeN,1727,Research: demonstrate if parallel SQL queries are worthwhile,9599,simonw,open,0,,,,,32,2022-04-27T18:54:21Z,2022-09-26T14:48:31Z,,OWNER,,"I added parallel SQL query execution here:
- https://github.com/simonw/datasette/issues/1723
My hunch is that this will take advantage of multiple cores, since Python's `sqlite3` module releases the GIL once a query is passed to SQLite.
I'd really like to prove this is the case though. Just not sure how to do it!
Larger question: is this performance optimization actually improving performance at all? Under what circumstances is it worthwhile?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1727/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1378640768,I_kwDOBm6k_c5SLGOA,1816,Validate settings.json on startup in configuration directory mode,9599,simonw,closed,0,,,,,2,2022-09-19T23:35:18Z,2022-09-20T01:15:48Z,2022-09-20T01:15:48Z,OWNER,,"> It might have been useful for Datasette to show an error when started against a `settings.json` file that contains an invalid setting though.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1814#issuecomment-1251677554_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1816/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1378495690,I_kwDOBm6k_c5SKizK,1814,Static files not served,4068,frafra,closed,0,,,,,2,2022-09-19T20:38:17Z,2022-09-19T23:35:06Z,2022-09-19T23:34:30Z,NONE,,"Folder structure:
```
bibliography/
bibliography/static-files
bibliography/static-files/styles.css
bibliography/bibliography.db
bibliography/metadata.json
bibliography/settings.json
```
```
$ cat bibliography/settings.json
{
""suggest_facets"": false,
""truncate_cells_html"": 1000,
""static"": ""assets:static-files/""
}
```
File `/assets/styles.css` is not found (HTTP 404, `Database not found: assets`).
Using datasette revision d0737e4de51ce178e556fc011ccb8cc46bbb6359.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1814/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1378636455,I_kwDOBm6k_c5SLFKn,1815,"`datasette publish provider .` to publish whole directory, similar to configuration directory mode",9599,simonw,open,0,,,,,0,2022-09-19T23:28:59Z,2022-09-19T23:29:11Z,,OWNER,,"> I haven't done this with any of my other `datasette publish` tools, but I do think it's a good idea. Being able to publish the entire directory - with templates and plugins and metadata - does seem very useful to me.
_Originally posted by @simonw in https://github.com/simonw/datasette-publish-fly/issues/23#issuecomment-1251673489_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1815/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1377811868,I_kwDOBm6k_c5SH72c,1813,missing next and next_url in JSON responses from an instance deployed on Fly ,883348,adipasquale,closed,0,,,,,1,2022-09-19T11:32:34Z,2022-09-19T11:34:45Z,2022-09-19T11:34:45Z,CONTRIBUTOR,,"👋 thank you for an incredibly useful project!
I have noticed that my deployed instance on Fly does not include the `next` and `next_url` keys even for a truncated response :
This is publically accessible here: `https://collectif-objets-datasette.fly.dev/collectif-objets.json?sql=select+*+from+mairies`
However when I run the dataset server locally with the same data I get these next keys for the exact same query:
I am wondering if I've missed some config or something specific to deployments on Fly.io?
I am running datasette v0.62, without any specific config :
- locally `poetry run datasette data/collectif-objets.sqlite`
- for the deploy : `poetry run datasette publish fly data/collectif-objets.sqlite`
as visible in [the Makefile](https://github.com/adipasquale/collectif-objets-datasette/blob/main/Makefile). _The very limited codebase is public but the sqlite db is not versioned yet because it is too large._",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1813/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1373595927,I_kwDOBm6k_c5R32kX,1809,`prepare_jinja2_environment()` hook should take `datasette` argument,9599,simonw,closed,0,,,,,11,2022-09-14T21:15:46Z,2022-09-17T03:39:05Z,2022-09-17T03:38:33Z,OWNER,,"That plugin hook's current signature is:
https://github.com/simonw/datasette/blob/610425460b519e9c16d386cb81aa081c9d730ef0/datasette/hookspecs.py#L28-L30
As a result in the first alpha release of `datasette-edit-templates` I had to include this horrific hack: https://github.com/simonw/datasette-edit-templates/blob/087f6a6cabc20020f2b0524f11aa3a7836320848/datasette_edit_templates/__init__.py#L72-L75
```python
@hookimpl
def prepare_jinja2_environment(env):
# TODO: This should ideally take datasette, but that's not an argument yet
datasette = inspect.currentframe().f_back.f_back.f_back.f_back.f_locals[""self""]
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1809/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1375930971,PR_kwDOBm6k_c4_GVBS,1812,Bump furo from 2022.6.21 to 2022.9.15,49699333,dependabot[bot],closed,0,,,,,3,2022-09-16T13:10:45Z,2022-09-16T19:50:53Z,2022-09-16T19:50:52Z,CONTRIBUTOR,simonw/datasette/pulls/1812,"Bumps [furo](https://github.com/pradyunsg/furo) from 2022.6.21 to 2022.9.15.
Changelog
Sourced from furo's changelog .
Changelog
2022.09.15 -- Pragmatic Pistachio
Add a minimum version constraint on pygments.
Add an explicit dependency on sass
.
Change right sidebar title from "Contents" to "On this page".
Correctly position sidebars on small screens.
Correctly select only Furo's own svg
in related pages nav
.
Make numpy-style documentation headers consistent.
Retitle the reference section.
Update npm dependencies.
2022.06.21 -- Opulent Opal
Fix docutils <= 0.17.x
compatibility.
Bump to the latest Node.js LTS.
2022.06.04.1 -- Naughty Nickel bugfix
Fix the URL used in the "Edit this page" for Read the Docs builds.
2022.06.04 -- Naughty Nickel
✨ Advertise Sphinx 5 compatibility.
✨ Change to basic-ng
as the base theme (from {pypi}sphinx-basic-ng
).
Document site-wide announcement banners.
Drop the pin on pygments.
Improve edit button, using basic-ng
's edit-this-page
component.
Tweak headings to better match what users expect.
Tweak how Sphinx's default HTML is rendered, using docutils post-transforms (this replaces parsing+modifying it with BeautifulSoup).
When built with docutils 0.18, footnotes are rendered differently and stylised differently in Furo.
2022.04.07 -- Magical Mauve
✨ Make sphinx-copybutton look better.
Add margin to indentations in line blocks.
Add styling for non-arabic list styles
Add support for html_baseurl
.
... (truncated)
Commits
08e6b38
Prepare release: 2022.09.15
9de7613
Update changelog
a064929
Tweak changelog content style
46f4adc
Revert "Add initial theme.conf content for eventual ablog support"
45b839b
Set a minimum constraint on pygments
a4af988
[pre-commit.ci] pre-commit autoupdate (#518 )
a72186f
[pre-commit.ci] pre-commit autoupdate (#504 )
9f41ee6
Add initial theme.conf content for eventual ablog support
75e0361
Make numpy-style documentation headers consistent
9d280e6
[pre-commit.ci] pre-commit autoupdate (#487 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.6.21&new-version=2022.9.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1812.org.readthedocs.build/en/1812/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1812/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1375792876,I_kwDOBm6k_c5SAO7s,1811,"Drop-down menu with ""REGEXP"" choice",562352,CharlesNepote,open,0,,,,,0,2022-09-16T11:06:18Z,2022-09-16T15:30:31Z,,NONE,,"Drop-down menu below could add ""REGEXP"" choice when REGEXP sqlite extension is installed and used
![image](https://user-images.githubusercontent.com/562352/190675352-810fbdca-0827-4034-8b9f-fd67d5c35afb.png)
Not sure. Close the issue if you don't find it relevant.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1811/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1374626873,I_kwDOBm6k_c5R7yQ5,1810,Featured table(s) on the homepage,9599,simonw,open,0,,,,,4,2022-09-15T14:30:49Z,2022-09-15T15:51:25Z,,OWNER,,"Many Datasette instances mainly exist to serve a single table - for example:
- https://global-power-plants.datasettes.com/global-power-plants/global-power-plants
- https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/images
It would be neat if the / homepage of those instances could be configured to highlight that specific table.
Or maybe more than one?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1810/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1368030952,I_kwDOBm6k_c5Rin7o,1808,Database() constructor currently defaults is_mutable to False,9599,simonw,closed,0,,,,,5,2022-09-09T16:02:41Z,2022-09-09T16:37:57Z,2022-09-09T16:19:25Z,OWNER,,"This is surprising. It caused a bug in `datasette-upload-dbs` because I didn't expect it to do that.
> I think this is an API design flaw in Datasette itself, but I can fix it here first.
_Originally posted by @simonw in https://github.com/simonw/datasette-upload-dbs/issues/6#issuecomment-1242150394_
Code in question: https://github.com/simonw/datasette/blob/bf8d84af5422606597be893cedd375020cb2b369/datasette/database.py#L29-L32",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1808/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1366915240,I_kwDOBm6k_c5ReXio,1807,Plugin ecosystem needs to avoid crashes due to no available databases,9599,simonw,open,0,,,,,1,2022-09-08T19:54:34Z,2022-09-08T20:14:05Z,,OWNER,,"Opening this here to track the issue first reported in:
- https://github.com/simonw/datasette-upload-dbs/issues/5
Plugins that expect to be able to write to a database need to not crash in situations where no writable database is available.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1807/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1365741480,I_kwDOBm6k_c5RZ4-o,1806,"UX to recover from Error 500: ""You can only execute one statement at a time.""",1470389,jieter,open,0,,,,,0,2022-09-08T08:01:27Z,2022-09-08T08:01:37Z,,NONE,,"When using the Custom SQL query view, when accidentally adding a semicolon in the middle of my query, datasette errors with:
> # Error 500
> You can only execute one statement at a time.
The error view doesn't contain the query textarea anymore, so it provides no easy way recover from the error. It would be nice if I could change and submit it again.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1806/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
507454958,MDU6SXNzdWU1MDc0NTQ5NTg=,596,Handle really wide tables better,9599,simonw,open,0,,,,,9,2019-10-15T20:05:46Z,2022-09-07T00:58:41Z,,OWNER,,"If a table has hundreds of columns the Datasette UI starts getting unwieldy.
Addressing this would be neat. One option would be to only select the first 30 columns by default and provide a UI for selecting more.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/596/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1363440999,I_kwDOBm6k_c5RRHVn,1804,Ability to set a custom facet_size per table,9599,simonw,closed,0,,,,,6,2022-09-06T15:11:40Z,2022-09-07T00:21:56Z,2022-09-06T18:06:53Z,OWNER,,"Suggestion from Discord: https://discord.com/channels/823971286308356157/823971286941302908/1016725586351247430
> Is it possible to limit the facet size per database or even per table?
This is a really good idea, it could be done in `metadata.yml`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1804/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1268121674,PR_kwDOBm6k_c45fz-O,1757,feat: add a wildcard for _json columns,163156,ytjohn,open,0,,,,,1,2022-06-11T01:01:17Z,2022-09-06T00:51:21Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1757,"This allows _json to accept a wildcard for when there are many JSON columns that the user wants to convert. I hope this is useful. I've tested it on our datasette and haven't ran into any issues. I imagine on a large set of results, there could be some performance issues, but it will probably be negligible for most use cases.
On a side note, I ran into an issue where I had to upgrade black on my system beyond the pinned version in setup.py. Here is the upstream issue < . I didn't include this in the PR yet since I didn't look into the issue too far, but I can if you would like.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1757/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1352931076,PR_kwDOBm6k_c495vvy,1794,fix word break in facets by adding ul.tight-bullets li word-break: break-all,128286,dmr,closed,0,,,,,1,2022-08-27T03:47:25Z,2022-09-06T00:45:41Z,2022-09-06T00:45:41Z,CONTRIBUTOR,simonw/datasette/pulls/1794,"I noticed that long words break the layout of facets:
![image](https://user-images.githubusercontent.com/128286/187013146-fb2bbb60-a225-441b-ba8e-b9e74fb04f93.png)
So I added CSS to add a line break. This is how the result looks now:
![image](https://user-images.githubusercontent.com/128286/187013175-a706fc72-9e69-4a75-9bdf-bdaa34a0cf51.png)
I don't know enough about facet edge cases to decide if this change might break other things but it looks better for me so maybe this is helpful.
----
:books: Documentation preview :books:: https://datasette--1794.org.readthedocs.build/en/1794/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1794/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1351949898,PR_kwDOBm6k_c492dPw,1793,Added a useful resource,111973926,MobiWancode,closed,0,,,,,1,2022-08-26T08:41:26Z,2022-09-06T00:41:25Z,2022-09-06T00:41:24Z,NONE,simonw/datasette/pulls/1793,"Have added a useful resource about the types of databases in SQL i.e SQLite, PostgreSQL, MySQL &, etc from the scaler topics.
----
:books: Documentation preview :books:: https://datasette--1793.org.readthedocs.build/en/1793/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1793/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1362402998,I_kwDOBm6k_c5RNJ62,1802,Tests reliably failing on Python 3.7,9599,simonw,closed,0,,,,,15,2022-09-05T19:21:16Z,2022-09-06T00:40:20Z,2022-09-06T00:40:20Z,OWNER,,"
https://github.com/simonw/datasette/runs/8194907739?check_suite_focus=true
I thought this might be an intermittent failure but attempts to re-run the tests have not made it pass.
End of that trace is:
```
/home/runner/work/datasette/datasette/datasette/app.py:234: in __init__
self._refresh_schemas_lock = asyncio.Lock()
/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/locks.py:161: in __init__
self._loop = events.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
def get_event_loop(self):
""""""Get the event loop for the current context.
Returns an instance of EventLoop or raises an exception.
""""""
if (self._local._loop is None and
not self._local._set_called and
isinstance(threading.current_thread(), threading._MainThread)):
self.set_event_loop(self.new_event_loop())
if self._local._loop is None:
raise RuntimeError('There is no current event loop in thread %r.'
> % threading.current_thread().name)
E RuntimeError: There is no current event loop in thread 'MainThread'.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1802/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1362567197,PR_kwDOBm6k_c4-ZxWD,1803,Workaround for test failure: RuntimeError: There is no current event loop,9599,simonw,closed,0,,,,,1,2022-09-06T00:31:06Z,2022-09-06T00:40:19Z,2022-09-06T00:40:19Z,OWNER,simonw/datasette/pulls/1803,"Closes #1802
----
:books: Documentation preview :books:: https://datasette--1803.org.readthedocs.build/en/1803/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1803/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
903986178,MDU6SXNzdWU5MDM5ODYxNzg=,1344,Test Datasette Docker images built for different architectures,9599,simonw,open,0,,,,,10,2021-05-27T16:52:29Z,2022-09-06T00:07:58Z,,OWNER,,"Continuing on from #1319 - now that we have the ability to build Datasette's Docker image against multiple architectures we should test that it works.
We can do this with QEMU emulation, see https://twitter.com/nevali/status/1397958044571602945",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1344/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1358848933,PR_kwDOBm6k_c4-NhzQ,1797,Bump black from 22.6.0 to 22.8.0,49699333,dependabot[bot],closed,0,,,,,0,2022-09-01T13:25:14Z,2022-09-05T18:51:52Z,2022-09-05T18:51:52Z,CONTRIBUTOR,simonw/datasette/pulls/1797,"Bumps [black](https://github.com/psf/black) from 22.6.0 to 22.8.0.
Release notes
Sourced from black's releases .
22.8.0
Highlights
Python 3.11 is now supported, except for blackd as aiohttp does not support 3.11 as of publishing (#3234 )
This is the last release that supports running Black on Python 3.6 (formatting 3.6 code will continue to be supported until further notice)
Reword the stability policy to say that we may, in rare cases, make changes that affect code that was not previously formatted by Black (#3155 )
Stable style
Fix an infinite loop when using # fmt: on/off
in the middle of an expression or code block (#3158 )
Fix incorrect handling of # fmt: skip
on colon (:
) lines (#3148 )
Comments are no longer deleted when a line had spaces removed around power operators (#2874 )
Preview style
Single-character closing docstring quotes are no longer moved to their own line as this is invalid. This was a bug introduced in version 22.6.0. (#3166 )
--skip-string-normalization
/ -S
now prevents docstring prefixes from being normalized as expected (#3168 )
When using --skip-magic-trailing-comma
or -C
, trailing commas are stripped from subscript expressions with more than 1 element (#3209 )
Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside parentheses (#3162 )
Fix a string merging/split issue when a comment is present in the middle of implicitly concatenated strings on its own line (#3227 )
Blackd
blackd
now supports enabling the preview style via the X-Preview
header (#3217 )
Configuration
Black now uses the presence of debug f-strings to detect target version (#3215 )
Fix misdetection of project root and verbose logging of sources in cases involving --stdin-filename
(#3216 )
Immediate .gitignore
files in source directories given on the command line are now also respected, previously only .gitignore
files in the project root and automatically discovered directories were respected (#3237 )
Documentation
Recommend using BlackConnect in IntelliJ IDEs (#3150 )
Integrations
Vim plugin: prefix messages with Black:
so it's clear they come from Black (#3194 )
Docker: changed to a /opt/venv installation + added to PATH to be available to non-root users (#3202 )
Output
Change from deprecated asyncio.get_event_loop()
to create our event loop which removes DeprecationWarning (#3164 )
Remove logging from internal blib2to3
library since it regularly emits error logs about failed caching that can and should be ignored (#3193 )
Parser
Type comments are now included in the AST equivalence check consistently so accidental deletion raises an error. Though type comments can't be tracked when running on PyPy 3.7 due to standard library limitations. (#2874 )
Performance
... (truncated)
Changelog
Sourced from black's changelog .
22.8.0
Highlights
Python 3.11 is now supported, except for blackd as aiohttp does not support 3.11 as
of publishing (#3234 )
This is the last release that supports running Black on Python 3.6 (formatting 3.6
code will continue to be supported until further notice)
Reword the stability policy to say that we may, in rare cases, make changes that
affect code that was not previously formatted by Black (#3155 )
Stable style
Fix an infinite loop when using # fmt: on/off
in the middle of an expression or code
block (#3158 )
Fix incorrect handling of # fmt: skip
on colon (:
) lines (#3148 )
Comments are no longer deleted when a line had spaces removed around power operators
(#2874 )
Preview style
Single-character closing docstring quotes are no longer moved to their own line as
this is invalid. This was a bug introduced in version 22.6.0. (#3166 )
--skip-string-normalization
/ -S
now prevents docstring prefixes from being
normalized as expected (#3168 )
When using --skip-magic-trailing-comma
or -C
, trailing commas are stripped from
subscript expressions with more than 1 element (#3209 )
Implicitly concatenated strings inside a list, set, or tuple are now wrapped inside
parentheses (#3162 )
Fix a string merging/split issue when a comment is present in the middle of implicitly
concatenated strings on its own line (#3227 )
Blackd
blackd
now supports enabling the preview style via the X-Preview
header (#3217 )
Configuration
Black now uses the presence of debug f-strings to detect target version (#3215 )
Fix misdetection of project root and verbose logging of sources in cases involving
--stdin-filename
(#3216 )
Immediate .gitignore
files in source directories given on the command line are now
also respected, previously only .gitignore
files in the project root and
automatically discovered directories were respected (#3237 )
Documentation
Recommend using BlackConnect in IntelliJ IDEs (#3150 )
Integrations
... (truncated)
Commits
2018e66
Prepare docs for release 22.8.0 (#3248 )
0019261
Update stable branch after publishing to PyPI (#3223 )
7757078
Improve & update release process to reflect recent changes (#3242 )
767604e
Use .gitignore files in the initial source directories (#3237 )
2c90480
Use strict mypy checking (#3222 )
ba618a3
Add parens around implicit string concatenations where it increases readabili...
c0cc19b
Delay worker count determination
afed2c0
Load .gitignore and exclude regex at time of use
e269f44
Lazily import parallelized format modules
c47b91f
Fix misdetection of project root with --stdin-filename
(#3216 )
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.6.0&new-version=22.8.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1797.org.readthedocs.build/en/1797/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1797/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1362363685,I_kwDOBm6k_c5RNAUl,1800,Remove upper bound dependencies as a default policy,9599,simonw,closed,0,,,,,3,2022-09-05T18:23:45Z,2022-09-05T18:39:52Z,2022-09-05T18:35:41Z,OWNER,,"https://iscinumpy.dev/post/bound-version-constraints/ has convinced me not to use upper bound dependencies unless I'm certain they are needed.
Relevant PR:
- https://github.com/simonw/datasette/pull/1799
Also:
https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L45-L46
https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L48-L49
https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L51-L55
https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L57-L59
https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L75-L78
https://github.com/simonw/datasette/blob/ba35105eee2d3ba620e4f230028a02b2e2571df2/setup.py#L81-L82
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1800/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1180778860,PR_kwDOBm6k_c41BFWj,1685,"Update jinja2 requirement from <3.1.0,>=2.10.3 to >=2.10.3,<3.2.0",49699333,dependabot[bot],closed,0,,,,,3,2022-03-25T13:12:13Z,2022-09-05T18:36:49Z,2022-09-05T18:36:48Z,CONTRIBUTOR,simonw/datasette/pulls/1685,"Updates the requirements on [jinja2](https://github.com/pallets/jinja) to permit the latest version.
Release notes
Sourced from jinja2's releases .
3.1.0
This is a feature release, which includes new features and removes previously deprecated features. The 3.1.x branch is now the supported bugfix branch, the 3.0.x branch has become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as pip-tools to pin all dependencies and control upgrades. We also encourage upgrading to MarkupSafe 2.1.1, the latest version at this time.
Changelog
Sourced from jinja2's changelog .
Version 3.1.0
Released 2022-03-24
Drop support for Python 3.6. :pr:1534
Remove previously deprecated code. :pr:1544
WithExtension
and AutoEscapeExtension
are built-in now.
contextfilter
and contextfunction
are replaced by
pass_context
. evalcontextfilter
and
evalcontextfunction
are replaced by pass_eval_context
.
environmentfilter
and environmentfunction
are replaced
by pass_environment
.
Markup
and escape
should be imported from MarkupSafe.
Compiled templates from very old Jinja versions may need to be
recompiled.
Legacy resolve mode for Context
subclasses is no longer
supported. Override resolve_or_missing
instead of
resolve
.
unicode_urlencode
is renamed to url_quote
.
Add support for native types in macros. :issue:1510
The {% trans %}
tag can use pgettext
and npgettext
by
passing a context string as the first token in the tag, like
{% trans "title" %}
. :issue:1430
Update valid identifier characters from Python 3.6 to 3.7.
:pr:1571
Filters and tests decorated with @async_variant
are pickleable.
:pr:1612
Add items
filter. :issue:1561
Subscriptions ([0]
, etc.) can be used after filters, tests, and
calls when the environment is in async mode. :issue:1573
The groupby
filter is case-insensitive by default, matching
other comparison filters. Added the case_sensitive
parameter to
control this. :issue:1463
Windows drive-relative path segments in template names will not
result in FileSystemLoader
and PackageLoader
loading from
drive-relative paths. :pr:1621
Version 3.0.3
Released 2021-11-09
Fix traceback rewriting internals for Python 3.10 and 3.11.
:issue:1535
Fix how the native environment treats leading and trailing spaces
when parsing values on Python 3.10. :pr:1537
... (truncated)
Commits
84c0e2c
Merge pull request #1625 from pallets/release-3.1.0
7b0c47f
release version 3.1.0
ede0f98
Merge pull request #1621 from pallets/template-safe-path
040088a
use posixpath.join
when loading template names
a292075
Merge pull request #1620 from janfilips/patch-1
6e4df02
Fix formatting in tricks.rst
3a050b1
Merge pull request #1617 from pallets/docs-prose
4b63cd8
rewrite include statement section
a98d482
clean up faq, move technical discussions
9de99f8
clean up engine comparisons
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1685/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1362242558,PR_kwDOBm6k_c4-Yqgo,1799,"Update aiofiles requirement from <0.9,>=0.4 to >=0.4,<22.2",49699333,dependabot[bot],closed,0,,,,,1,2022-09-05T16:13:48Z,2022-09-05T18:36:44Z,2022-09-05T18:36:43Z,CONTRIBUTOR,simonw/datasette/pulls/1799,"Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version.
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
----
:books: Documentation preview :books:: https://datasette--1799.org.readthedocs.build/en/1799/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1799/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1362367821,PR_kwDOBm6k_c4-ZGW6,1801,"Don't use upper bound dependencies, refs #1800",9599,simonw,closed,0,,,,,1,2022-09-05T18:29:28Z,2022-09-05T18:35:41Z,2022-09-05T18:35:41Z,OWNER,simonw/datasette/pulls/1801,"See https://iscinumpy.dev/post/bound-version-constraints/
----
:books: Documentation preview :books:: https://datasette--1801.org.readthedocs.build/en/1801/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1801/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
849978964,MDU6SXNzdWU4NDk5Nzg5NjQ=,1293,Show column metadata plus links for foreign keys on arbitrary query results,9599,simonw,open,0,,,,,51,2021-04-04T22:59:42Z,2022-09-02T17:34:09Z,,OWNER,,"Related to #620. It would be _really_ cool if Datasette could magically detect the source of the data displayed in an arbitrary query and, if that data represents a foreign key, display it as a hyperlink.
Compare https://latest.datasette.io/fixtures/facetable
To https://latest.datasette.io/fixtures?sql=select+pk%2C+created%2C+planet_int%2C+on_earth%2C+state%2C+city_id%2C+neighborhood%2C+tags%2C+complex_array%2C+distinct_some_null+from+facetable+order+by+pk+limit+101
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1293/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened
1359557737,I_kwDOBm6k_c5RCTRp,1798,"Parts of YAML file do not work when db name is ""off""",562352,CharlesNepote,closed,0,,,,,4,2022-09-01T22:10:57Z,2022-09-02T00:02:53Z,2022-09-01T23:56:33Z,NONE,,"I guess this issue is not very important and probably rare.
To reproduce:
* create and populate a db named `off.db`
* in the yaml file, add any kind of information below `databases:\n off:`
* the data are not taken into account (because ""off"" is interpreted as ""false"")
YAML file:
```yaml
title: Some title
description_html: |-
This is an experiment.
databases:
off:
tables:
products_from_owners:
title: products_from_owners*
description_html: |-
Description
```
The result for http://xxxx.xxx/-/metadata gives:
```json
{
""title"": ""Some title"",
""description_html"": ""This is an experiment.
"",
""databases"": {
""false"": {
""tables"": {
""products_from_owners"": {
""title"": ""products_from_owners*"",
""description_html"": ""Description
""
}
}
}
}
}
```
=> see the `""false""` instead of `""off""`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1798/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
855476501,MDU6SXNzdWU4NTU0NzY1MDE=,1298,improve table horizontal scroll experience,192568,mroswell,open,0,,,,,4,2021-04-12T01:55:16Z,2022-08-30T21:11:49Z,,CONTRIBUTOR,,"Wide tables aren't a huge problem if you know to click and drag right. But it's not at all obvious to do that. (it also tends to blue-select any content as it's dragging.) Depending on column widths, public users might entirely miss all the columns to the right.
There is a scrollbar at the bottom of the table, but I'm displaying ALL my records because it's the only way for datasette-vega to make accurate charts. So that bottom scrollbar is likely to be missed. I wonder if some sort of javascript-y mouseover to an arrow might help, similar to those seen in image carousels. Ah: here's a perfect example:
1. Visit http://google.com
2. Search for: animals endangered
3. Note the 'g-right-button' (in the code) that looks like a right-facing caret in a circle.
4. Click on that and the carousel scrolls right (and 'g-left-button' appears on the left).
Might be tricky to do that on a table, rather than a one-row carousel, but it's worth experimenting with.
Another option is just to put the scrollbars at the top of the table, too.
Meantime, I'm trying to build a button like the ""View/hide all columns on https://salaries.news.baltimoresun.com/salaries-be494cf/2019+Maryland+state+salaries
Might be nice to have that available by default, with settings in the metadata showing which are on by default.
(I saw some other closed issues related to horizontal scrolling, and admit I don't entirely understand them. For instance, the animated gif at https://github.com/simonw/datasette/issues/998#issuecomment-714117534 confuses me. )
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1298/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1353088849,I_kwDOBm6k_c5Qpn9R,1795,Consider automatically cleaning up curly quotes in searches,9599,simonw,open,0,,,,,0,2022-08-27T16:35:25Z,2022-08-27T16:35:25Z,,OWNER,,"If your phone helpfully adds curly quotes for you then phrase searches against FTS won't work: “Rebecca Sugar”
In regular (not `?_searchmode=raw` search mode Datasette could clean these up for you to help avoid that mistake.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1795/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1348394901,PR_kwDOBm6k_c49qmC2,1792,Test `--load-extension` in GitHub Actions,9599,simonw,closed,0,,,,,3,2022-08-23T18:43:29Z,2022-08-24T00:11:46Z,2022-08-24T00:11:45Z,OWNER,simonw/datasette/pulls/1792,"Refs:
- #1789
----
:books: Documentation preview :books:: https://datasette--1792.org.readthedocs.build/en/1792/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1792/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1344823170,PR_kwDOBm6k_c49e3_k,1789,Add new entrypoint option to `--load-extension`,15178711,asg017,closed,0,,,,,9,2022-08-19T19:27:47Z,2022-08-23T18:42:52Z,2022-08-23T18:34:30Z,CONTRIBUTOR,simonw/datasette/pulls/1789,"Closes #1784
The `--load-extension` flag can now accept an optional ""entrypoint"" value, to specify which entrypoint SQLite should load from the given extension.
```bash
# would load default entrypoint like before
datasette data.db --load-extension ext
# loads the extensions with the ""sqlite3_foo_init"" entrpoint
datasette data.db --load-extension ext:sqlite3_foo_init
# loads the extensions with the ""sqlite3_bar_init"" entrpoint
datasette data.db --load-extension ext:sqlite3_bar_init
```
For testing, I added a small SQLite extension in C at `tests/ext.c`. If compiled, then pytest will run the unit tests in `test_load_extensions.py`to verify that Datasette loads in extensions correctly (and loads the correct entrypoints). Compiling the extension requires a C compiler, I compiled it on my Mac with:
```
gcc ext.c -I path/to/sqlite -fPIC -shared -o ext.dylib
```
Where `path/to/sqlite` is a directory that contains the SQLite amalgamation header files.
Re documentation: I added a bit to the help text for `--load-extension` (which I believe should auto-add to documentation?), and the existing extension documentation is spatialite specific. Let me know if a new extensions documentation page would be helpful!",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1789/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1339663518,I_kwDOBm6k_c5P2aSe,1784,"Include ""entrypoint"" option on `--load-extension`?",15178711,asg017,closed,0,,,,,2,2022-08-16T00:22:57Z,2022-08-23T18:34:31Z,2022-08-23T18:34:31Z,CONTRIBUTOR,,"## Problem
SQLite extensions have the option to define multiple ""entrypoints"" in each loadable extension. For example, the upcoming version of `sqlite-lines` will have 2 entrypoints: the default `sqlite3_lines_init` (which SQLite will automatically guess for) and `sqlite3_lines_noread_init`. The `sqlite3_lines_noread_init` version omits functions that read from the filesystem, which is necessary for security purposes when running untrusted SQL (which Datasette does).
(Similar multiple entrypoints will also be added for sqlite-http).
The `--load-extension` flag, however, doesn't give the option to specify a different entrypoint, so the default one is always used.
## Proposal
I want there to be a new command line option of the `--load-extension` flag to specify a custom entrypoint like so:
```
datasette my.db \
--load-extension ./lines0 sqlite3_lines0_noread_init
```
Then, under the hood, this line of code:
https://github.com/simonw/datasette/blob/7af67b54b7d9bca43e948510fc62f6db2b748fa8/datasette/app.py#L562
Would look something like this:
```python
conn.execute(""SELECT load_extension(?, ?)"", [extension, entrypoint])
```
One potential problem: For backward compatibility, I'm not sure if Click allows cli flags to have variable number of options (""arity""). So I guess it could also use a `:` delimiter like `--static`:
```
datasette my.db \
--load-extension ./lines0:sqlite3_lines0_noread_init
```
Or maybe even a new flag name?
```
datasette my.db \
--load-extension-entrypoint ./lines0 sqlite3_lines0_noread_init
```
Personally I prefer the `:` option... and maybe even `--load-extension` -> `--load`? Definitely out of scope for this issue tho
```
datasette my.db \
--load./lines0:sqlite3_lines0_noread_init
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1784/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1347717749,I_kwDOBm6k_c5QVIp1,1791,Updating metadata.json on Datasette for MacOS,1780782,ment4list,open,0,,,,,1,2022-08-23T10:41:16Z,2022-08-23T13:29:51Z,,NONE,,"I've installed Datasette for Mac as per [the documentation](https://docs.datasette.io/en/stable/installation.html#datasette-desktop-for-mac) and it's working great!
However, I'm not sure how to go about adding something like ""[Canned Queries](https://docs.datasette.io/en/stable/sql_queries.html#canned-queries)"" or utilising other advanced features or settings by manipulating the `metadata.json` or `settings.json` files.
I can view these files from the Datasette App from the top right ""burger"" menu but it only shows the contents of the file with no way to edit or change it.
Am I missing something? Where can I update the `metadata.json` file using the MacOS App?
PS: This is a fantastic tool! Thanks so much for all the effort and especially adding a bunch of different ways to get started quickly!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1791/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1345561209,I_kwDOBm6k_c5QM6J5,1790,A better HTML title for canned query pages,9599,simonw,open,0,,,,,0,2022-08-21T18:27:46Z,2022-08-21T18:27:46Z,,OWNER,,"https://scotrail.datasette.io/scotrail/assemble_sentence?terms=This+train+is+formed+of%2Cbomb+which
Current title is:
`scotrail: with phrases as ( select key, value from json_each('["' || replace(:terms, ',', '","') || '"]')),matches as (select phrases.key, phrases.value, ( select File from announcements where announcements.Transcription like '%' || trim(phrases.value) || '%' order by length(announcements.Transcription) limit 1 ) as Filefrom phrases),results as ( select key, announcements.Transcription, announcements.mp3 from announcements join matches on announcements.File = matches.File order by key)select 'Combined sentence:' as mp3, group_concat(Transcription, ' ') as Transcription, -1 as keyfrom results unionselect mp3, Transcription, keyfrom resultsorder by key`
I think a better title would be:
`scotrail: assemble_sentence, terms = This train is formed of,bomb which`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1790/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1334415381,PR_kwDOBm6k_c488nq6,1778,Use Read the Docs action v1,244656,humitos,closed,0,,,,,0,2022-08-10T10:30:50Z,2022-08-20T00:04:17Z,2022-08-20T00:04:17Z,CONTRIBUTOR,simonw/datasette/pulls/1778,"Read the Docs repository was renamed from `readthedocs/readthedocs-preview` to `readthedocs/actions/`. Now, the `preview` action is under `readthedocs/actions/preview` and is tagged as `v1`
----
:books: Documentation preview :books:: https://datasette--1778.org.readthedocs.build/en/1778/
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1778/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1343732788,I_kwDOBm6k_c5QF7w0,1788,Make it more obvious that Datasette publish can publish multiple databases,9599,simonw,closed,0,,,,,0,2022-08-18T22:57:51Z,2022-08-18T23:06:16Z,2022-08-18T23:06:16Z,OWNER,,Feedback initially for `datasette-publish-fly` but it applies to the others too.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1788/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1343422749,I_kwDOBm6k_c5QEwEd,1787,"Move ""datasette --get"" from Getting Started to CLI Reference",9599,simonw,closed,0,,,,,5,2022-08-18T17:53:39Z,2022-08-18T21:57:09Z,2022-08-18T21:56:21Z,OWNER,,It really shouldn't be here: https://docs.datasette.io/en/0.62/getting_started.html#datasette-get,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1787/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1340900019,I_kwDOBm6k_c5P7IKz,1785,Can't use cog menu to facet by first column in a view,9599,simonw,open,0,,,,,0,2022-08-16T21:27:23Z,2022-08-16T21:27:23Z,,OWNER,,"https://latest.datasette.io/fixtures/paginated_view
Compare with:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1785/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1337541526,I_kwDOBm6k_c5PuUOW,1780,`facet_time_limit_ms` and `sql_time_limit_ms` overlap?,53165,davepeck,open,0,,,,,1,2022-08-12T17:55:37Z,2022-08-15T23:50:08Z,,NONE,,"I needed more than the default 200ms to facet a specific column in a database I was working with, so I ran `datasette` with `--setting facet_time_limit_ms 30000` — definitely overkill!
But it still didn't work; it took a moment to realize I also needed to up my `sql_time_limit_ms` to something larger too.
I'm happy to submit a PR that documents this behavior if it's helpful. Or, if there's a code change we'd like to make (like making sure `sql_time_limit_ms` is always set to the larger of itself and `facet_time_limit_ms`), happy to do that too.
Apologies if I missed this somewhere in the docs. And: thanks. I'm really enjoying the simple, effective tooling datasette gives me out of the box for exploring my databases!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1780/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1339444565,I_kwDOBm6k_c5P1k1V,1783,Better guidance as to what to do after you've installed Datasette,9599,simonw,open,0,,,,,2,2022-08-15T20:11:06Z,2022-08-15T20:14:01Z,,OWNER,,"Feedback [from Discord](https://discord.com/channels/823971286308356157/823971286941302908/1008822978793984060):
> hello, love the project and came for help and to point out a possible gap in the docs. starting with ""getting started"" and ""installation"" every thing looks great, but then there's a giant leap after you have it installed and running. from the user perspective of ""i have a csv of set of csvs that i want to turn into a table(s), what do i do next?"" --- so something like maybe a page for creating your first project should go after ""installation"".
- https://docs.datasette.io/en/0.62/getting_started.html
- https://docs.datasette.io/en/0.62/installation.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1783/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1084193403,PR_kwDOBm6k_c4wDKmb,1574,introduce new option for datasette package to use a slim base image,33631,fs111,closed,0,,,,,6,2021-12-19T21:18:19Z,2022-08-15T08:49:31Z,2022-08-15T08:49:31Z,NONE,simonw/datasette/pulls/1574,"The official python images on docker hub come with a slim variant that is significantly smaller than the default. The diff does not change the default, but allows to switch to the `slim` variant with commandline switch (`--slim-base-image`)
Size comparison:
```
$ datasette package some.db -t fat --install ""datasette-basemap datasette-cluster-map""
$ datasette package some.db -t slim --slim-base-image --install ""datasette-basemap datasette-cluster-map""
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
fat latest 807b393ace0d 9 seconds ago 978MB
slim latest 31bc5e63505c 8 minutes ago 191MB
```",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1574/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1334628400,I_kwDOBm6k_c5PjNAw,1779,google cloudrun updated their limits on maxscale based on memory and cpu count,536941,fgregg,closed,0,,,8303187,Datasette 0.62,13,2022-08-10T13:27:21Z,2022-08-14T19:42:59Z,2022-08-14T17:07:34Z,CONTRIBUTOR,,"if you don't set an explicit limit on container scaling, then [google defaults to 100](https://cloud.google.com/run/docs/configuring/max-instances#limits)
google recently updated the [limits on container scaling](https://cloud.google.com/run/docs/configuring/max-instances#limits), such that if you set up datasette to use more memory or cpu, then you need to set the maxScale argument much smaller than 100.
would be nice if `datasette publish` could do this math for you and set the right maxScale.
[Log of an failing publish run](https://github.com/labordata/warehouse/runs/7764725972?check_suite_focus=true#step:8:332).
```
ERROR: (gcloud.run.deploy) spec.template.spec.containers[0].resources.limits.cpu: Invalid value specified for cpu. For the specified value, maxScale may not exceed 15.
Consider running your workload in a region with greater capacity, decreasing your requested cpu-per-instance, or requesting an increase in quota for this region if you are seeing sustained usage near this limit, see https://cloud.google.com/run/quotas. Your project may gain access to further scaling by adding billing information to your account.
Traceback (most recent call last):
File ""/home/runner/.local/bin/datasette"", line 8, in
sys.exit(cli())
File ""/home/runner/.local/lib/python3.8/site-packages/click/core.py"", line 1128, in __call__
return self.main(*args, **kwargs)
File ""/home/runner/.local/lib/python3.8/site-packages/click/core.py"", line 1053, in main
rv = self.invoke(ctx)
File ""/home/runner/.local/lib/python3.8/site-packages/click/core.py"", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/home/runner/.local/lib/python3.8/site-packages/click/core.py"", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/home/runner/.local/lib/python3.8/site-packages/click/core.py"", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/home/runner/.local/lib/python3.8/site-packages/click/core.py"", line 754, in invoke
return __callback(*args, **kwargs)
File ""/home/runner/.local/lib/python3.8/site-packages/datasette/publish/cloudrun.py"", line 160, in cloudrun
check_call(
File ""/usr/lib/python3.8/subprocess.py"", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/labordata/datasette warehouse --memory 8Gi --cpu 2' returned non-zero exit status 1.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1779/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1338278056,I_kwDOBm6k_c5PxICo,1782,Release notes for Datasette 0.62,9599,simonw,closed,0,,,8303187,Datasette 0.62,2,2022-08-14T15:26:45Z,2022-08-14T17:40:45Z,2022-08-14T17:32:54Z,OWNER,,"I've written a lot of these already for the alphas:
- https://github.com/simonw/datasette/releases/tag/0.62a0
- https://github.com/simonw/datasette/releases/tag/0.62a1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1782/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1218133366,I_kwDOBm6k_c5Imz12,1728,Writable canned queries fail with useless non-error against immutable databases,127565,wragge,closed,0,,,8303187,Datasette 0.62,13,2022-04-28T03:10:34Z,2022-08-14T16:34:40Z,2022-08-14T16:34:40Z,CONTRIBUTOR,,"I've been banging my head against a wall for a while and would appreciate any pointers...
- I have a writeable canned query to update rows in the db.
- I'm using the github-oauth plugin for authentication.
- I have `allow` set on the query to accept my GitHub id and a GH organisation.
- Authentication seems to work as expected both locally and on Cloudrun -- viewing `/-/actor` gives the same result in both environments
- I can access the 'padlocked' canned query in both environments.
Everything seems to be the same, but the canned query works perfectly when run locally, and fails when I try it on Cloudrun. I'm redirected back to the canned query page and the db is not changed. There's nothing in the Cloudstor logs to indicate an error.
Any clues as to where I should be looking for the problem?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1728/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223527226,I_kwDOBm6k_c5I7Ys6,1738,"""Cannot use _sort and _sort_desc at the same time""",9599,simonw,closed,0,,,8303187,Datasette 0.62,2,2022-05-03T01:06:24Z,2022-08-14T16:13:55Z,2022-08-14T16:13:55Z,OWNER,,"Triggered this error while playing with the sort desc checkbox and the apply button that are only visible on this page at mobile screen width:
https://latest.datasette.io/fixtures/compound_three_primary_keys?_sort_desc=pk1
Navigate to that page (with the browser narrow enough to show the box), un-check the box and click Apply:
![sort-bug](https://user-images.githubusercontent.com/9599/166390804-cb289b29-63dc-4986-b7f9-81cf2ae04914.gif)
Also notable: I managed to get to a page with `?_sort_desk=pk1` in the URL three times by clicking around with that button.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1738/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1318907685,I_kwDOBm6k_c5OnO8l,1773,500 error if sorted by a column not in the ?_col= list,9599,simonw,closed,0,,,8303187,Datasette 0.62,4,2022-07-27T01:20:27Z,2022-08-14T16:06:25Z,2022-08-14T15:44:05Z,OWNER,,"For example: https://latest.datasette.io/fixtures/sortable?_sort_desc=sortable&_col=sortable_with_nulls
That's `?_sort_desc=sortable&_col=sortable_with_nulls`
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1773/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1296222572,I_kwDOBm6k_c5NQsls,1768,Upgrade to 3.10.6-slim-bullseye Docker base image,9599,simonw,closed,0,,,8303187,Datasette 0.62,5,2022-07-06T18:37:49Z,2022-08-14T15:54:36Z,2022-08-14T15:54:11Z,OWNER,,For the package published to Docker Hub and also the containers used by `datasette package` and `datasette publish cloudrun`.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1768/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1306492437,I_kwDOBm6k_c5N334V,1770,`handle_exception` plugin hook for custom error handling,9599,simonw,closed,0,,,8303187,Datasette 0.62,14,2022-07-15T20:52:49Z,2022-08-14T15:25:51Z,2022-08-14T15:25:51Z,OWNER,,"I need this for a couple of plugins, both of which are broken at the moment:
- https://github.com/simonw/datasette-sentry/issues/1
- https://github.com/simonw/datasette-show-errors/issues/2",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1770/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1338137350,I_kwDOBm6k_c5PwlsG,1781,Ensure Datasette Lite is promoted in docs and README,9599,simonw,closed,0,,,8303187,Datasette 0.62,1,2022-08-14T05:12:35Z,2022-08-14T15:24:40Z,2022-08-14T15:24:40Z,OWNER,,As of 0.62 https://lite.datasette.io is a supported piece of the overall Datasette ecosystem.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1781/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1323332006,I_kwDOBm6k_c5O4HGm,1774,Request of feature for mongo,428820,johnfelipe,open,0,,,,,0,2022-07-31T01:00:05Z,2022-07-31T01:00:05Z,,NONE,,Will love if can we use datasette for mongo and all pipelines and workflows,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1774/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
838245338,MDU6SXNzdWU4MzgyNDUzMzg=,1272,Unit tests for the Dockerfile,9599,simonw,open,0,,,,,3,2021-03-23T01:36:29Z,2022-07-29T10:22:59Z,,OWNER,,"Working on the Dockerfile in #1249 made me wish for automated tests - to confirm that it boots up correctly, can run SpatiaLite and doesn't have weird bugs like the `/db` page hanging.
These could run in CI too, but maybe only if the `Dockerfile` is updated.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1272/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
779156520,MDU6SXNzdWU3NzkxNTY1MjA=,1175,Use structlog for logging,9599,simonw,open,0,,,,,4,2021-01-05T15:11:36Z,2022-07-26T12:52:10Z,,OWNER,,To solve #241 JSON logging.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1175/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
663145122,MDU6SXNzdWU2NjMxNDUxMjI=,903,Add temporary plugin testing pattern to the testing docs,9599,simonw,closed,0,,,,,1,2020-07-21T16:22:34Z,2022-07-18T21:34:33Z,2022-07-18T21:31:22Z,OWNER,,"https://til.simonwillison.net/pytest/registering-plugins-in-tests
Would be useful to include this pattern on https://datasette.readthedocs.io/en/stable/testing_plugins.html",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/903/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1307359454,PR_kwDOBm6k_c47iWbd,1772,Convert to setup.cfg,89725,kfdm,open,0,,,,,0,2022-07-18T03:39:53Z,2022-07-18T03:39:53Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1772,"Recent versions of setuptools can run most things from setup.cfg so one can have a simpler version that does not require executing code on install.
The bulk of the changes were automated by running https://pypi.org/project/setup-py-upgrade/ with a few minor edits for the bits that it can not auto convert (the initial `get_long_description()` and `get_version()` can not be automatically converted)",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1772/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1292368833,I_kwDOBm6k_c5NB_vB,1764,Keep track of config_dir in directory mode (for plugins),25778,eyeseast,closed,0,,,,,0,2022-07-03T16:57:49Z,2022-07-18T01:12:45Z,2022-07-18T01:12:45Z,CONTRIBUTOR,,"I started working on using `config_dir` with my [datasette-query-files plugin](https://github.com/eyeseast/datasette-query-files) and realized Datasette doesn't actually hold onto the `config_dir` argument. It gets used in `__init__` but then forgotten. It would be nice to be able to use it in plugins, though.
Here's the reference issue: https://github.com/eyeseast/datasette-query-files/issues/4
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1764/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1292377561,PR_kwDOBm6k_c46wdOW,1766,Keep track of config_dir,25778,eyeseast,closed,0,,,,,2,2022-07-03T17:37:02Z,2022-07-18T01:12:45Z,2022-07-18T01:12:45Z,CONTRIBUTOR,simonw/datasette/pulls/1766,"Closes #1764
Small change that adds `self.config_dir = config_dir` to `Datasette.__init__`. This will let plugins also use `config_dir`, if available.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1766/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1306020162,PR_kwDOBm6k_c47eFtx,1769,"Update pytest-asyncio requirement from <0.19,>=0.17 to >=0.17,<0.20",49699333,dependabot[bot],closed,0,,,,,1,2022-07-15T13:10:15Z,2022-07-18T01:06:38Z,2022-07-18T01:06:38Z,CONTRIBUTOR,simonw/datasette/pulls/1769,"Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
Release notes
Sourced from pytest-asyncio's releases .
pytest-asyncio 0.19.0
title: 'pytest-asyncio: pytest support for asyncio'
pytest-asyncio is an Apache2 licensed library, written in Python, for
testing asyncio code with pytest.
asyncio code is usually written in the form of coroutines, which makes
it slightly more difficult to test using normal testing tools.
pytest-asyncio provides useful fixtures and markers to make testing
easier.
@pytest.mark.asyncio
async def test_some_asyncio_code():
res = await library.do_something()
assert b"expected result" == res
pytest-asyncio has been strongly influenced by
pytest-tornado .
Features
fixtures for creating and injecting versions of the asyncio event
loop
fixtures for injecting unused tcp/udp ports
pytest markers for treating tests as asyncio coroutines
easy testing with non-default event loops
support for [async def]{.title-ref} fixtures and async generator
fixtures
support auto mode to handle all async fixtures and tests
automatically by asyncio; provide strict mode if a test suite
should work with different async frameworks simultaneously, e.g.
asyncio
and trio
.
Installation
... (truncated)
Changelog
Sourced from pytest-asyncio's changelog .
0.19.0 (22-07-13)
BREAKING: The default asyncio_mode
is now strict . [#293](https://github.com/pytest-dev/pytest-asyncio/issues/293) <https://github.com/pytest-dev/pytest-asyncio/issues/293>
_
Removes setup.py
since all relevant configuration is present setup.cfg
. Users requiring an editable installation of pytest-asyncio need to use pip v21.1 or newer. [#283](https://github.com/pytest-dev/pytest-asyncio/issues/283) <https://github.com/pytest-dev/pytest-asyncio/issues/283>
_
Declare support for Python 3.11.
0.18.3 (22-03-25)
Adds pytest-trio <https://pypi.org/project/pytest-trio/>
_ to the test dependencies
Fixes a bug that caused pytest-asyncio to try to set up async pytest_trio fixtures in strict mode. [#298](https://github.com/pytest-dev/pytest-asyncio/issues/298) <https://github.com/pytest-dev/pytest-asyncio/issues/298>
_
0.18.2 (22-03-03)
Fix asyncio auto mode not marking static methods. [#295](https://github.com/pytest-dev/pytest-asyncio/issues/295) <https://github.com/pytest-dev/pytest-asyncio/issues/295>
_
Fix a compatibility issue with Hypothesis 6.39.0. [#302](https://github.com/pytest-dev/pytest-asyncio/issues/302) <https://github.com/pytest-dev/pytest-asyncio/issues/302>
_
0.18.1 (22-02-10)
Fixes a regression that prevented async fixtures from working in synchronous tests. [#286](https://github.com/pytest-dev/pytest-asyncio/issues/286) <https://github.com/pytest-dev/pytest-asyncio/issues/286>
_
0.18.0 (22-02-07)
Raise a warning if @pytest
.mark.asyncio is applied to non-async function. [#275](https://github.com/pytest-dev/pytest-asyncio/issues/275) <https://github.com/pytest-dev/pytest-asyncio/issues/275>
_
Support parametrized event_loop
fixture. [#278](https://github.com/pytest-dev/pytest-asyncio/issues/278) <https://github.com/pytest-dev/pytest-asyncio/issues/278>
_
0.17.2 (22-01-17)
Require typing-extensions
on Python`_
Fix a regression in tests collection introduced by 0.17.1, the plugin works fine with non-python tests again. [#267](https://github.com/pytest-dev/pytest-asyncio/issues/267) <https://github.com/pytest-dev/pytest-asyncio/issues/267>
_
0.17.1 (22-01-16)
Fixes a bug that prevents async Hypothesis tests from working without explicit asyncio
marker when --asyncio-mode=auto
is set. [#258](https://github.com/pytest-dev/pytest-asyncio/issues/258) <https://github.com/pytest-dev/pytest-asyncio/issues/258>
_
Fixed a bug that closes the default event loop if the loop doesn't exist [#257](https://github.com/pytest-dev/pytest-asyncio/issues/257) <https://github.com/pytest-dev/pytest-asyncio/issues/257>
_
Added type annotations. [#198](https://github.com/pytest-dev/pytest-asyncio/issues/198) <https://github.com/pytest-dev/pytest-asyncio/issues/198>
_
Show asyncio mode in pytest report headers. [#266](https://github.com/pytest-dev/pytest-asyncio/issues/266) <https://github.com/pytest-dev/pytest-asyncio/issues/266>
_
Relax asyncio_mode
type definition; it allows to support pytest 6.1+. [#262](https://github.com/pytest-dev/pytest-asyncio/issues/262) <https://github.com/pytest-dev/pytest-asyncio/issues/262>
_
0.17.0 (22-01-13)
pytest-asyncio
no longer alters existing event loop policies. [#168](https://github.com/pytest-dev/pytest-asyncio/issues/168) <https://github.com/pytest-dev/pytest-asyncio/issues/168>
, [#188](https://github.com/pytest-dev/pytest-asyncio/issues/188) <https://github.com/pytest-dev/pytest-asyncio/issues/168>
Drop support for Python 3.6
Fixed an issue when pytest-asyncio was used in combination with flaky
or inherited asynchronous Hypothesis tests. [#178](https://github.com/pytest-dev/pytest-asyncio/issues/178) <https://github.com/pytest-dev/pytest-asyncio/issues/178>
_ [#231](https://github.com/pytest-dev/pytest-asyncio/issues/231) <https://github.com/pytest-dev/pytest-asyncio/issues/231>
_
Added flaky <https://pypi.org/project/flaky/>
_ to test dependencies
Added unused_udp_port
and unused_udp_port_factory
fixtures (similar to unused_tcp_port
and unused_tcp_port_factory
counterparts. [#99](https://github.com/pytest-dev/pytest-asyncio/issues/99) <https://github.com/pytest-dev/pytest-asyncio/issues/99>
_
Added the plugin modes: strict , auto , and legacy . See documentation <https://github.com/pytest-dev/pytest-asyncio#modes>
_ for details. [#125](https://github.com/pytest-dev/pytest-asyncio/issues/125) <https://github.com/pytest-dev/pytest-asyncio/issues/125>
_
Correctly process KeyboardInterrupt
during async fixture setup phase [#219](https://github.com/pytest-dev/pytest-asyncio/issues/219) <https://github.com/pytest-dev/pytest-asyncio/issues/219>
_
... (truncated)
Commits
2da33c4
docs: Prepare v0.19.0 release. (#385 )
07beb80
opt into strict mode by default (#380 )
25c54a5
Clarify documentation of event_loop fixture (#375 )
49f07a4
Bump typing-extensions from 4.2.0 to 4.3.0 in /dependencies/default (#382 )
739198b
Bump hypothesis from 6.48.0 to 6.48.3 in /dependencies/default (#381 )
db72f25
Bump importlib-metadata from 4.11.4 to 4.12.0 in /dependencies/default (#378 )
4cf16cf
Bump hypothesis from 6.47.3 to 6.48.0 in /dependencies/default (#377 )
f13c85f
docs: Fix typo in README.
b463f72
Python 3.11 support (#370 )
860ff51
Bump hypothesis from 6.47.2 to 6.47.3 in /dependencies/default (#373 )
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1769/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1280136357,PR_kwDOBm6k_c46Hsvj,1760,Bump furo from 2022.4.7 to 2022.6.21,49699333,dependabot[bot],closed,0,,,,,1,2022-06-22T13:22:31Z,2022-07-18T01:06:27Z,2022-07-18T01:06:27Z,CONTRIBUTOR,simonw/datasette/pulls/1760,"Bumps [furo](https://github.com/pradyunsg/furo) from 2022.4.7 to 2022.6.21.
Changelog
Sourced from furo's changelog .
Changelog
2022.06.21 -- Opulent Opal
Fix docutils <= 0.17.x
compatibility
Bump to the latest Node.js LTS
2022.06.04.1 -- Naughty Nickel bugfix
Fix the URL used in the "Edit this page" for Read the Docs builds.
2022.06.04 -- Naughty Nickel
✨ Advertise Sphinx 5 compatibility.
✨ Change to basic-ng
as the base theme (from {pypi}sphinx-basic-ng
).
Document site-wide announcement banners.
Drop the pin on pygments.
Improve edit button, using basic-ng
's edit-this-page
component.
Tweak headings to better match what users expect.
Tweak how Sphinx's default HTML is rendered, using docutils post-transforms (this replaces parsing+modifying it with BeautifulSoup).
When built with docutils 0.18, footnotes are rendered differently and stylised differently in Furo.
2022.04.07 -- Magical Mauve
✨ Make sphinx-copybutton look better.
Add margin to indentations in line blocks.
Add styling for non-arabic list styles
Add support for html_baseurl
.
Improve "Edit this page" icon to be more accessible.
Improve html_sidebars
example.
Tweak positioning of back to top on desktop.
2022.03.04 -- Lucent Lilac
Improve support for print media.
Reduce heading sizes for h3 and below.
Don't allow selecting headerlink content.
Improve how overflow wrapping is handled.
Add a reference from the configuration variables to the color customisation page.
2022.02.23 -- Keen Kobi
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.4.7&new-version=2022.6.21)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1760/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
728905098,MDU6SXNzdWU3Mjg5MDUwOTg=,1048,Documentation and unit tests for urls.row() urls.row_blob() methods,9599,simonw,open,0,,,,,7,2020-10-25T00:13:53Z,2022-07-10T16:23:57Z,,OWNER,,https://github.com/simonw/datasette/blob/5db7ae3ce165ded57c7fb1cfbdb3258b1cf06c10/datasette/app.py#L1307-L1313,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1048/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
860625833,MDU6SXNzdWU4NjA2MjU4MzM=,1300,Make row available to `render_cell` plugin hook,3243482,abdusco,closed,0,,,,,5,2021-04-18T10:14:37Z,2022-07-07T16:34:05Z,2022-07-07T16:31:22Z,CONTRIBUTOR,,"*Original title: **Generating URL for a row inside `render_cell` hook***
Hey,
I am using Datasette to view a database that contains video metadata. It has BLOB columns that contain video thumbnails in JPG format (around 100-500KB per row).
I've registered an output formatter that extends `datasette.blob_renderer.render_blob` function and serves the column with `image/jpeg` content type.
```python
from datasette.blob_renderer import render_blob
async def render_jpg(datasette, database, rows, columns, request, table, view_name):
response = await render_blob(datasette, database, rows, columns, request, table, view_name)
response.content_type = ""image/jpeg""
response.headers[""Content-Disposition""] = f'inline; filename=""image.jpg""'
return response
@hookimpl
def register_output_renderer():
return {
""extension"": ""jpg"",
""render"": render_jpg,
""can_render"": lambda: True,
}
```
This works well. I can visit `http://localhost:8001/mydb/videos/1.jpg?_blob_column=thumbnail` and view the image.
I want to display the image directly with an ` ` tag (lazy-loaded of course). So, I need a URL, because embedding base64 would increase the page size too much (each image > 100KB).
Datasette generates a link with `.blob` extension for blob columns. It does this by calling `datasette.urls.row_blob`
https://github.com/simonw/datasette/blob/7a2ed9f8a119e220b66d67c7b9e07cbab47b1196/datasette/views/table.py#L169-L179
But I have no way of getting the row inside the `render_cell` hook.
```python
@hookimpl
def render_cell(value, column, table, database, datasette):
if isinstance(value, bytes) and imghdr.what(None, value):
# generate url
return '$renderedLink'
```
Any pointers?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1300/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1294641696,I_kwDOBm6k_c5NKqog,1767,Ability to set a custom favicon,9599,simonw,open,0,,,,,9,2022-07-05T18:41:12Z,2022-07-05T18:56:43Z,,OWNER,,"If you're running a website on Datasette, like https://www.niche-museums.com/ or https://til.simonwillison.net/ - you should have the ability to easily specify a custom favicon.
Currently the `/favicon.ico` view is hard-coded to do this: https://github.com/simonw/datasette/blob/9f1eb0d4eac483b953392157bd9fd6cc4df37de7/datasette/app.py#L179-L188",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1767/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1203943272,I_kwDOBm6k_c5Hwrdo,1713,Datasette feature for publishing snapshots of query results,9599,simonw,open,0,,,,,5,2022-04-14T01:42:00Z,2022-07-04T05:16:35Z,,OWNER,,"https://twitter.com/simonw/status/1514392335718645760
> Maybe [@datasetteproj](https://twitter.com/datasetteproj) should grow a feature that lets you cache the results of a query and give that snapshot a stable permalink
>
> A plugin that publishes the JSON output of a query to an S3 bucket would be pretty neat... especially if it could also be configured to re-publish the results on a schedule
A lot of people said they would find this useful.
Probably going to build this as a plugin.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1713/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1287325944,PR_kwDOBm6k_c46ftHo,1763,Bump black from 22.1.0 to 22.6.0,49699333,dependabot[bot],closed,0,,,,,1,2022-06-28T13:11:32Z,2022-06-28T17:40:25Z,2022-06-28T17:40:25Z,CONTRIBUTOR,simonw/datasette/pulls/1763,"Bumps [black](https://github.com/psf/black) from 22.1.0 to 22.6.0.
Release notes
Sourced from black's releases .
22.6.0
Style
Fix unstable formatting involving #fmt: skip
and # fmt:skip
comments (notice the lack of spaces) (#2970 )
Preview style
Docstring quotes are no longer moved if it would violate the line length limit (#3044 )
Parentheses around return annotations are now managed (#2990 )
Remove unnecessary parentheses around awaited objects (#2991 )
Remove unnecessary parentheses in with
statements (#2926 )
Remove trailing newlines after code block open (#3035 )
Integrations
Add scripts/migrate-black.py
script to ease introduction of Black to a Git project (#3038 )
Output
Output Python version and implementation as part of --version
flag (#2997 )
Packaging
Use tomli
instead of tomllib
on Python 3.11 builds where tomllib
is not available (#2987 )
Parser
PEP 654 syntax (for example, except *ExceptionGroup:
) is now supported (#3016 )
PEP 646 syntax (for example, Array[Batch, *Shape]
or def fn(*args: *T) -> None
) is now supported (#3071 )
Vim Plugin
Fix strtobool
function. It didn't parse true/on/false/off. (#3025 )
Full Changelog : https://github.com/psf/black/compare/22.3.0...22.6.0
Thank you!
@jpy-git
for improving our parentheses formatting significantly
@siuryan
for fixing a fmt: skip bug, making it a little less annoying to use :)
@isidentical
for implementing support for PEP 654 and 646 syntax
@defntvdm
for fixing our vim plugin, especially as we (the maintainers) don't really know vim script sadly
@idorrington92
for fixing the docstring bug where Black would move the closing quotes causing it to violate the line length limit (whoops!)
@hbrunn
for contributing the migrate-black script
@saroad2
for improving newline handling after code blocks and test infrastructure improvements
... and everyone else who contributed documentation, tests, or other improvements to the Black project!
... (truncated)
Changelog
Sourced from black's changelog .
22.6.0
Style
Fix unstable formatting involving #fmt: skip
and # fmt:skip
comments (notice the
lack of spaces) (#2970 )
Preview style
Docstring quotes are no longer moved if it would violate the line length limit (#3044 )
Parentheses around return annotations are now managed (#2990 )
Remove unnecessary parentheses around awaited objects (#2991 )
Remove unnecessary parentheses in with
statements (#2926 )
Remove trailing newlines after code block open (#3035 )
Integrations
Add scripts/migrate-black.py
script to ease introduction of Black to a Git project
(#3038 )
Output
Output Python version and implementation as part of --version
flag (#2997 )
Packaging
Use tomli
instead of tomllib
on Python 3.11 builds where tomllib
is not
available (#2987 )
Parser
PEP 654 syntax (for example,
except *ExceptionGroup:
) is now supported (#3016 )
PEP 646 syntax (for example,
Array[Batch, *Shape]
or def fn(*args: *T) -> None
) is now supported (#3071 )
Vim Plugin
Fix strtobool
function. It didn't parse true/on/false/off. (#3025 )
22.3.0
Preview style
Code cell separators #%%
are now standardised to # %%
(#2919 )
Remove unnecessary parentheses from except
statements (#2939 )
Remove unnecessary parentheses from tuple unpacking in for
loops (#2945 )
Avoid magic-trailing-comma in single-element subscripts (#2942 )
Configuration
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.1.0&new-version=22.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1763/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1184850337,PR_kwDOBm6k_c41OrSL,1693,Bump black from 22.1.0 to 22.3.0,49699333,dependabot[bot],closed,0,,,,,3,2022-03-29T13:11:09Z,2022-06-28T13:11:38Z,2022-06-28T13:11:36Z,CONTRIBUTOR,simonw/datasette/pulls/1693,"Bumps [black](https://github.com/psf/black) from 22.1.0 to 22.3.0.
Release notes
Sourced from black's releases .
22.3.0
Preview style
Code cell separators #%%
are now standardised to # %%
(#2919 )
Remove unnecessary parentheses from except
statements (#2939 )
Remove unnecessary parentheses from tuple unpacking in for
loops (#2945 )
Avoid magic-trailing-comma in single-element subscripts (#2942 )
Configuration
Do not format __pypackages__
directories by default (#2836 )
Add support for specifying stable version with --required-version
(#2832 ).
Avoid crashing when the user has no homedir (#2814 )
Avoid crashing when md5 is not available (#2905 )
Fix handling of directory junctions on Windows (#2904 )
Documentation
Update pylint config documentation (#2931 )
Integrations
Move test to disable plugin in Vim/Neovim, which speeds up loading (#2896 )
Output
In verbose, mode, log when Black is using user-level config (#2861 )
Packaging
Fix Black to work with Click 8.1.0 (#2966 )
On Python 3.11 and newer, use the standard library's tomllib
instead of tomli
(#2903 )
black-primer
, the deprecated internal devtool, has been removed and copied to a
separate repository (#2924 )
Parser
Black can now parse starred expressions in the target of for
and async for
statements, e.g for item in *items_1, *items_2: pass
(#2879 ).
Changelog
Sourced from black's changelog .
22.3.0
Preview style
Code cell separators #%%
are now standardised to # %%
(#2919 )
Remove unnecessary parentheses from except
statements (#2939 )
Remove unnecessary parentheses from tuple unpacking in for
loops (#2945 )
Avoid magic-trailing-comma in single-element subscripts (#2942 )
Configuration
Do not format __pypackages__
directories by default (#2836 )
Add support for specifying stable version with --required-version
(#2832 ).
Avoid crashing when the user has no homedir (#2814 )
Avoid crashing when md5 is not available (#2905 )
Fix handling of directory junctions on Windows (#2904 )
Documentation
Update pylint config documentation (#2931 )
Integrations
Move test to disable plugin in Vim/Neovim, which speeds up loading (#2896 )
Output
In verbose, mode, log when Black is using user-level config (#2861 )
Packaging
Fix Black to work with Click 8.1.0 (#2966 )
On Python 3.11 and newer, use the standard library's tomllib
instead of tomli
(#2903 )
black-primer
, the deprecated internal devtool, has been removed and copied to a
separate repository (#2924 )
Parser
Black can now parse starred expressions in the target of for
and async for
statements, e.g for item in *items_1, *items_2: pass
(#2879 ).
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=22.1.0&new-version=22.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1693/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1280799259,I_kwDOBm6k_c5MV3Ib,1761,ensure_ascii=False,1473102,mustafa0x,open,0,,,,,0,2022-06-22T19:58:13Z,2022-06-22T19:58:30Z,,NONE,,"Hi, thanks for the project!
For the JSON output, I would consider defaulting to `ensure_ascii=False` (UTF-8 seems pretty universal) or making it an option. When dealing with non-Latin text, `ensure_ascii=True` (the default) can triple the size of the output.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1761/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1261826957,PR_kwDOBm6k_c45Kojn,1753,Bump furo from 2022.4.7 to 2022.6.4.1,49699333,dependabot[bot],closed,0,,,,,2,2022-06-06T13:10:22Z,2022-06-22T13:22:37Z,2022-06-22T13:22:35Z,CONTRIBUTOR,simonw/datasette/pulls/1753,"Bumps [furo](https://github.com/pradyunsg/furo) from 2022.4.7 to 2022.6.4.1.
Changelog
Sourced from furo's changelog .
Changelog
2022.06.04.1 -- Naughty Nickel bugfix
Fix the URL used in the "Edit this page" for Read the Docs builds.
2022.06.04 -- Naughty Nickel
✨ Advertise Sphinx 5 compatibility.
✨ Change to basic-ng
as the base theme (from {pypi}sphinx-basic-ng
).
Document site-wide announcement banners.
Drop the pin on pygments.
Improve edit button, using basic-ng
's edit-this-page
component.
Tweak headings to better match what users expect.
Tweak how Sphinx's default HTML is rendered, using docutils post-transforms (this replaces parsing+modifying it with BeautifulSoup).
When built with docutils 0.18, footnotes are rendered differently and stylised differently in Furo.
2022.04.07 -- Magical Mauve
✨ Make sphinx-copybutton look better.
Add margin to indentations in line blocks.
Add styling for non-arabic list styles
Add support for html_baseurl
.
Improve "Edit this page" icon to be more accessible.
Improve html_sidebars
example.
Tweak positioning of back to top on desktop.
2022.03.04 -- Lucent Lilac
Improve support for print media.
Reduce heading sizes for h3 and below.
Don't allow selecting headerlink content.
Improve how overflow wrapping is handled.
Add a reference from the configuration variables to the color customisation page.
2022.02.23 -- Keen Kobi
✨ Add a "Back to Top" button that shows up when scrolling up.
Add a URL to GitHub in Project-URLs.
Break long words in the prev/next buttons.
Fix includes in Kitchen sink.
... (truncated)
Commits
1142fad
Prepare release: 2022.06.04.1
211abb4
Update changelog
06cdba6
Fix the edit this page URL
43ce491
Back to development
fb6e486
Prepare release: 2022.06.04
090b02e
Update changelog
098d51d
Fix the Just the Docs link
7fa8d08
Change to a post-transform for wrapping math blocks and table
51f1e52
Speed up determining if there's multiple toc entries
99a6ff8
Update caniuse-lite NPM package
Additional commits viewable in compare view
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=furo&package-manager=pip&previous-version=2022.4.7&new-version=2022.6.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1753/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1275523220,PR_kwDOBm6k_c454SlE,1759,Extract facet portions of table.html out into included templates,19872,nsmgr8,closed,0,,,,,3,2022-06-17T22:04:04Z,2022-06-20T18:05:45Z,2022-06-20T18:05:45Z,CONTRIBUTOR,simonw/datasette/pulls/1759,To allow users customise the facet content as they would prefer such as sorting of facet results. ordering of suggested facets etc.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1759/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1060631257,I_kwDOBm6k_c4_N_LZ,1528,"Add new `""sql_file""` key to Canned Queries in metadata?",15178711,asg017,open,0,,,,,3,2021-11-22T21:58:01Z,2022-06-10T03:23:08Z,,CONTRIBUTOR,,"Currently for canned queries, you have to inline SQL in your `metadata.yaml` like so:
```yaml
databases:
fixtures:
queries:
neighborhood_search:
sql: |-
select neighborhood, facet_cities.name, state
from facetable
join facet_cities on facetable.city_id = facet_cities.id
where neighborhood like '%' || :text || '%'
order by neighborhood
title: Search neighborhoods
```
This works fine, but for a few reasons, I usually have my canned queries already written in separate `.sql` files. I'd like to instead re-use those instead of re-writing it.
So, I'd like to see a new `""sql_file""` key that works like so:
`metadata.yaml`:
```yaml
databases:
fixtures:
queries:
neighborhood_search:
sql_file: neighborhood_search.sql
title: Search neighborhoods
```
`neighborhood_search.sql`:
```sql
select neighborhood, facet_cities.name, state
from facetable
join facet_cities on facetable.city_id = facet_cities.id
where neighborhood like '%' || :text || '%'
order by neighborhood
```
Both of these would work in the exact same way, where Datasette would instead open + include `neighborhood_search.sql` on startup.
A few reasons why I'd like to keep my canned queries SQL separate from metadata.yaml:
- Keeping SQL in standalone SQL files means syntax highlighting and other text editor integrations in my code
- Multiline strings in yaml, while functional, are a tad cumbersome and are hard to edit
- Works well with other tools (can pipe `.sql` files into the `sqlite3` CLI, or use with other SQLite clients easier)
- Typically my canned queries are quite long compared to everything else in my metadata.yaml, so I'd love to separate it where possible
Let me know if this is a feature you'd like to see, I can try to send up a PR if this sounds right!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1528/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1266329095,I_kwDOBm6k_c5LeqYH,1756,Mechanism for creating databases in WAL mode,9599,simonw,open,0,,,,,0,2022-06-09T15:39:28Z,2022-06-09T15:39:28Z,,OWNER,,"The `--create` option currently creates databases if they are missing, but does not enable WAL mode for them.
It turns out WAL mode is useful for databases that are accepting writes!
I think a `--create-wal` option that both creates them AND sets WAL mode on any that are created would be a good idea.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1756/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1266207143,I_kwDOBm6k_c5LeMmn,1755,Gunicorn,1176293,ar-jan,open,0,,,,,0,2022-06-09T14:18:46Z,2022-06-09T14:18:46Z,,NONE,,"I've read issue #514 which resulted in running Datasette via systemd as recommended approach. We've also adopted this (for now), but I notice that Uvicorn [says the following](https://www.uvicorn.org/#running-with-gunicorn):
> Uvicorn includes a Gunicorn worker class allowing you to run ASGI applications, with all of Uvicorn's performance benefits, while also giving you Gunicorn's fully-featured process management.
>
> This allows you to increase or decrease the number of worker processes on the fly, restart worker processes gracefully, or perform server upgrades without downtime.
>
> For production deployments we recommend using gunicorn with the uvicorn worker class.
We usually deploy Python applications via Gunicorn for these process management features (e.g. `--daemon` and `--pid`). Is this something that would/could work with Datasette as well?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1755/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1226106354,PR_kwDOBm6k_c43U1z7,1740,chore: Set permissions for GitHub actions,172697,naveensrinivasan,closed,0,,,,,1,2022-05-05T01:03:08Z,2022-05-31T19:28:41Z,2022-05-31T19:28:40Z,CONTRIBUTOR,simonw/datasette/pulls/1740," Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1740/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1251739062,I_kwDOBm6k_c5KnAW2,1752,Research if I can drop Janus,9599,simonw,open,0,,,,,0,2022-05-28T22:46:52Z,2022-05-28T22:46:52Z,,OWNER,,"> It seems to me Janus dependency is not necessary, `async with app.database_write_mutex(): out = await app.transaction(func)` may be enough.
Comment here: https://lobste.rs/s/fki4tj/architecture_notes_datasette#c_a2ihon",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1752/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1251710928,I_kwDOBm6k_c5Km5fQ,1751,Add scrollbars to table presentation in default layout,408765,knutwannheden,closed,0,,,,,1,2022-05-28T19:44:57Z,2022-05-28T19:52:17Z,2022-05-28T19:52:17Z,NONE,,"(As you will be able to tell from the terminology I use, I am not a frontend guy, but I hope you will understand.)
When a table is wide and needs horizontal scrolling to see the columns towards the end, the user needs to scroll horizontally. However, since the container for the HTML table (`div` with class `table-wrapper`) isn't limited by the window size, I first need to vertically scroll near to the bottom of the page in order to scroll horizontally. Then I can scroll back up again. This isn't very user friendly. Instead, I think it would make sense to constrain the table's size (when necessary), so that the vertical and horizontal scrollbars either always are visible or at least not far out of reach.
I understand that I could provide my own template and / or CSS, but I think it would probably make sense to adjust the default in this regard.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1751/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1251700382,I_kwDOBm6k_c5Km26e,1750,Allow `label_column` to specify array of columns,408765,knutwannheden,open,0,,,,,0,2022-05-28T18:45:48Z,2022-05-28T18:45:48Z,,NONE,,"I think it would be great if the Datasette metadata would allow the `label_column` table key to list multiple columns. Something like:
```json
""tables"": {
""person"": {
""label_column"": [""first_name"", ""last_name""]
},
```
It would even be interesting with a ""label expression"" similar to a Python f-string. E.g. `{row.last_name}, {row.first_name}`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1750/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1247315144,I_kwDOBm6k_c5KWITI,1749,LDAP auth plugin,380241,benswift,open,0,,,,,0,2022-05-25T01:35:12Z,2022-05-25T01:35:12Z,,NONE,,"A [search of the plugins directory](https://datasette.io/plugins?q=ldap) doesn't turn up anything, but is is possible to set up a Datasette app which uses my organisation's LDAP for auth?
If not, how much work would it be to write one (I _may_ have some spare cycles on my team to do this, but we haven't written a datasette plugin before).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1749/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1243498298,I_kwDOBm6k_c5KHkc6,1746,Switch documentation theme to Furo,9599,simonw,closed,0,,,,,21,2022-05-20T18:42:17Z,2022-05-20T21:28:29Z,2022-05-20T21:28:29Z,OWNER,,"https://github.com/pradyunsg/furo
I just did this for `shot-scraper` and I really like it: https://shot-scraper.datasette.io/en/latest/
- https://github.com/simonw/shot-scraper/issues/77",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1746/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1243517592,I_kwDOBm6k_c5KHpKY,1748,Add copy buttons next to code examples in the documentation,9599,simonw,closed,0,,,,,2,2022-05-20T19:09:00Z,2022-05-20T19:15:00Z,2022-05-20T19:11:32Z,OWNER,,Similar to the ones in `datasette-copyable` which are implemented here: https://github.com/executablebooks/sphinx-copybutton/tree/f84c001a0507f8ec46779d0701b079a265564583,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1748/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1243512344,I_kwDOBm6k_c5KHn4Y,1747,Add tutorials to the getting started guide,9599,simonw,closed,0,,,,,1,2022-05-20T19:01:52Z,2022-05-20T19:12:30Z,2022-05-20T19:05:34Z,OWNER,,On https://docs.datasette.io/en/stable/getting_started.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1747/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1239008850,I_kwDOBm6k_c5J2cZS,1744,`--nolock` feature for opening locked databases,9599,simonw,closed,0,,,,,7,2022-05-17T18:25:16Z,2022-05-17T19:46:38Z,2022-05-17T19:40:30Z,OWNER,,"The getting started docs currently suggest you try this to browse your Chrome history:
datasette ~/Library/Application\ Support/Google/Chrome/Default/History
But if Chrome is running you will likely get this error:
sqlite3.OperationalError: database is locked
Turns out there's a workaround for this which I just spotted [on the SQLite forum](https://sqlite.org/forum/forumpost/86a67f6995):
> You can do this using a [URI filename](https://sqlite.org/uri.html):
> ```
> sqlite3 'file:places.sqlite?mode=ro&nolock=1'
> ```
> That opens the file `places.sqlite` in read-only mode with locking disabled. This isn't safe, in that changes to the database made by other corrections are likely to cause this connection to return incorrect results or crash. Read-only mode should at least mean that you don't corrupt the database in the process.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1744/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1239080102,I_kwDOBm6k_c5J2tym,1745,Documentation on running cog,9599,simonw,closed,0,,,,,1,2022-05-17T19:41:06Z,2022-05-17T19:45:51Z,2022-05-17T19:43:45Z,OWNER,,Noticed that `cog -r docs/*.rst` isn't documented in https://docs.datasette.io/en/latest/contributing.html#editing-and-building-the-documentation,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1745/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1237871948,I_kwDOBm6k_c5JyG1M,1743,`datasette.utils.to_css_class()` should be a documented internal,9599,simonw,open,0,,,,,0,2022-05-16T23:57:26Z,2022-05-16T23:57:26Z,,OWNER,,"Because I'm using it in this plugin:
- https://github.com/simonw/datasette-upload-dbs/issues/1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1743/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1237586379,I_kwDOBm6k_c5JxBHL,1742,?_trace=1 fails with datasette-geojson for some reason,9599,simonw,open,0,,,,,4,2022-05-16T19:06:05Z,2022-05-16T19:42:13Z,,OWNER,,view-source:https://calands.datasettes.com/calands/CPAD_2020a_SuperUnits.geojson?_sort=id&id__exact=4&_labels=on&_trace=1 is showing me a blank page.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1742/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
607223136,MDU6SXNzdWU2MDcyMjMxMzY=,741,"Replace ""datasette publish --extra-options"" with ""--setting""",9599,simonw,open,0,,,3268330,Datasette 1.0,9,2020-04-27T04:29:04Z,2022-05-12T19:21:16Z,,OWNER,,"See https://github.com/simonw/datasette-publish-now/issues/9#issuecomment-618155764 - the `--extra-options` mechanism is in practice just used to set `--config` options in data that you publish, but that means you end up with pretty messy looking commands:
datasette publish my.db --extra-options=""--config default_page_size:50 --config sql_time_limit_ms:3500""
A neater design would be to support `--config` as an option for `datasette publish` directly:
datasette publish my.db --config default_page_size:50 --config sql_time_limit_ms:3500
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/741/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1223699280,I_kwDOBm6k_c5I8CtQ,1739,.db downloads should be served with an ETag,9599,simonw,closed,0,,,,,6,2022-05-03T05:11:21Z,2022-05-04T18:21:18Z,2022-05-03T14:59:51Z,OWNER,,"I noticed that my Pyodide Datasette prototype is downloading the same database file every single time rather than browser caching it:
![image](https://user-images.githubusercontent.com/9599/166407074-dee19587-0667-4424-9e88-d3b5b90fd819.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1739/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223234932,I_kwDOBm6k_c5I6RV0,1733,Get Datasette compatible with Pyodide,9599,simonw,closed,0,,,,,9,2022-05-02T19:01:58Z,2022-05-04T15:14:01Z,2022-05-02T20:15:27Z,OWNER,,"I've already got this working as a prototype. Here are the changes I had to make:
- Replace the two dependencies that don't publish pure Python wheels to PyPI: `click-default-group` and `python-baseconv`
- Get Datasette to work without threading - which it turns out is exclusively used for database connections
- Make the `uvicorn` dependency optional (only needed when Datasette runs in the CLI)
TODO:
- [x] Switch to `click-default-group-wheel`
- [x] https://github.com/simonw/datasette/issues/1734
- [x] Work around `uvicorn` import error
- [x] https://github.com/simonw/datasette/issues/1735
- [x] #1737
Goal is to be able to do the following directly in https://pyodide.org/en/stable/console.html
```python
import micropip
await micropip.install(""datasette"")
from datasette.app import Datasette
ds = Datasette()
await ds.client.get(""/.json"")
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1733/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1221849746,I_kwDOBm6k_c5I0_KS,1732,Custom page variables aren't decoded,52649,tannewt,open,0,,,,,2,2022-04-30T14:55:46Z,2022-05-03T01:50:45Z,,NONE,,"I have a page `templates/filer/{filer_id}.html`. It uses `filer_id` in a `sql()` call to fetch data. With 0.61.1 this no longer works because the spaces in IDs isn't preserved. Instead, the escaped version is passed into the template and the id isn't present in my db.
Datasette should unescape the url component before passing them into the template.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1732/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1223459734,I_kwDOBm6k_c5I7IOW,1737,Automated test for Pyodide compatibility,9599,simonw,closed,0,,,,,4,2022-05-02T23:24:25Z,2022-05-02T23:40:50Z,2022-05-02T23:40:50Z,OWNER,,"Refs:
- #1733
Need something in the test suite such that if Datasette breaks against Pyodide in the future we hear about it.
I'm thinking this is an opportunity to use [shot-scraper javascript](https://github.com/simonw/shot-scraper#scraping-pages-using-javascript).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1737/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223241647,I_kwDOBm6k_c5I6S-v,1734,Remove python-baseconv dependency,9599,simonw,closed,0,,,,,3,2022-05-02T19:08:37Z,2022-05-02T23:25:49Z,2022-05-02T19:39:20Z,OWNER,,"> I was going to vendor `baseconv.py`, but then I reconsidered - what if there are plugins out there that expect `import baseconv` to work because they have depended on Datasette?
>
> I used https://cs.github.com/ and as far as I can tell there aren't any!
>
> So I'm going to remove that dependency and work out a smarter way to do this - probably by providing a utility function within Datasette itself.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115258737_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1734/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223263540,I_kwDOBm6k_c5I6YU0,1735,Datasette setting to disable threading (for Pyodide),9599,simonw,closed,0,,,,,1,2022-05-02T19:31:08Z,2022-05-02T23:25:49Z,2022-05-02T20:13:52Z,OWNER,,"> I'm going to add a Datasette setting to disable threading entirely, designed for usage in this particular case.
>
> I thought about adding a new setting, then I noticed this:
>
> datasette mydatabase.db --setting num_sql_threads 10
>
> I'm going to let users set that to `0` to disable threaded execution of SQL queries.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115278325_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1735/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223312279,PR_kwDOBm6k_c43MIU0,1736,Clean up compatibility with Pyodide,9599,simonw,closed,0,,,,,0,2022-05-02T20:14:38Z,2022-05-02T20:15:28Z,2022-05-02T20:15:27Z,OWNER,simonw/datasette/pulls/1736,"Closes #1735, closes #1733",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1736/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1219398983,I_kwDOBm6k_c5Iro1H,1730,SQL tracing should much more closely track the SQL query execution,9599,simonw,open,0,,,,,0,2022-04-28T22:41:04Z,2022-04-28T22:41:10Z,,OWNER,,"In #1727 I realized that the SQL tracing was measuring a whole bunch of stuff outside of the SQL query itself.
I started experimenting with this fix for that but it didn't work - I got back an empty JSON array of traces for some reason:
```diff
diff --git a/datasette/database.py b/datasette/database.py
index ba594a8..d7f9172 100644
--- a/datasette/database.py
+++ b/datasette/database.py
@@ -7,7 +7,7 @@ import sys
import threading
import uuid
-from .tracer import trace
+from .tracer import trace, trace_child_tasks
from .utils import (
detect_fts,
detect_primary_keys,
@@ -207,30 +207,31 @@ class Database:
time_limit_ms = custom_time_limit
with sqlite_timelimit(conn, time_limit_ms):
- try:
- cursor = conn.cursor()
- cursor.execute(sql, params if params is not None else {})
- max_returned_rows = self.ds.max_returned_rows
- if max_returned_rows == page_size:
- max_returned_rows += 1
- if max_returned_rows and truncate:
- rows = cursor.fetchmany(max_returned_rows + 1)
- truncated = len(rows) > max_returned_rows
- rows = rows[:max_returned_rows]
- else:
- rows = cursor.fetchall()
- truncated = False
- except (sqlite3.OperationalError, sqlite3.DatabaseError) as e:
- if e.args == (""interrupted"",):
- raise QueryInterrupted(e, sql, params)
- if log_sql_errors:
- sys.stderr.write(
- ""ERROR: conn={}, sql = {}, params = {}: {}\n"".format(
- conn, repr(sql), params, e
+ with trace(""sql"", database=self.name, sql=sql.strip(), params=params):
+ try:
+ cursor = conn.cursor()
+ cursor.execute(sql, params if params is not None else {})
+ max_returned_rows = self.ds.max_returned_rows
+ if max_returned_rows == page_size:
+ max_returned_rows += 1
+ if max_returned_rows and truncate:
+ rows = cursor.fetchmany(max_returned_rows + 1)
+ truncated = len(rows) > max_returned_rows
+ rows = rows[:max_returned_rows]
+ else:
+ rows = cursor.fetchall()
+ truncated = False
+ except (sqlite3.OperationalError, sqlite3.DatabaseError) as e:
+ if e.args == (""interrupted"",):
+ raise QueryInterrupted(e, sql, params)
+ if log_sql_errors:
+ sys.stderr.write(
+ ""ERROR: conn={}, sql = {}, params = {}: {}\n"".format(
+ conn, repr(sql), params, e
+ )
)
- )
- sys.stderr.flush()
- raise
+ sys.stderr.flush()
+ raise
if truncate:
return Results(rows, truncated, cursor.description)
@@ -238,9 +239,8 @@ class Database:
else:
return Results(rows, False, cursor.description)
- with trace(""sql"", database=self.name, sql=sql.strip(), params=params):
- results = await self.execute_fn(sql_operation_in_thread)
- return results
+ with trace_child_tasks():
+ return await self.execute_fn(sql_operation_in_thread)
@property
def size(self):
```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1727#issuecomment-1111602802_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1730/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1129052172,I_kwDOBm6k_c5DS_gM,1633,base_url or prefix does not work with _exact match,6613091,henrikek,open,0,,,,,2,2022-02-09T21:45:07Z,2022-04-28T09:12:56Z,,NONE,,"When i hit ""Apply"" button to search with ""_exact"" for a column syntax the URL prefix is removed from the url.
![image](https://user-images.githubusercontent.com/6613091/153293758-0b757d55-5757-4987-992e-9426e69a7956.png)
And the result is:
![image](https://user-images.githubusercontent.com/6613091/153294672-87be7809-bb7b-455d-bf1a-41e90bbfa4ae.png)
If I add the marked row to url_builder.py it seams to work:
![image](https://user-images.githubusercontent.com/6613091/153295231-bdd52e37-efcf-4b21-9d37-69f182a922f4.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1633/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1065432388,I_kwDOBm6k_c4_gTVE,1534,Maybe return JSON from HTML pages if `Accept: application/json` is sent,9599,simonw,closed,0,,,,,4,2021-11-28T20:48:09Z,2022-04-27T21:59:34Z,2022-02-02T23:39:33Z,OWNER,,"Relates to #1533 - and to the work I've been doing on the https://github.com/simonw/datasette-table Web Component.
It would be useful to support users pasting in a URL to a Datasette table or query without first having to add the `.json` extension themselves - since then other systems could hit that URL with `Accept: application/json` to get back the JSON representation without first needing to read the `Link: ` header from #1533 to figure out what the URL to that JSON is.
(There is weird logic deep in Datasette that says that you add `.json` to the path UNLESS the table name itself ends with `.json`, in which case you add `?_format=json` - this is super-confusing).
[Update: I removed that confusing feature here: [https://simonwillison.net/2022/Mar/19/weeknotes/](https://simonwillison.net/2022/Mar/19/weeknotes/)]",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1534/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1216508080,I_kwDOBm6k_c5IgnCw,1723,Research running SQL in table view in parallel using `asyncio.gather()`,9599,simonw,closed,0,,,,,5,2022-04-26T21:42:48Z,2022-04-27T18:53:44Z,2022-04-26T22:19:09Z,OWNER,,"Spun off from:
- #1715",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1723/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1217014076,I_kwDOBm6k_c5Iiik8,1726,Security page in the documentation,9599,simonw,open,0,,,,,0,2022-04-27T08:43:30Z,2022-04-27T08:43:30Z,,OWNER,,"A page talking about how to run Datasette securely, and security concerns to take into account.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1726/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1216619276,I_kwDOBm6k_c5IhCMM,1724,?_trace=1 doesn't work on Global Power Plants demo,9599,simonw,closed,0,,,,,3,2022-04-27T00:15:02Z,2022-04-27T06:15:14Z,2022-04-27T00:18:30Z,OWNER,,"https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_trace=1 is not showing the trace JSON at the bottom of the page.
Confirmed that `trace_debug` is `true` on https://global-power-plants.datasettes.com/-/settings
Possibly related:
- https://github.com/simonw/datasette-total-page-time/issues/1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1724/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1216622905,I_kwDOBm6k_c5IhDE5,1725,Performance question - what is happening in this gap?,9599,simonw,open,0,,,,,0,2022-04-27T00:21:11Z,2022-04-27T00:21:11Z,,OWNER,,"Trace from https://latest-with-plugins.datasette.io/github/commits?_facet=repo&_trace=1&_facet=committer
![CleanShot 2022-04-26 at 17 20 06@2x](https://user-images.githubusercontent.com/9599/165413811-db2cd599-2acc-46ce-b9c2-f9bc45b879e9.png)
What's going on in that gap? Can I improve the tracing output to show some non-SQL queries to figure that out?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1725/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1216479167,I_kwDOBm6k_c5Igf-_,1722,`db.primary_keys()` and `db.table_columns()` don't show up in traces,9599,simonw,open,0,,,,,0,2022-04-26T21:08:36Z,2022-04-26T21:08:36Z,,OWNER,,"Noticed this while working on:
- #1715
This code here isn't showing up in traces: https://github.com/simonw/datasette/blob/579f59dcec43a91dd7d404e00b87a00afd8515f2/datasette/views/table.py#L218-L220
Because those functions don't use the regular trace-instrumented `db.execute()` code path - they work directly against a connection instead: https://github.com/simonw/datasette/blob/579f59dcec43a91dd7d404e00b87a00afd8515f2/datasette/utils/__init__.py#L610-L626
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1722/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1213683988,I_kwDOBm6k_c5IV1kU,1718,Code examples in the documentation should be formatted with Black,9599,simonw,closed,0,,,,,12,2022-04-24T15:22:50Z,2022-04-24T16:24:14Z,2022-04-24T16:18:03Z,OWNER,,"For example on this page: https://docs.datasette.io/en/stable/writing_plugins.html#packaging-a-plugin
I wonder if there's an easy way for me to enforce this for Sphinx documentation?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1718/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1213281044,PR_kwDOBm6k_c42qyUI,1717,Add timeout option to Cloudrun build,127565,wragge,closed,0,,,,,2,2022-04-23T11:51:21Z,2022-04-24T14:03:08Z,2022-04-24T14:03:08Z,CONTRIBUTOR,simonw/datasette/pulls/1717,I've found that the Cloudrun build phase often hits a timeout limit with large databases. I believe the default timeout is 10 minutes. This pull request just adds a `--timeout` option to the cloudrun `publish` command and passes the value on to the build step.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1717/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1212838949,I_kwDOBm6k_c5ISnQl,1716,Configure git blame to ignore Black commit,9599,simonw,closed,0,,,,,1,2022-04-22T21:56:37Z,2022-04-22T22:02:19Z,2022-04-22T22:02:19Z,OWNER,,"GitHub can support this in blame views now too:
https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1716/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
810507413,MDExOlB1bGxSZXF1ZXN0NTc1MTg3NDU3,1229,ensure immutable databses when starting in configuration directory mode with,295329,camallen,closed,0,,,,,3,2021-02-17T20:18:26Z,2022-04-22T13:16:36Z,2021-03-29T00:17:32Z,CONTRIBUTOR,simonw/datasette/pulls/1229,"fixes #1224
This PR ensures all databases found in a configuration directory that match the files in `inspect-data.json` will be set to `immutable` as outlined in https://docs.datasette.io/en/latest/settings.html#configuration-directory-mode
specifically on building the `datasette` instance it checks:
- if `immutables` is an empty tuple - as passed by the cli code
- if `immutables` is the default function value `None` - when it's not explicitly set
And correctly builds the immutable database list from the `inspect-data[file]` keys.
Note for this to work the `inspect-data.json` file must contain `file` paths which are relative to the configuration directory otherwise the file paths won't match and the dbs won't be set to immutable.
I couldn't find an easy way to test this due to the way `make_app_client` works, happy to take directions on adding a test for this.
I've updated the relevant docs as well, i.e. use the `inspect` cli cmd from the config directory path to create the relevant file
```
cd $config_dir
datasette inspect *.db --inspect-file=inspect-data.json
```
https://docs.datasette.io/en/latest/performance.html#using-datasette-inspect",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1229/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1202227104,I_kwDOBm6k_c5HqIeg,1712,"Make """" easier to read",9599,simonw,closed,0,,,,,3,2022-04-12T18:17:07Z,2022-04-12T19:12:22Z,2022-04-12T18:44:20Z,OWNER,,"`Binary: 2,427,344 bytes` would be nicer - even better, include a tooltip showing that size translated using this function: https://github.com/simonw/datasette/blob/138e4d9a53e3982137294ba383303c3a848cfca4/datasette/utils/__init__.py#L837-L846
![CleanShot 2022-04-12 at 11 15 04@2x](https://user-images.githubusercontent.com/9599/163027324-b0b6092e-6e11-438b-8077-789025d0bb37.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1712/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1200649889,I_kwDOBm6k_c5HkHah,1710,Guide for plugin authors to upgrade their plugins for 1.0,9599,simonw,closed,0,,,,,1,2022-04-11T22:58:25Z,2022-04-11T23:04:01Z,2022-04-11T23:03:25Z,OWNER,,I'll also encourage testing against both Datasette 0.x and Datasette 1.0 using a GitHub Actions matrix.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1710/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1200224939,I_kwDOBm6k_c5Hifqr,1707,[feature] expanded detail page,536941,fgregg,open,0,,,,,1,2022-04-11T16:29:17Z,2022-04-11T16:33:00Z,,CONTRIBUTOR,,"Right now, if click on the detail page for a row you get the info for the row and links to related tables:
![Screenshot 2022-04-11 at 12-27-26 lm20 filing](https://user-images.githubusercontent.com/536941/162786802-90ac1a71-4624-47c4-ae55-b783f4f6c92d.png)
It would be very cool if there was an option to expand the rows of the related tables from within this detail view.
If you had that then datasette could fulfill a pretty common use case where you want to search for an entity and get a consolidate detail view about what you know about that entity.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1707/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1193090967,I_kwDOBm6k_c5HHR-X,1699,Proposal: datasette query,25778,eyeseast,open,0,,,,,6,2022-04-05T12:36:43Z,2022-04-11T01:32:12Z,,CONTRIBUTOR,,"I started sketching out a plugin to add a `datasette query` subcommand to export data from the command line. This is based on discussions in #1356 and #1605. Before I get too far down this rabbit hole, I figure it's worth getting some feedback here (unless this should happen in `Discussions`). Here's what I'm thinking:
At its most basic, it will write the results of a query to STDOUT.
```sh
datasette query -d data.db 'select * from data' > results.json
```
This isn't much improvement over using [sqlite-utils](https://github.com/simonw/sqlite-utils). To make better use of datasette and its ecosystem, run `datasette query` using a canned query defined in a `metadata.yml` file.
For example, using the metadata file from [alltheplaces-datasette](https://github.com/eyeseast/alltheplaces-datasette/blob/main/metadata.yml):
```sh
cd alltheplaces-datasette
datasette query -d alltheplaces.db -m metadata.yml count_by_spider
```
That query would be good to get as CSV, and we can auto-discover metadata and databases in the current directory:
```sh
cd alltheplaces-datasette
datasette query count_by_spider -f csv
```
In this case, `count_by_spider` is a canned query defined on the `alltheplaces` database. If the same query is defined on multiple databases or its otherwise unclear which database `query` should use, pass the `-d` or `--database` option.
If a query takes parameters, I can pass them in at runtime, using the `--param` or `-p` option:
```sh
datasette query -d data.db -p value something 'select * from neighborhoods where some_column = :value'
```
I'm very interested in feedback on this, including whether it should be a plugin or in Datasette core. (I don't have a strong opinion about this, but I'm prototyping it as a plugin to start.)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1699/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1197925865,I_kwDOBm6k_c5HZuXp,1704,File PRs against incompatible plugins pinning to datasette<1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-04-08T23:15:30Z,2022-04-08T23:15:30Z,,OWNER,,"As part of the preparation for the 1.0 release, test all existing known plugins against the alpha.
For any that break, submit a PR suggesting they pin to a version <1.0 - and include a link to the documentation on how to upgrade the plugin for 1.0.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1704/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1184850675,PR_kwDOBm6k_c41OrWq,1694,"Update click requirement from <8.1.0,>=7.1.1 to >=7.1.1,<8.2.0",49699333,dependabot[bot],closed,0,,,,,1,2022-03-29T13:11:23Z,2022-04-08T23:05:10Z,2022-04-08T23:05:09Z,CONTRIBUTOR,simonw/datasette/pulls/1694,"Updates the requirements on [click](https://github.com/pallets/click) to permit the latest version.
Release notes
Sourced from click's releases .
8.1.0
This is a feature release, which includes new features and removes previously deprecated features. The 8.1.x branch is now the supported bugfix branch, the 8.0.x branch will become a tag marking the end of support for that branch. We encourage everyone to upgrade, and to use a tool such as pip-tools to pin all dependencies and control upgrades.
Changelog
Sourced from click's changelog .
Version 8.1.0
Released 2022-03-28
Drop support for Python 3.6. :pr:2129
Remove previously deprecated code. :pr:2130
Group.resultcallback
is renamed to result_callback
.
autocompletion
parameter to Command
is renamed to
shell_complete
.
get_terminal_size
is removed, use
shutil.get_terminal_size
instead.
get_os_args
is removed, use sys.argv[1:]
instead.
Rely on :pep:538
and :pep:540
to handle selecting UTF-8 encoding
instead of ASCII. Click's locale encoding detection is removed.
:issue:2198
Single options boolean flags with show_default=True
only show
the default if it is True
. :issue:1971
The command
and group
decorators can be applied with or
without parentheses. :issue:1359
The Path
type can check whether the target is executable.
:issue:1961
Command.show_default
overrides Context.show_default
, instead
of the other way around. :issue:1963
Parameter decorators and @group
handles cls=None
the same as
not passing cls
. @option
handles help=None
the same as
not passing help
. :issue:[#1959](https://github.com/pallets/click/issues/1959)
A flag option with required=True
requires that the flag is
passed instead of choosing the implicit default value. :issue:1978
Indentation in help text passed to Option
and Command
is
cleaned the same as using the @option
and @command
decorators does. A command's epilog
and short_help
are also
processed. :issue:1985
Store unprocessed Command.help
, epilog
and short_help
strings. Processing is only done when formatting help text for
output. :issue:2149
Allow empty str input for prompt()
when
confirmation_prompt=True
and default=""
. :issue:2157
Windows glob pattern expansion doesn't fail if a value is an invalid
pattern. :issue:2195
It's possible to pass a list of params
to @command
. Any
params defined with decorators are appended to the passed params.
:issue:2131
.
@command
decorator is annotated as returning the correct type if
a cls
argument is used. :issue:2211
A Group
with invoke_without_command=True
and chain=False
will invoke its result callback with the group function's return
value. :issue:2124
... (truncated)
Commits
e4aceee
Merge pull request #2224 from pallets/release-8.1.0
f8d811e
release version 8.1.0
20c88f0
Merge pull request #2223 from pallets/env-var
8d7f03d
treat empty auto_envvar as None
ef11be6
Merge pull request #2041 from spanglerco/shell-completion-option-values
f2e579a
shell completion prioritizes option values over new options
d251cb0
Merge pull request #2219 from pallets/paramtype-name
e003331
fix ParamType.to_info_dict() with no name
19be092
Merge pull request #2217 from pallets/group-return
7d3a871
group without command passes return value to result callback
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1694/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1197298420,PR_kwDOBm6k_c4132NJ,1703,"Update beautifulsoup4 requirement from <4.11.0,>=4.8.1 to >=4.8.1,<4.12.0",49699333,dependabot[bot],closed,0,,,,,1,2022-04-08T13:08:53Z,2022-04-08T22:51:05Z,2022-04-08T22:51:05Z,CONTRIBUTOR,simonw/datasette/pulls/1703,"Updates the requirements on [beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/bs4/) to permit the latest version.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1703/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1196327155,I_kwDOBm6k_c5HToDz,1702,Be more consistent with column quoting,9599,simonw,open,0,,,,,0,2022-04-07T16:59:20Z,2022-04-07T16:59:20Z,,OWNER,,"This tutorial made me notice that Datasette is pretty inconsistent with how column quoting works: https://datasette.io/tutorials/learn-sql
It has examples of each of `""table_name""` and `[table_name]` and `table_name`, and it uses single quoted values too.
Datasette should generate SQL as consistently as possible to support learners.
That tutorial should also provide a tiny bit of extra information about what's going on here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1702/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1194790504,I_kwDOBm6k_c5HNw5o,1701,Use + for spaces instead of ~20,9599,simonw,closed,0,,,3268330,Datasette 1.0,0,2022-04-06T15:40:48Z,2022-04-06T15:55:10Z,2022-04-06T15:55:05Z,OWNER,,"Tilde encoding introduced in #1657 means that database files with spaces in the name - e.g. the Apple Mail `Envelope Index` database - end up with URLs like this:
http://127.0.0.1:8001/Envelope~20Index
I think this would be prettier:
http://127.0.0.1:9933/Envelope+Index",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1701/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1077620955,I_kwDOBm6k_c5AOzDb,1549,Redesign CSV export to improve usability,536941,fgregg,open,0,,,3268330,Datasette 1.0,5,2021-12-11T19:02:12Z,2022-04-04T11:17:13Z,,CONTRIBUTOR,,"*Original title: Set content type for CSV so that browsers will attempt to download instead opening in the browser*
Right now, if the user clicks on the CSV related to a table or a query, the response header for the content type is
""content-type: text/plain; charset=utf-8""
Most browsers will try to open a file with this content-type in the browser.
This is not what most people want to do, and lots of folks don't know that if they want to download the CSV and open it in the a spreadsheet program they next need to save the page through their browser.
It would be great if the response header could be something like
```
'Content-type: text/csv');
'Content-disposition: attachment;filename=MyVerySpecial.csv');
```
which would lead browsers to open a download dialog.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1549/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1190828163,I_kwDOBm6k_c5G-piD,1698,Add a warning about bots and Cloud Run,9599,simonw,closed,0,,,,,1,2022-04-03T05:57:17Z,2022-04-03T06:10:24Z,2022-04-03T06:10:24Z,OWNER,,Recommend the https://github.com/simonw/datasette-block-robots plugin if you are going to run a large database in Cloud Run (one with a lot of rows).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1698/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1189113609,I_kwDOBm6k_c5G4G8J,1697,"`Request.fake(..., url_vars={})`",9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2022-04-01T01:48:40Z,2022-04-01T02:02:18Z,2022-04-01T02:02:10Z,OWNER,,"I just created an alternative `.fake()` method because I wanted to fake the `url_vars` captured in the route as well:
```python
from datasette.utils.asgi import Request
class Request(Request):
@classmethod
def fake(cls, path_with_query_string, method=""GET"", scheme=""http"", url_vars=None):
""""""Useful for constructing Request objects for tests""""""
path, _, query_string = path_with_query_string.partition(""?"")
scope = {
""http_version"": ""1.1"",
""method"": method,
""path"": path,
""raw_path"": path_with_query_string.encode(""latin-1""),
""query_string"": query_string.encode(""latin-1""),
""scheme"": scheme,
""type"": ""http"",
}
if url_vars:
scope[""url_route""] = {
""kwargs"": url_vars
}
return cls(scope, None)
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1697/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1182227211,I_kwDOBm6k_c5Gd1sL,1692,[plugins][feature request]: Support additional script tag attributes when loading custom JS,9020979,hydrosquall,open,0,,,,,2,2022-03-27T01:16:03Z,2022-03-30T06:14:51Z,,CONTRIBUTOR,,"## Motivation
- The build system for my new [plugin](https://github.com/hydrosquall/datasette-nteract-data-explorer) has two output JS files, one for browsers that support ES modules, one for browsers that don't. At present, I'm only passing one of them into Datasette.
- I'd like to specify the non-es-module script as a fallback for older browsers. I don't want to load it by default, because browsers will only need one, and it's heavy, so for now I'm only supporting modern browsers.
To be able to support legacy browsers without slowing down users with modern browsers, I would like to be able to set additional HTML attributes on the tag fallback script, `nomodule` and `defer`. My injected scripts should look something like this:
```html
```
## Proposal
To achieve this, I propose additional optional properties to the API accepted by the `extra_js_urls` hook and custom JS field the `metadata.json` [described here](https://docs.datasette.io/en/stable/custom_templates.html#custom-css-and-javascript).
Under this API, I'd write something like this to get the above HTML rendered in Datasette.
```json
{
""extra_js_urls"": [
{
""url"": ""/index.my-es-module-bundle.js"",
""module"": true,
},
{
""url"": ""/index.my-legacy-fallback-bundle.js"",
""nomodule"": """",
""defer"": true
}
]
}
```
## Resources
- [MDN on the script tag](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script)
- There may be other properties that could be added that are potentially valuable, like `async` or `referrerpolicy`, but I don't have an immediate need for those.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1692/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1185868354,I_kwDOBm6k_c5GrupC,1695,Option to un-filter facet not shown for `?col__exact=value`,9599,simonw,open,0,,,,,2,2022-03-30T04:44:02Z,2022-03-30T04:46:18Z,,OWNER,,"Spotted this on a page with `COUNTY__exact=Lee` in the URL:
![CleanShot 2022-03-29 at 21 41 46@2x](https://user-images.githubusercontent.com/9599/160752849-a9039343-3770-4655-920b-f19e25687a57.png)
With `COUNTY=Lee` you get this instead:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1695/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1181432624,I_kwDOBm6k_c5Gazsw,1688,[plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins?,9020979,hydrosquall,closed,0,,,,,3,2022-03-26T01:17:44Z,2022-03-27T01:01:14Z,2022-03-26T21:34:47Z,CONTRIBUTOR,,"I'm trying to make a small plugin that depends on static assets, by following the guide [here](https://docs.datasette.io/en/stable/writing_plugins.html#writing-one-off-plugins). I made a `plugins/` directory with `datasette_nteract_data_explorer.py`.
I am trying to follow the example of `datasette_vega`, and serving static assets. I created a `statics/` directory within `plugins/` to serve my JS and CSS.
https://github.com/simonw/datasette-vega/blob/00de059ab1ef77394ba9f9547abfacf966c479c4/datasette_vega/__init__.py#L13
Unfortunately, datasette doesn't seem to be able to find my assets.
Input:
```bash
datasette ~/Library/Safari/History.db --plugins-dir=plugins/
```
![Image 2022-03-25 at 9 18 17 PM](https://user-images.githubusercontent.com/9020979/160218979-a3ff474b-5255-4a76-85d1-6f90ab2e3b44.jpg)
Output:
![Image 2022-03-25 at 9 11 00 PM](https://user-images.githubusercontent.com/9020979/160218733-ca5144cf-f23f-43d8-a8d3-e3a871e57f3a.jpg)
I suspect this issue might go away if I move away from ""one-off"" plugin mode, but it's been a while since I created a new python package so I'm not sure how much work there is to go between ""one off"" and ""packaged for PyPI"". I'd like to try to avoid needing to repackage a new `tar.gz` file and or reinstall my library repeatedly when developing new python code.
1. Is there a way to serve a static assets when using the `plugins/` directory method instead of installing plugins as a new python package?
2. If not, is there a way I can work on developing a plugin without creating and repackaging tar.gz files after every change, or is that the recommended path?
Thanks for your help!
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1688/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1182143895,I_kwDOBm6k_c5GdhWX,1691,Bug in pytest-httpx example,9599,simonw,closed,0,,,,,0,2022-03-26T22:45:30Z,2022-03-26T22:46:09Z,2022-03-26T22:46:09Z,OWNER,,"https://docs.datasette.io/en/0.61.1/testing_plugins.html#testing-outbound-http-calls-with-pytest-httpx says:
```python
async def test_outbound_http_call(httpx_mock):
httpx_mock.add_response(
url='https://www.example.com/',
data='Hello world',
)
```
That's wrong - `data=` should be `text=`.
https://github.com/Colin-b/pytest_httpx/blob/v0.20.0/README.md#reply-with-custom-body",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1691/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1182141761,I_kwDOBm6k_c5Gdg1B,1690,"Idea: `datasette.set_actor_cookie(response, actor)`",9599,simonw,open,0,,,,,2,2022-03-26T22:41:52Z,2022-03-26T22:43:00Z,,OWNER,,"I just wrote this code in a plugin and it felt like it could benefit from an abstraction: https://github.com/simonw/datasette-auth0/blob/152e6eb21e96e9b73bd9c205f9749a1297d0ef0b/datasette_auth0/__init__.py#L79-L92
```python
redirect_response = Response.redirect(""/"")
expires_at = int(time.time()) + (24 * 60 * 60)
redirect_response.set_cookie(
""ds_actor"",
datasette.sign(
{
""a"": profile_response.json(),
""e"": baseconv.base62.encode(expires_at),
},
""actor"",
),
)
return redirect_response
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1690/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1182065616,I_kwDOBm6k_c5GdOPQ,1689,datasette.add_message() documentation is incorrect,9599,simonw,closed,0,,,,,1,2022-03-26T20:49:42Z,2022-03-26T21:35:57Z,2022-03-26T20:51:21Z,OWNER,,"https://docs.datasette.io/en/0.61.1/internals.html#add-message-request-message-message-type-datasette-info says:
`.add_message(request, message, message_type=datasette.INFO)`
But in the code it's:
https://github.com/simonw/datasette/blob/6b99e4a66ba0ed8fca8ee41ceb7206928b60d5d1/datasette/app.py#L582",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1689/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1181364043,I_kwDOBm6k_c5Gai9L,1687,Make show_json.html or a similar mechanism stable for plugins,9599,simonw,open,0,,,,,0,2022-03-25T23:42:45Z,2022-03-25T23:42:45Z,,OWNER,,"I used `show_json.html` in the new `datasette-packages` plugin, which means it will break if that template changes:
- https://github.com/simonw/datasette-packages/issues/3
It would be useful if it (or something like it) was documented and stable for plugins to use.
Also relevant:
- #878",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1687/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1181037277,I_kwDOBm6k_c5GZTLd,1686,heroku bails if app name specifed in datasette publish is the same as existing app,2115933,tlongers,open,0,,,,,0,2022-03-25T17:10:34Z,2022-03-25T17:10:34Z,,NONE,,"Seem that `heroku` does not accept an app overwrite triggered by specifying the app name using `datasette publish`, as below:
```
datasette publish heroku some.db --name ""jazzy-name""
```
The resulting error has the below traceback:
```
Creating jazzy-name... !
▸ Name jazzy-name is already taken
Traceback (most recent call last):
File ""/opt/homebrew/bin/datasette"", line 33, in
sys.exit(load_entry_point('datasette==0.60.1', 'console_scripts', 'datasette')())
File ""/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py"", line 1128, in __call__
return self.main(*args, **kwargs)
File ""/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py"", line 1053, in main
rv = self.invoke(ctx)
File ""/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py"", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py"", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py"", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/click/core.py"", line 754, in invoke
return __callback(*args, **kwargs)
File ""/opt/homebrew/Cellar/datasette/0.60.1/libexec/lib/python3.10/site-packages/datasette/publish/heroku.py"", line 127, in heroku
create_output = check_output(cmd).decode(""utf8"")
File ""/opt/homebrew/Cellar/python@3.10/3.10.2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py"", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File ""/opt/homebrew/Cellar/python@3.10/3.10.2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py"", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['heroku', 'apps:create', 'jazzy-name', '--json']' returned non-zero exit status 1.
```
It's a solid failsafe, but does `datasette publish` have a way to force an overwrite?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1686/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1179998071,I_kwDOBm6k_c5GVVd3,1684,Mechanism for disabling faceting on large tables only,9599,simonw,open,0,,,,,1,2022-03-24T20:06:11Z,2022-03-24T20:13:19Z,,OWNER,,"Forest turned off faceting on https://labordata.bunkum.us/ because it was causing performance problems on some of the huge tables - but it would be nice if it could still be an option on smaller tables such as https://labordata.bunkum.us/voluntary_recognitions-4421085/voluntary_recognitions
One option: a new setting that automatically disables faceting (and facet suggestion) for tables that have either more than X rows or that are so big that the count could not be completed within the time limit.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1684/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1179928510,I_kwDOBm6k_c5GVEe-,1683,allow_facet: False should be respected by column cog menu,9599,simonw,closed,0,,,,,0,2022-03-24T19:05:06Z,2022-03-24T19:16:36Z,2022-03-24T19:16:36Z,OWNER,,"The column cog menu currently shows ""Facet by this"" even if faceting is disabled for the Datasette instance.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1683/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1089529555,I_kwDOBm6k_c5A8ObT,1581,"when hashed urls are turned on, the _memory db has improperly long-lived cache expiry",536941,fgregg,closed,0,,,,,1,2021-12-28T00:05:48Z,2022-03-24T04:08:18Z,2022-03-24T04:08:18Z,CONTRIBUTOR,,"if hashed_urls are on, then a -000 suffix is added to the `_memory` database, and the cache settings are set just as if it was a normal hashed database.
in particular, this header is set:
`cache-control: max-age=31536000`
this is not appropriate because the `_memory-000` database isn't really hashed based on the contents of the databases (see #1561).
Either the cache-control header should be changed, or the _memory db should have a hash suffix that does depend on the contents of the databases.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1581/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1090055810,PR_kwDOBm6k_c4wWDxH,1582,don't set far expiry if hash is '000',536941,fgregg,closed,0,,,,,1,2021-12-28T18:16:13Z,2022-03-24T04:07:58Z,2022-03-24T04:07:58Z,CONTRIBUTOR,simonw/datasette/pulls/1582,"This will close #1581.
I couldn't find any unit tests related to the testing hashed urls, and I know that you want to break that code out of the core application (#1561), so I'm not quite sure what you would like me to for testing.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1582/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1173828092,PR_kwDOBm6k_c40q1eP,1665,Pin setup-gcloud to v0 instead of master,408570,sethvargo,closed,0,,,,,1,2022-03-18T17:17:22Z,2022-03-23T19:31:10Z,2022-03-23T17:55:39Z,NONE,simonw/datasette/pulls/1665,"setup-gcloud will be updating the branch name from master to main in
a future release. Even though GitHub will establish redirects, this
will break any GitHub Actions workflows that pin to master. This PR
updates your GitHub Actions workflows to pin to v0, which is the
recommended best practice.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1665/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1178521513,I_kwDOBm6k_c5GPs-p,1682,SQL queries against databases with different routes are broken,9599,simonw,closed,0,,,,,1,2022-03-23T18:42:57Z,2022-03-23T18:48:16Z,2022-03-23T18:48:16Z,OWNER,,"500 error on https://datasette-hashed-urls-preview.vercel.app/fixtures-09f8f95?sql=select+*+from+facetable
Here's the trace:
```
File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py"", line 54, in data
return await QueryView(self.ds).data(
File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py"", line 232, in data
self.ds.get_database(database), sql
File ""/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/app.py"", line 401, in get_database
return self.databases[name]
KeyError: 'fixtures-aa7318b'
```
It looks like this is a Datasette bug, which is frustrating because I just shipped Datasette 0.61 five minutes ago!
_Originally posted by @simonw in https://github.com/simonw/datasette-hashed-urls/issues/13#issuecomment-1076693667_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1682/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174423568,I_kwDOBm6k_c5GAEgQ,1670,Ship Datasette 0.61,9599,simonw,closed,0,,,,,6,2022-03-20T02:47:54Z,2022-03-23T18:32:32Z,2022-03-23T18:32:03Z,OWNER,,"Let the alpha bake for a while, since #1668 is a big last-minute change.
After shipping, release a new `datasette-hashed-urls` that depends on it, also this:
- https://github.com/simonw/datasette-hashed-urls/issues/11",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1670/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1177101697,I_kwDOBm6k_c5GKSWB,1681,Potential bug in numeric handling where_clause for filters,9599,simonw,open,0,,,,,2,2022-03-22T17:43:50Z,2022-03-22T17:49:09Z,,OWNER,,"> Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like `>`:
>
> https://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/datasette/filters.py#L203-L212
>
> Though... it looks like there's a bug in that? It doesn't account for `float` values - `""3.5"".isdigit()` return `False` - probably for the best, because `int(3.5)` would break that value anyway.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1681/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1174655187,I_kwDOBm6k_c5GA9DT,1671,Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply,9308268,rayvoelker,open,0,,,,,8,2022-03-20T19:17:24Z,2022-03-22T17:43:12Z,,NONE,,"I found a strange behavior, and I'm not sure if it's related to views and boolean values perhaps, or if there's something else weird going on here, but I'll provide an example that may help show what I'm seeing happen.
```bash
#!/bin/bash
echo ""\""id\"",\""expiration_date\""
0,2018-01-04
1,2019-01-05
2,2020-01-06
3,2021-01-07
4,2022-01-08
5,2023-01-09
6,2024-01-10
7,2025-01-11
8,2026-01-12
9,2027-01-13
"" > test.csv
csvs-to-sqlite test.csv test.db
sqlite-utils create-view --replace test.db test_view ""select id, expiration_date, case when julianday('NOW') >= julianday(expiration_date) then 1 else 0 end as has_expired FROM test""
```
```bash
datasette test.db
```
![image](https://user-images.githubusercontent.com/9308268/159178745-9c6152f7-eac6-4bf9-bef5-a2d63d3ee13f.png)
![image](https://user-images.githubusercontent.com/9308268/159178824-c8952137-270c-42a4-ad1c-f6ad2c51e499.png)
![image](https://user-images.githubusercontent.com/9308268/159178877-23e00b36-443a-43ef-83e5-e0bdddd3fdcd.png)
![image](https://user-images.githubusercontent.com/9308268/159178918-65922cc7-2514-4735-a72d-4904b99976d4.png)
Thanks again and let me know if you want me to provide anything else!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1671/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
340396247,MDU6SXNzdWUzNDAzOTYyNDc=,339,Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way,12617395,bsilverm,closed,0,,,,,4,2018-07-11T20:38:06Z,2022-03-21T22:22:40Z,2022-03-21T22:22:34Z,NONE,,"Is it possible to configure the sql_time_limit_ms beyond 60 seconds? It seems queries are still timing out at 60 seconds when sql_time_limit_ms is set to 180000. We have a very large data set and often encounter timeouts when testing new queries from the datasette UI. We are optimizing our database as much as we can, but still may require more than 60 seconds for complex queries.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/339/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
324835838,MDU6SXNzdWUzMjQ4MzU4Mzg=,276,Handle spatialite geometry columns better,45057,russss,closed,0,,,,,21,2018-05-21T08:46:55Z,2022-03-21T22:22:20Z,2022-03-21T22:22:20Z,CONTRIBUTOR,,"I'd like to see spatialite geometry columns rendered more sensibly - at the moment they come through as well-known-binary unless you use custom SQL, and WKB isn't of much use to anyone on the web.
In HTML: they should be shown either as simple lat/long (if it's just a point, for example), or as a sensible placeholder if they're more complex geometries.
In JSON: they should be GeoJSON geometries, (which means they can be automatically fed into a leaflet map with no further messing around).
In CSV: they should be WKT.
I briefly wondered if this should go into a plugin, but I suspect it needs hooking in at a deeper level than the plugin architecture will support any time soon.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/276/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
953218043,MDU6SXNzdWU5NTMyMTgwNDM=,1403,Labels explaining what hidden tables are for,9599,simonw,open,0,,,,,0,2021-07-26T19:29:22Z,2022-03-21T22:20:37Z,,OWNER,,"A reasonable question: ""What are those hidden tables for?""
This could be answered by adding a small piece of explanatory text to each table - based on if it's related to FTS or to SpatiaLite or configured to be hidden for some other reason.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1403/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1175854982,I_kwDOBm6k_c5GFh-G,1679,Research: how much overhead does the n=1 time limit have?,9599,simonw,closed,0,,,3268330,Datasette 1.0,11,2022-03-21T19:27:46Z,2022-03-21T21:55:57Z,2022-03-21T21:55:56Z,OWNER,,"https://github.com/simonw/datasette/blob/1a7750eb29fd15dd2eea3b9f6e33028ce441b143/datasette/utils/__init__.py#L181-L200
```python
@contextmanager
def sqlite_timelimit(conn, ms):
deadline = time.perf_counter() + (ms / 1000)
# n is the number of SQLite virtual machine instructions that will be
# executed between each check. It's hard to know what to pick here.
# After some experimentation, I've decided to go with 1000 by default and
# 1 for time limits that are less than 50ms
n = 1000
if ms < 50:
n = 1
def handler():
if time.perf_counter() >= deadline:
return 1
conn.set_progress_handler(handler, n)
try:
yield
finally:
conn.set_progress_handler(None, n)
```
How often do I set a time limit of 50 or less? How much slower does it go thanks to this code?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1679/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175894898,I_kwDOBm6k_c5GFrty,1680,Consider simplifying permissions for 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-03-21T20:17:29Z,2022-03-21T20:17:29Z,,OWNER,,"Permission checks right now can express one of three opinions:
- `False` means ""so not grant this permisson""
- `True` means ""grant this permission""
- `None` means ""I have no opinion""
But... there's also a concept of a ""default"" for a given permission check, which might be `False` or `True`.
I worry this is too complicated. Could this be simplified before 1.0? In particular the default concept.
See also:
- #1676 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1680/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1170144879,I_kwDOBm6k_c5Fvv5v,1660,Refactor and simplify Datasette routing and views,9599,simonw,closed,0,,,3268330,Datasette 1.0,8,2022-03-15T19:56:56Z,2022-03-21T19:19:12Z,2022-03-21T19:19:01Z,OWNER,,"While working on:
- https://github.com/simonw/datasette/issues/1657
- https://github.com/simonw/datasette/issues/1439
It became very clear that the least maintainable part of Datasette at the moment is the way routing to the database, table and row views work - in particular the subclassing mechanism with BaseView and DataView, but also the complex variety of ways in which the URL routes capture different named regular expression groups.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1660/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175715988,I_kwDOBm6k_c5GFACU,1678,Make `check_visibility()` a documented API,9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2022-03-21T17:30:34Z,2022-03-21T19:04:03Z,2022-03-21T19:01:46Z,OWNER,,"Spotted this while working on:
- #1677
https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/datasette/utils/__init__.py#L1005-L1021",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1678/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175694248,I_kwDOBm6k_c5GE6uo,1677,Remove `check_permission()` from `BaseView`,9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2022-03-21T17:18:18Z,2022-03-21T18:45:04Z,2022-03-21T18:45:03Z,OWNER,,"Follow-on from:
- #1675
Refs:
- #1660",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1677/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175648453,I_kwDOBm6k_c5GEvjF,1675,Extract out `check_permissions()` from `BaseView,9599,simonw,closed,0,,,,,7,2022-03-21T16:39:46Z,2022-03-21T17:14:31Z,2022-03-21T17:13:21Z,OWNER,,"> I'm going to refactor this stuff out and document it so it can be easily used by plugins:
https://github.com/simonw/datasette/blob/4a4164b81191dec35e423486a208b05a9edc65e4/datasette/views/base.py#L69-L103
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1074136176_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1675/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
780153562,MDU6SXNzdWU3ODAxNTM1NjI=,1177,Ability to stream all rows as newline-delimited JSON,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2021-01-06T07:10:48Z,2022-03-21T15:08:52Z,,OWNER,,"> Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas:
>
> pandas.read_json(""https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on"", lines=True)
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1101#issuecomment-755128038_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1177/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1174717287,I_kwDOBm6k_c5GBMNn,1674,Tweak design of /.json,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2022-03-20T22:58:01Z,2022-03-20T22:58:40Z,,OWNER,,"https://latest.datasette.io/.json
Currently:
```json
{
""_memory"": {
""name"": ""_memory"",
""hash"": null,
""color"": ""a6c7b9"",
""path"": ""/_memory"",
""tables_and_views_truncated"": [],
""tables_and_views_more"": false,
""tables_count"": 0,
""table_rows_sum"": 0,
""show_table_row_counts"": false,
""hidden_table_rows_sum"": 0,
""hidden_tables_count"": 0,
""views_count"": 0,
""private"": false
},
""fixtures"": {
""name"": ""fixtures"",
""hash"": ""645005884646eb941c89997fbd1c0dd6be517cb1b493df9816ae497c0c5afbaa"",
""color"": ""645005"",
""path"": ""/fixtures"",
""tables_and_views_truncated"": [
{
""name"": ""compound_three_primary_keys"",
""columns"": [
""pk1"",
""pk2"",
""pk3"",
""content""
],
""primary_keys"": [
""pk1"",
""pk2"",
""pk3""
],
""count"": 1001,
""hidden"": false,
""fts_table"": null,
""num_relationships_for_sorting"": 0,
""private"": false
},
```
As-of this issue the `""path""` key is confusing, it doesn't match what https://latest.datasette.io/-/databases returns:
- #1668",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1674/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
910088936,MDU6SXNzdWU5MTAwODg5MzY=,1355,datasette --get should efficiently handle streaming CSV,9599,simonw,open,0,,,,,2,2021-06-03T04:40:40Z,2022-03-20T22:38:53Z,,OWNER,,"It would be great if you could use `datasette --get` to run queries that return streaming CSV data without running out of RAM.
Current implementation looks like it loads the entire result into memory first: https://github.com/simonw/datasette/blob/f78ebdc04537a6102316d6dbbf6c887565806078/datasette/cli.py#L546-L552",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1355/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1174708375,I_kwDOBm6k_c5GBKCX,1673,Streaming CSV spends a lot of time in `table_column_details`,9599,simonw,open,0,,,,,1,2022-03-20T22:25:28Z,2022-03-20T22:34:06Z,,OWNER,,"At least I think it does. I tried running `py-spy top -p $PID` against a Datasette process that was trying to do:
datasette covid.db --get '/covid/ny_times_us_counties.csv?_size=10&_stream=on'
While investigating:
- #1355
And spotted this:
```
datasette covid.db --get /covid/ny_times_us_counties.csv?_size=10&_stream=on' (python v3.10.2)
Total Samples 5800
GIL: 71.00%, Active: 98.00%, Threads: 4
%Own %Total OwnTime TotalTime Function (filename:line)
8.00% 8.00% 4.32s 4.38s sql_operation_in_thread (datasette/database.py:212)
5.00% 5.00% 3.77s 3.93s table_column_details (datasette/utils/__init__.py:614)
6.00% 6.00% 3.72s 3.72s _worker (concurrent/futures/thread.py:81)
7.00% 7.00% 2.98s 2.98s _read_from_self (asyncio/selector_events.py:120)
5.00% 6.00% 2.35s 2.49s detect_fts (datasette/utils/__init__.py:571)
4.00% 4.00% 1.34s 1.34s _write_to_self (asyncio/selector_events.py:140)
```
Relevant code: https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/__init__.py#L609-L625
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1673/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1174697144,I_kwDOBm6k_c5GBHS4,1672,Refactor CSV handling code out of DataView,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2022-03-20T21:47:00Z,2022-03-20T21:52:39Z,,OWNER,,"> I think the way to get rid of most of the remaining complexity in `DataView` is to refactor how CSV stuff works - pulling it in line with other export factors and extracting the streaming mechanism. Opening a fresh issue for that.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1073355032_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1672/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1174306154,I_kwDOBm6k_c5F_n1q,1668,"Introduce concept of a database `route`, separate from its name",9599,simonw,closed,0,,,3268330,Datasette 1.0,20,2022-03-19T16:48:28Z,2022-03-20T16:43:16Z,2022-03-20T16:43:16Z,OWNER,,"Some issues came up in the new `datasette-hashed-urls` plugin relating to the way it renames databases on startup to achieve unique URLs that depend on the database SHA-256 content:
- https://github.com/simonw/datasette-hashed-urls/issues/10
- https://github.com/simonw/datasette-hashed-urls/issues/9
- https://github.com/simonw/datasette-hashed-urls/issues/8
All three of these could be addressed by making the ""path"" concept for a database (the `/foo` bit where it is served) work independently of the database's name, which would be used for default display and also as the alias when configuring cross-database aliases.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1668/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174404647,I_kwDOBm6k_c5F__4n,1669,Release 0.61 alpha,9599,simonw,closed,0,,,,,2,2022-03-20T00:35:35Z,2022-03-20T01:24:36Z,2022-03-20T01:24:36Z,OWNER,,"> I'm going to release this as a 0.61 alpha so I can more easily depend on it from `datasette-hashed-urls`.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1669/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174302994,I_kwDOBm6k_c5F_nES,1667,Make route matched pattern groups more consistent,9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2022-03-19T16:32:35Z,2022-03-19T20:37:42Z,2022-03-19T20:37:41Z,OWNER,,"> ... highlights how inconsistent the way the capturing works is. Especially `as_format` which can be `None` or `""""` or `.json` or `json` or not used at all in the case of `TableView`.
https://github.com/simonw/datasette/blob/764738dfcb16cd98b0987d443f59d5baa9d3c332/tests/test_routes.py#L12-L36
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1666#issuecomment-1073039670_
Part of:
- #1660",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1667/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174162781,I_kwDOBm6k_c5F_E1d,1666,Refactor URL routing to enable testing,9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2022-03-19T03:52:29Z,2022-03-19T16:32:03Z,2022-03-19T16:32:03Z,OWNER,,"I ran into some bugs earlier with URL routing - having more robust testing around this (especially since they are defined using regular expressions) would be really useful.
- A utility function that resolves a path against a list of reflexes and returns the match
- Make the routes and regular expressions available from a private Datasette method
- Add tests that exercise them
Related:
- #1660",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1666/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
648435885,MDU6SXNzdWU2NDg0MzU4ODU=,878,"New pattern for views that return either JSON or HTML, available for plugins",9599,simonw,open,0,,,3268330,Datasette 1.0,26,2020-06-30T19:26:13Z,2022-03-19T16:19:30Z,,OWNER,,"Can be part of #870 - refactoring existing views to use `register_routes()`.
> I'm going to put the new `check_permissions()` method on `BaseView` as well. If I want that method to be available to plugins I can do so by turning that `BaseView` class into a documented API that plugins are encouraged to use themselves.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/832#issuecomment-651995453_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/878/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
531755959,MDU6SXNzdWU1MzE3NTU5NTk=,647,Move hashed URL mode out to a plugin,9599,simonw,closed,0,,,3268330,Datasette 1.0,9,2019-12-03T06:29:03Z,2022-03-19T11:56:05Z,2022-03-15T23:13:06Z,OWNER,,"They used to be the default until #418. Since making them optional I haven't felt the need to use them even once.
That suggests to me that they should be removed. I think their effect could be entirely handled by an ASGI wrapping plugin.
https://datasette.readthedocs.io/en/0.32/performance.html#hashed-url-mode",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/647/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
810397025,MDU6SXNzdWU4MTAzOTcwMjU=,1228,500 error caused by faceting if a column called `n` exists,7107523,Kabouik,closed,0,,,,,5,2021-02-17T17:41:20Z,2022-03-19T06:44:40Z,2022-03-19T01:38:04Z,NONE,,"I recently discovered `datasette` thanks to your great talk at FOSDEM and would like to use it for some projects. However, when trying to use it on databases created from some csv ot tsv files, I am sometimes getting this issue when going to http://127.0.0.1:8001/databasetest/databasetest and I don't exactly understand what it refers to.
So far, I couldn't find anything relevant when reviewing the raw text files that could explain this issue, nor could I find something obvious between the files that generate this issue and those that don't. Does the error ring a bell and, if so, could you please point me to the right direction?
```
$ datasette databasetest.db
INFO: Started server process [1408482]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
INFO: 127.0.0.1:56394 - ""GET / HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56394 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56396 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56398 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
Traceback (most recent call last):
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/app.py"", line 1099, in route_path
response = await view(request, send)
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 147, in view
request, **request.scope[""url_route""][""kwargs""]
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 121, in dispatch_request
return await handler(request, *args, **kwargs)
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 260, in get
request, database, hash, correct_hash_provided, **kwargs
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/base.py"", line 434, in view_get
request, database, hash, **kwargs
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/views/table.py"", line 782, in data
suggested_facets.extend(await facet.suggest())
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in suggest
and any(r[""n""] > 1 for r in distinct_values)
File ""/home/kabouik/.local/lib/python3.7/site-packages/datasette/facets.py"", line 168, in
and any(r[""n""] > 1 for r in distinct_values)
TypeError: '>' not supported between instances of 'str' and 'int'
INFO: 127.0.0.1:56402 - ""GET /databasetest/databasetest HTTP/1.1"" 500 Internal Server Error
INFO: 127.0.0.1:56402 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56404 - ""GET / HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56404 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56406 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56408 - ""GET /databasetest HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56408 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56404 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56410 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56414 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56408 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56408 - ""GET /databasetest?sql=select+*+from+databasetest HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56410 - ""GET /-/static/app.css?4e362c HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56408 - ""GET /-/static-plugins/datasette_vega/main.2acbb312.css HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56412 - ""GET /-/static/codemirror-5.57.0.min.css HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56404 - ""GET /-/static/sql-formatter-2.3.3.min.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56406 - ""GET /-/static/codemirror-5.57.0.min.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56414 - ""GET /-/static-plugins/datasette_vega/main.08f5d3d8.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56408 - ""GET /-/static/codemirror-5.57.0-sql.min.js HTTP/1.1"" 200 OK
INFO: 127.0.0.1:56410 - ""GET /databasetest.json?sql=select+*+from+databasetest&_shape=array&_shape=array HTTP/1.1"" 200 OK
^CINFO: Shutting down
INFO: Waiting for application shutdown.
INFO: Application shutdown complete.
INFO: Finished server process [1408482]
```
Note that there is no error if I go to http://127.0.0.1:8001/databasetest and then click on `Run SQL`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1228/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1082765654,I_kwDOBm6k_c5AibFW,1561,"add hash id to ""_memory"" url if hashed url mode is turned on and crossdb is also turned on",536941,fgregg,closed,0,,,,,3,2021-12-17T00:45:12Z,2022-03-19T04:45:40Z,2022-03-19T04:45:40Z,CONTRIBUTOR,,"If hashed_url mode is turned on and crossdb is also turned on, then queries to _memory should have a hash_id.
One way that it could work is to have the _memory hash be a hash of all the individual databases.
Otherwise, crossdb queries can get quit out of data if using aggressive caching.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1561/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1161584460,I_kwDOBm6k_c5FPF9M,1651,Get rid of the no-longer necessary ?_format=json hack for tables called x.json,9599,simonw,closed,0,,,3268330,Datasette 1.0,8,2022-03-07T15:40:42Z,2022-03-19T04:04:50Z,2022-03-15T18:25:42Z,OWNER,,"Tidy up from:
- #1439",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1651/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1169840669,I_kwDOBm6k_c5Fulod,1658,Revert main to version that passes tests,9599,simonw,closed,0,,,3268330,Datasette 1.0,1,2022-03-15T15:37:02Z,2022-03-19T04:04:50Z,2022-03-15T15:42:58Z,OWNER,,"> I've made a real mess of this. I'm going to revert Datasette`main` back to the last commit that passed the tests and try this again in a branch.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1657#issuecomment-1068125636_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1658/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1170554975,I_kwDOBm6k_c5FxUBf,1663,Document the internals that were used in datasette-hashed-urls,9599,simonw,closed,0,,,3268330,Datasette 1.0,2,2022-03-16T05:17:08Z,2022-03-19T04:04:50Z,2022-03-17T21:32:38Z,OWNER,,"The https://github.com/simonw/datasette-hashed-urls used a couple of currently undocumented features:
- `db.hash`
- `Datasette(..., immutables=[...])`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1663/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1108235694,I_kwDOBm6k_c5CDlWu,1603,A proper favicon,9599,simonw,closed,0,,,3268330,Datasette 1.0,19,2022-01-19T15:24:55Z,2022-03-19T04:04:49Z,2022-01-20T06:07:31Z,OWNER,,"Tips here: https://adamj.eu/tech/2022/01/18/how-to-add-a-favicon-to-your-django-site/ - I think a PNG served at `/favicon.ico` is the best option, since safari doesn't support SVG yet.
Relevant code: https://github.com/simonw/datasette/blob/cb29119db9115b1f40de2fb45263ed77e3bfbb3e/datasette/app.py#L182-L183
I can reuse the icon for https://datasette.io/desktop",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1603/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1114147905,I_kwDOBm6k_c5CaIxB,1612,Move canned queries closer to the SQL input area,639012,jsfenfen,closed,0,,,3268330,Datasette 1.0,5,2022-01-25T17:06:39Z,2022-03-19T04:04:49Z,2022-01-25T18:34:21Z,CONTRIBUTOR,,"*Original title: Consider placing example queries above the sql input?*
Hi! Have been enjoying deploying ad hoc datasettes for collaborators to pick over!
I keep finding myself manually ""fixing"" the database.html template so that the ""example queries"" (canned queries) appear directly *over* the sql box? So they are sorta more a suggestion for collaborators who aren't inclined to write their own queries?
My sense is any time I go to the trouble of writing canned queries my users should see 'em?
(( I have also considered a client-side reactive-ish option where selecting a query just places the raw SQL in the box and doesn't execute it, but this seems to end up being an inconvenience, rather than a teaching tool. ))
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1612/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1122413719,I_kwDOBm6k_c5C5qyX,1621,Test against Python 3.11 dev version,9599,simonw,closed,0,,,3268330,Datasette 1.0,0,2022-02-02T21:38:57Z,2022-03-19T04:04:49Z,2022-02-02T21:58:54Z,OWNER,,"To avoid another surprise like we got with 3.10: https://simonwillison.net/2021/Oct/9/finding-and-reporting-a-bug/
From a quick GitHub code search it looks like `3.11-dev` should work: https://cs.github.com/urllib3/urllib3/blob/7bec77e81aa0a194c98381053225813f5347c9d2/.github/workflows/ci.yml#L60",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1621/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1122416919,I_kwDOBm6k_c5C5rkX,1623,/-/patterns returns link: alternate JSON header to 404,9599,simonw,closed,0,,,3268330,Datasette 1.0,2,2022-02-02T21:42:49Z,2022-03-19T04:04:49Z,2022-02-02T21:48:56Z,OWNER,,"Bug from:
- #1620
```
% curl -s -I 'https://latest.datasette.io/-/patterns' | grep link
link: https://latest.datasette.io/-/patterns.json; rel=""alternate""; type=""application/json+datasette""
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1623/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1126604194,I_kwDOBm6k_c5DJp2i,1632,"datasette one.db one.db opens database twice, as one and one_2",9599,simonw,closed,0,,,3268330,Datasette 1.0,6,2022-02-07T23:14:47Z,2022-03-19T04:04:49Z,2022-02-07T23:50:01Z,OWNER,,"> ```
> % mkdir /tmp/data
> % cp ~/Dropbox/Development/datasette/fixtures.db /tmp/data
> % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852
> ...
> INFO: Uvicorn running on http://127.0.0.1:8852 (Press CTRL+C to quit)
> ^CINFO: Shutting down
> % datasette /tmp/data/*.db /tmp/data/created.db --create -p 8852
> ...
> INFO: 127.0.0.1:49533 - ""GET / HTTP/1.1"" 200 OK
> ```
> The first time I ran Datasette I got two databases - `fixtures` and `created`
>
> BUT... when I ran Datasette the second time it looked like this:
>
>
>
> This is the same result you get if you run:
>
> datasette /tmp/data/fixtures.db /tmp/data/created.db /tmp/data/created.db
>
> This is caused by this Datasette issue:
> - https://github.com/simonw/datasette/issues/509
>
> So... either I teach Datasette to de-duplicate multiple identical file paths passed to the command, or I can't use `/data/*.db` in the `Dockerfile` here and I need to go back to other solutions for the challenge described in this comment: https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1031971831
_Originally posted by @simonw in https://github.com/simonw/datasette-publish-fly/pull/12#issuecomment-1032029874_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1632/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1170497629,I_kwDOBm6k_c5FxGBd,1662,[feature request] Publish to fully static website,32609395,contrun,closed,0,,,,,1,2022-03-16T03:32:28Z,2022-03-19T00:42:23Z,2022-03-19T00:42:23Z,NONE,,"It seems currently all datasette publish requires a real backend server which is able to query the database and send results back to the frontend. There are a few projects to on-demand download a portion of data from the database from a sqlite lite database url, and present it directly to the user. These methods leverages web assembly under the hood. I think datasette is a perfect use case for this technology. Below are a few examples of querying sqlite database from frontend directly.
* [Using sqlite3 as a notekeeping document graph with automatic reference indexing](https://epilys.github.io/bibliothecula/notekeeping.html)
* [Hosting SQLite databases on Github Pages - (or any static file hoster) - phiresky's blog](https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/)
* [Static torrent website with peer-to-peer queries over BitTorrent on 2M records](https://boredcaveman.xyz/post/0x2_static-torrent-website-p2p-queries.html)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1662/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1170355774,I_kwDOBm6k_c5FwjY-,1661,Remove Hashed URL mode,9599,simonw,closed,0,,,3268330,Datasette 1.0,10,2022-03-15T23:13:56Z,2022-03-19T00:37:37Z,2022-03-19T00:37:36Z,OWNER,,"It's now handled by a plugin instead:
- #647
- https://github.com/simonw/datasette-hashed-urls/issues/3
https://github.com/simonw/datasette-hashed-urls
Sub-tasks:
- [x] Remove hashed URL mode implementation
- [x] Update documentation
- [x] Ensure `--setting hash_urls 1` shows a useful message",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1661/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1173017980,PR_kwDOBm6k_c40oRq-,1664,Remove hashed URL mode,9599,simonw,closed,0,,,,,7,2022-03-17T23:19:10Z,2022-03-19T00:12:04Z,2022-03-19T00:12:04Z,OWNER,simonw/datasette/pulls/1664,Refs #1661.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1664/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1065429936,I_kwDOBm6k_c4_gSuw,1532,Use datasette-table Web Component to guide the design of the JSON API for 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,4,2021-11-28T20:37:18Z,2022-03-16T20:13:34Z,,OWNER,,"I realized that one of the reasons I'm having trouble committing to nailing down the JSON API for 1.0 is that I don't use it much myself - I use the `?_shape=array` one quite often, but I don't have any projects that are using the default, more fully-featured API.
As an experiment I built a Web Component for embedding Datasette tables on pages - https://github.com/simonw/datasette-table - and I think it's actually going to be a really useful tool for helping me dog food the v1.0 API design.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1532/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
973139047,MDU6SXNzdWU5NzMxMzkwNDc=,1439,Rethink how .ext formats (v.s. ?_format=) works before 1.0,9599,simonw,closed,0,,,3268330,Datasette 1.0,48,2021-08-17T23:32:51Z,2022-03-15T20:51:26Z,2022-03-15T20:51:26Z,OWNER,,"Datasette currently has surprising special behaviour for if a table name ends in `.csv` - which can happen when a tool like `csvs-to-sqlite` creates tables that match the filename that they were imported from.
https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv illustrates this behaviour: it links to `.csv` and `.json` that look like this:
- https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=json
- https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv?_format=csv&_size=max
Where normally Datasette would add the `.csv` or `.json` extension to the path component of the URL (as seen on other pages such as https://latest.datasette.io/fixtures/facet_cities) here the [path_with_format() function](https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/utils/__init__.py#L710) notices that there is already a `.` in the path and instead adds `?_format=csv` to the query string instead.
The problem with this mechanism is that it's pretty surprising. Anyone writing external code to Datasette who wants to get back the `.csv` or `.json` version giving the URL to a table page will need to know about and implement this behaviour themselves. That's likely to cause all kinds of bugs in the future.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1439/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
531502365,MDU6SXNzdWU1MzE1MDIzNjU=,646,Make database level information from metadata.json available in the index.html template,18017473,lagolucas,open,0,,,3268330,Datasette 1.0,3,2019-12-02T19:55:10Z,2022-03-15T20:50:34Z,,NONE,,"Did a search on the issues here and didn't find anything related to what I want.
I want to have information that is on the database level of the JSON like title, source and source_url, and use it on the index page.
I tried some small tweaks on the python and html files, but failed to get that result.
Is there a way? Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/646/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
626593402,MDU6SXNzdWU2MjY1OTM0MDI=,780,Internals documentation for datasette.metadata() method,9599,simonw,open,0,,,3268330,Datasette 1.0,2,2020-05-28T15:14:22Z,2022-03-15T20:50:34Z,,OWNER,,https://github.com/simonw/datasette/blob/40885ef24e32d91502b6b8bbad1c7376f50f2830/datasette/app.py#L297-L328,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/780/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1054243511,I_kwDOBm6k_c4-1nq3,1509,Datasette 1.0 JSON API (and documentation),9599,simonw,open,0,,,3268330,Datasette 1.0,3,2021-11-15T23:22:45Z,2022-03-15T20:38:56Z,,OWNER,,"The new JSON API in a stable, documented form.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1509/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
646737558,MDU6SXNzdWU2NDY3Mzc1NTg=,870,Refactor default views to use register_routes,9599,simonw,open,0,,,,,10,2020-06-27T18:53:12Z,2022-03-15T20:07:18Z,,OWNER,,"It would be much cleaner if Datasette's default views were all registered using the new `register_routes()` plugin hook. Could dramatically reduce the code in `datasette/app.py`.
> The ideal fix here would be to rework my `BaseView` subclass mechanism to work with `register_routes()` so that those views don't have any special privileges above plugin-provided views.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/864#issuecomment-648580556_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/870/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1058072543,I_kwDOBm6k_c4_EOff,1518,Complete refactor of TableView and table.html template,9599,simonw,open,0,,,3268330,Datasette 1.0,45,2021-11-19T02:55:16Z,2022-03-15T18:35:49Z,,OWNER,,"Split from #878. The current `TableView` class is by far the most complex part of Datasette, and the most difficult to work on: https://github.com/simonw/datasette/blob/0.59.2/datasette/views/table.py
In #878 I started exploring a new pattern for building views. In doing so it became clear that `TableView` is the first beast that I need to slay - if I can refactor that into something neat the pattern for building other views will emerge as a natural consequence.
I've been trying to build this as a `register_routes()` plugin, as originally suggested in #870 - though unfortunately it looks like those plugins can't replace existing Datasette default views at the moment, see #1517. [UPDATE: I was wrong about this, plugins can over-ride default views just fine]
I also know that I want to have a fully documented template context for `table.html` as a major step on the way to Datasette 1.0, see #1510.
All of this adds up to the `TableView` factor being a major project that will unblock a whole flurry of other things - so I'm going to work on that in this separate issue.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1518/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1168995756,I_kwDOBm6k_c5FrXWs,1657,Tilde encoding: use ~ instead of - for dash-encoding,9599,simonw,closed,0,,,3268330,Datasette 1.0,12,2022-03-14T22:55:17Z,2022-03-15T18:25:11Z,2022-03-15T18:01:58Z,OWNER,,Refs #1439,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1657/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1168357113,PR_kwDOBm6k_c40ZRDA,1656,"Update pytest requirement from <7.1.0,>=5.2.2 to >=5.2.2,<7.2.0",49699333,dependabot[bot],closed,0,,,,,1,2022-03-14T13:11:53Z,2022-03-15T18:03:03Z,2022-03-15T18:03:02Z,CONTRIBUTOR,simonw/datasette/pulls/1656,"Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
Release notes
Sourced from pytest's releases .
7.1.0
pytest 7.1.0 (2022-03-13)
Breaking Changes
#8838 : As per our policy, the following features have been deprecated in the 6.X series and are now
removed:
pytest._fillfuncargs
function.
pytest_warning_captured
hook - use pytest_warning_recorded
instead.
-k -foobar
syntax - use -k 'not foobar'
instead.
-k foobar:
syntax.
pytest.collect
module - import from pytest
directly.
For more information consult
Deprecations and Removals in the docs.
#9437 : Dropped support for Python 3.6, which reached end-of-life at 2021-12-23.
Improvements
#5192 : Fixed test output for some data types where -v
would show less information.
Also, when showing diffs for sequences, -q
would produce full diffs instead of the expected diff.
#9362 : pytest now avoids specialized assert formatting when it is detected that the default __eq__
is overridden in attrs
or dataclasses
.
#9536 : When -vv
is given on command line, show skipping and xfail reasons in full instead of truncating them to fit the terminal width.
#9644 : More information about the location of resources that led Python to raise ResourceWarning
{.interpreted-text role="class"} can now
be obtained by enabling tracemalloc
{.interpreted-text role="mod"}.
See resource-warnings
{.interpreted-text role="ref"} for more information.
#9678 : More types are now accepted in the ids
argument to @pytest.mark.parametrize
.
Previously only [str]{.title-ref}, [float]{.title-ref}, [int]{.title-ref} and [bool]{.title-ref} were accepted;
now [bytes]{.title-ref}, [complex]{.title-ref}, [re.Pattern]{.title-ref}, [Enum]{.title-ref} and anything with a [__name__]{.title-ref} are also accepted.
#9692 : pytest.approx
{.interpreted-text role="func"} now raises a TypeError
{.interpreted-text role="class"} when given an unordered sequence (such as set
{.interpreted-text role="class"}).
Note that this implies that custom classes which only implement __iter__
and __len__
are no longer supported as they don't guarantee order.
Bug Fixes
#8242 : The deprecation of raising unittest.SkipTest
{.interpreted-text role="class"} to skip collection of
tests during the pytest collection phase is reverted - this is now a supported
feature again.
#9493 : Symbolic link components are no longer resolved in conftest paths.
This means that if a conftest appears twice in collection tree, using symlinks, it will be executed twice.
... (truncated)
Commits
1dbffcc
[pre-commit.ci] auto fixes from pre-commit.com hooks
d53a5fb
Prepare release version 7.1.0
d306ec0
Update upcoming trainings (#9744 )
3e4c14b
Merge pull request #9751 from fabianegli/main
7f924b1
Fix typo in deprecation documentation
4a8f8ad
build(deps): Bump django from 4.0.2 to 4.0.3 in /testing/plugins_integration ...
c0fd2d8
build(deps): Bump pytest-asyncio from 0.18.1 to 0.18.2 in /testing/plugins_in...
843e018
Merge pull request #9732 from nicoddemus/9730-toml-failure
bc43d66
[automated] Update plugin list (#9733 )
e38d1ca
Improve error message for malformed pyproject.toml files
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1656/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1169895600,PR_kwDOBm6k_c40eW7C,1659,Tilde encoding,9599,simonw,closed,0,,,,,1,2022-03-15T16:19:07Z,2022-03-15T18:01:58Z,2022-03-15T18:01:57Z,OWNER,simonw/datasette/pulls/1659,Refs #1657,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1659/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
930807135,MDU6SXNzdWU5MzA4MDcxMzU=,1384,Plugin hook for dynamic metadata,9599,simonw,open,0,,,,,22,2021-06-26T22:36:03Z,2022-03-14T00:36:42Z,,OWNER,,"@brandonrobertz contributed an implementation of this in PR #1368, which I just merged. Opening this ticket to track further work on this before it goes out in a Datasette release (likely preceded by an alpha).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1384/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1131295060,I_kwDOBm6k_c5DbjFU,1634,Update Dockerfile generated by `datasette publish`,9599,simonw,open,0,,,3268330,Datasette 1.0,4,2022-02-11T00:07:26Z,2022-03-11T17:38:08Z,,OWNER,,"The generated `Dockerfile` currently looks something like this:
```Dockerfile
FROM python:3.8
COPY . /app
WORKDIR /app
ENV DATASETTE_SECRET 'edab49cbc5d5f6f33238f54852037e3fee710821960b73edd2ce743454182ae2'
RUN pip install -U datasette datasette-auth-passwords datasette-tiddlywiki datasette-graphql
RUN datasette inspect fixtures.db other.db --inspect-file inspect-data.json
ENV PORT 8080
EXPOSE 8080
CMD datasette serve --host 0.0.0.0 -i fixtures.db -i other.db --cors --inspect-file inspect-data.json --metadata metadata.json --create --port $PORT /data/*.db
```
This is still on Python 3.8, and it generates a pretty large image compared to the `Dockerfile` used for https://hub.docker.com/datasetteproject/datasette - https://github.com/simonw/datasette/blob/0.60.2/Dockerfile
Here's the code that generates it: https://github.com/simonw/datasette/blob/7d24fd405f3c60e4c852c5d746c91aa2ba23cf5b/datasette/utils/__init__.py#L389-L400",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1634/reactions"", ""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
678760988,MDU6SXNzdWU2Nzg3NjA5ODg=,932,End-user documentation,9599,simonw,open,0,,,3268330,Datasette 1.0,6,2020-08-13T22:04:39Z,2022-03-08T15:20:48Z,,OWNER,,"Datasette's documentation is aimed at people who install and configure it.
What about end users of preconfigured and deployed Datasette instances?
Something that can be linked to from the Datasette UI would be really useful.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/932/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1154399841,I_kwDOBm6k_c5Ezr5h,1645,"Sensible `cache-control` headers for static assets, including those served by plugins",697092,curiousleo,open,0,,,3268330,Datasette 1.0,4,2022-02-28T18:12:03Z,2022-03-08T02:59:29Z,,NONE,,"## What I'm seeing
With `default_cache_ttl = 86400`, I see the following:
A table view returns `Cache-control: max-age=86400`:
![Screenshot_20220228_190000](https://user-images.githubusercontent.com/697092/156034352-4d64683e-39c8-49af-81df-0217a5957bbd.png)
A static asset returns no `Cache-control` header:
![Screenshot_20220228_185933](https://user-images.githubusercontent.com/697092/156034363-d0b03cc2-5889-4ed2-b601-8c1846b8469a.png)
## What I expected to see
I expected the static asset to return a `Cache-control` header indicating that this response can be cached.
## Why this matters
I'm productionising a Datasette deployment right now and was looking into putting it behind a Varnish instance. I was surprised to see requests for static assets being served from Datasette rather than Varnish, this is what led me to look more closely at the response headers.
While Datasette serves those static assets pretty quickly, I don't see why Datasette should serve them. By their nature, static assets like images and JS files are very cacheable, so it should be easy to serve them from a cache like Varnish.
(Note that Varnish can easily be configured to override this header, enabling caching for static assets. But it would be better if this override was not necessary.)
## Discussion
It seems clear to me that serving static assets without a `Cache-control` header is not ideal.
I see two options here:
A. Static assets use the same logic as table / SQL views to set the `Cache-control` header based on `default_cache_ttl`.
B. An additional setting for static assets is introduced (`default_static_cache_ttl`, say).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1645/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1161969891,I_kwDOBm6k_c5FQkDj,1654,Adopt a code of conduct,9599,simonw,closed,0,,,,,5,2022-03-07T22:00:24Z,2022-03-07T22:19:35Z,2022-03-07T22:19:35Z,OWNER,,"This is long overdue, especially given the size of the project now.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1654/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1161937073,I_kwDOBm6k_c5FQcCx,1653,Mechanism to default a table to sorting by multiple columns,9599,simonw,open,0,,,,,2,2022-03-07T21:20:11Z,2022-03-07T21:23:39Z,,OWNER,,"### Discussed in https://github.com/simonw/datasette/discussions/1652
Originally posted by **zaneselvans** March 7, 2022
It's easy to tell datasette to sort tables using a single column, as [described in the docs](https://docs.datasette.io/en/stable/metadata.html#setting-a-default-sort-order):
```yaml
databases:
ferc1:
tables:
f1_edcfu_epda:
sort: created_time
```
But is there some way to tell it to sort using a composite key, like you would in an `ORDER BY` clause instead? For example, the way it's being done **[in this query](https://data.catalyst.coop/ferc1?sql=select%0D%0A++rowid%2C%0D%0A++respondent_id%2C%0D%0A++report_year%2C%0D%0A++spplmnt_num%2C%0D%0A++row_number%2C%0D%0A++row_seq%2C%0D%0A++row_prvlg%2C%0D%0A++acct_num%2C%0D%0A++depr_plnt_base%2C%0D%0A++est_avg_srvce_lf%2C%0D%0A++net_salvage%2C%0D%0A++apply_depr_rate%2C%0D%0A++mrtlty_crv_typ%2C%0D%0A++avg_remaining_lf%2C%0D%0A++report_prd%0D%0Afrom%0D%0A++f1_edcfu_epda%0D%0Awhere%0D%0A++respondent_id+%3D+210%0D%0A++AND+report_year+%3D+2020%0D%0Aorder+by%0D%0A++report_year%2C+report_prd%2C+respondent_id%2C+spplmnt_num%2C+row_number%0D%0Alimit%0D%0A++1000)** on our Datasette?
```sql
SELECT
respondent_id,
report_year,
spplmnt_num,
row_number,
row_seq,
row_prvlg,
acct_num,
depr_plnt_base,
est_avg_srvce_lf,
net_salvage,
apply_depr_rate,
mrtlty_crv_typ,
avg_remaining_lf,
report_prd
FROM
f1_edcfu_epda
WHERE
respondent_id = 210
AND report_year = 2020
ORDER BY
report_year, report_prd, respondent_id, spplmnt_num, row_number
LIMIT
1000
```
The problem here is that by default it's using `rowid` (the SQLite assigned autoincrementing integer key) to order the records, but the table **should** have a natural composite primary key, but the original database that this data is being migrated from doesn't enforce unique primary keys, so there are dupes, and we don't want to drop those rows, and the records are somehow getting jumbled in the database (the `rowid` ordering isn't lined up with the expected ordering based on the composite primary key, though it's close) and this jumbling is confusing to users that expect to see the data ordered based on the natural primary key.
I've tried setting the `sort` metadata parameter to a list of column names, a tuple of column names, a quoted string of comma-separated column names, a quoted string of a tuple of column names...
```yaml
databases:
ferc1:
tables:
f1_edcfu_epda:
sort: ""(report_year, report_prd, respondent_id, spplmnt_num, row_number)""
```
and they all give me server errors like:
```
Cannot sort table by (report_year, report_prd, respondent_id, spplmnt_num, row_number)
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1653/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1160750713,I_kwDOBm6k_c5FL6Z5,1650,Implement redirects from old % encoding to new dash encoding,9599,simonw,closed,0,,,3268330,Datasette 1.0,5,2022-03-06T23:40:02Z,2022-03-07T19:26:15Z,2022-03-07T19:26:14Z,OWNER,,"> One big advantage to this scheme is that redirecting old links to `%2F` pages (e.g. https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators) is easy - if you see a `%` in the `raw_path`, redirect to that page with the `%` replaced by `-`.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1439#issuecomment-1060044007_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1650/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1160432941,PR_kwDOBm6k_c4z_p6S,1648,Use dash encoding for table names and row primary keys in URLs,9599,simonw,closed,0,,,,,7,2022-03-05T19:50:45Z,2022-03-07T15:38:30Z,2022-03-07T15:38:30Z,OWNER,simonw/datasette/pulls/1648,"Refs #1439.
- [x] Build `dash_encode` / `dash_decode` functions
- [x] Use dash encoding for row primary keys
- [x] Use dash encoding for `?_next=` pagination tokens
- [x] Use dash encoding for table names in URLs
- [x] Use dash encoding for database name
- ~~Implement redirects from previous `%` URLs that replace those with `-`~~ - separate issue: #1650",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1648/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1160677684,PR_kwDOBm6k_c40AW_v,1649,Add /opt/homebrew to where spatialite extension can be found,2182,danp,closed,0,,,,,1,2022-03-06T18:09:35Z,2022-03-06T22:46:00Z,2022-03-06T19:39:15Z,CONTRIBUTOR,simonw/datasette/pulls/1649,"Helps homebrew on Apple Silicon setups find spatialite without needing
a full path.
Similar to #1114",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1649/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1098275181,PR_kwDOBm6k_c4wwNCl,1589,Typo in docs about default redirect status code,3556,davidbgk,closed,0,,,,,1,2022-01-10T19:14:36Z,2022-03-06T02:27:49Z,2022-03-06T01:58:32Z,CONTRIBUTOR,simonw/datasette/pulls/1589,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1589/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1108084641,PR_kwDOBm6k_c4xQ0uZ,1602,"Update pytest-timeout requirement from <2.1,>=1.4.2 to >=1.4.2,<2.2",49699333,dependabot[bot],closed,0,,,,,1,2022-01-19T13:11:50Z,2022-03-06T01:41:50Z,2022-03-06T01:41:49Z,CONTRIBUTOR,simonw/datasette/pulls/1602,"Updates the requirements on [pytest-timeout](https://github.com/pytest-dev/pytest-timeout) to permit the latest version.
Commits
8e4800e
Fixup readme
dc1efca
2.1.0 release
dd9d608
Add custom hooks specifications for overriding setup_timeout and teardown_tim...
ed8ecd6
module names, they're difficult
3ab4319
Add changelog
4f7ebae
Replace deprecated py.io.get_terminal_width() with shutil.get_terminal_size()...
b8a2fa6
Prep release
951972d
Update changelog
748a9c3
Making detection of whether a debugger is currently attached more flexible. (...
f8a46a1
Github removed the git protocol (#112 )
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1602/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1112633417,PR_kwDOBm6k_c4xfryi,1610,"Update asgiref requirement from <3.5.0,>=3.2.10 to >=3.2.10,<3.6.0",49699333,dependabot[bot],closed,0,,,,,0,2022-01-24T13:14:18Z,2022-03-06T01:30:27Z,2022-03-06T01:30:27Z,CONTRIBUTOR,simonw/datasette/pulls/1610,"Updates the requirements on [asgiref](https://github.com/django/asgiref) to permit the latest version.
Changelog
Sourced from asgiref's changelog .
3.5.0 (2022-01-22)
Python 3.6 is no longer supported, and asyncio calls have been changed to
use only the modern versions of the APIs as a result
Several causes of RuntimeErrors in cases where an event loop was assigned
to a thread but not running
Speed improvements in the Local class
3.4.1 (2021-07-01)
Fixed an issue with the deadlock detection where it had false positives
during exception handling.
3.4.0 (2021-06-27)
Calling sync_to_async directly from inside itself (which causes a deadlock
when in the default, thread-sensitive mode) now has deadlock detection.
asyncio usage has been updated to use the new versions of get_event_loop,
ensure_future, wait and gather, avoiding deprecation warnings in Python 3.10.
Python 3.6 installs continue to use the old versions; this is only for 3.7+
sync_to_async and async_to_sync now have improved type hints that pass
through the underlying function type correctly.
All Websocket* types are now spelled WebSocket, to match our specs and the
official spelling. The old names will work until release 3.5.0, but will
raise deprecation warnings.
The typing for WebSocketScope and HTTPScope's extensions
key has been
fixed.
3.3.4 (2021-04-06)
The async_to_sync type error is now a warning due the high false negative
rate when trying to detect coroutine-returning callables in Python.
3.3.3 (2021-04-06)
... (truncated)
Commits
8b61513
Releasing 3.5.0
b2e1c9d
Fixed pytest_asyncio deprecation warning.
2eda551
Added testing for Python 3.10.
02fecb6
Drop Python 3.6 (#307 )
6689c0a
Added stacklevel to warning in AsyncToSync.
4364f9b
Changed how StatelessServer handles event loops
7bc055c
Update implementations.rst (#295 )
c758984
Move current_task import choice to module definition time
dfe87b2
Fixed #292 : Use get_event_loop in class-level code
b3a65e3
Removed class variable which has been unused since a0bbe90
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1610/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1124191982,PR_kwDOBm6k_c4yFTCp,1629,"Update pytest requirement from <6.3.0,>=5.2.2 to >=5.2.2,<7.1.0",49699333,dependabot[bot],closed,0,,,,,1,2022-02-04T13:14:10Z,2022-03-06T01:30:06Z,2022-03-06T01:30:06Z,CONTRIBUTOR,simonw/datasette/pulls/1629,"Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
Release notes
Sourced from pytest's releases .
7.0.0
pytest 7.0.0 (2022-02-03)
(Please see the full set of changes for this release also in the 7.0.0rc1 notes below )
Deprecations
#9488 : If custom subclasses of nodes like pytest.Item
{.interpreted-text role="class"} override the
__init__
method, they should take **kwargs
. See
uncooperative-constructors-deprecated
{.interpreted-text role="ref"} for details.
Note that a deprection warning is only emitted when there is a conflict in the
arguments pytest expected to pass. This deprecation was already part of pytest
7.0.0rc1 but wasn't documented.
Bug Fixes
#9355 : Fixed error message prints function decorators when using assert in Python 3.8 and above.
#9396 : Ensure pytest.Config.inifile
{.interpreted-text role="attr"} is available during the pytest_cmdline_main <_pytest.hookspec.pytest_cmdline_main>
{.interpreted-text role="func"} hook (regression during 7.0.0rc1
).
Improved Documentation
#9404 : Added extra documentation on alternatives to common misuses of [pytest.warns(None)]{.title-ref} ahead of its deprecation.
#9505 : Clarify where the configuration files are located. To avoid confusions documentation mentions
that configuration file is located in the root of the repository.
Trivial/Internal Changes
#9521 : Add test coverage to assertion rewrite path.
pytest 7.0.0rc1 (2021-12-06)
Breaking Changes
#7259 : The Node.reportinfo() <non-python tests>
{.interpreted-text role="ref"} function first return value type has been expanded from [py.path.local | str]{.title-ref} to [os.PathLike[str] | str]{.title-ref}.
Most plugins which refer to [reportinfo()]{.title-ref} only define it as part of a custom pytest.Item
{.interpreted-text role="class"} implementation.
Since [py.path.local]{.title-ref} is a [os.PathLike[str]]{.title-ref}, these plugins are unaffacted.
Plugins and users which call [reportinfo()]{.title-ref}, use the first return value and interact with it as a [py.path.local]{.title-ref}, would need to adjust by calling [py.path.local(fspath)]{.title-ref}.
Although preferably, avoid the legacy [py.path.local]{.title-ref} and use [pathlib.Path]{.title-ref}, or use [item.location]{.title-ref} or [item.path]{.title-ref}, instead.
Note: pytest was not able to provide a deprecation period for this change.
... (truncated)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1629/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1125973221,PR_kwDOBm6k_c4yK44E,1631,"Update pytest-asyncio requirement from <0.17,>=0.10 to >=0.10,<0.19",49699333,dependabot[bot],closed,0,,,,,1,2022-02-07T13:13:19Z,2022-03-06T01:29:54Z,2022-03-06T01:29:53Z,CONTRIBUTOR,simonw/datasette/pulls/1631,"Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
Release notes
Sourced from pytest-asyncio's releases .
pytest-asyncio 0.18.0
title: 'pytest-asyncio: pytest support for asyncio'
pytest-asyncio is an Apache2 licensed library, written in Python, for
testing asyncio code with pytest.
asyncio code is usually written in the form of coroutines, which makes
it slightly more difficult to test using normal testing tools.
pytest-asyncio provides useful fixtures and markers to make testing
easier.
@pytest.mark.asyncio
async def test_some_asyncio_code():
res = await library.do_something()
assert b"expected result" == res
pytest-asyncio has been strongly influenced by
pytest-tornado .
Features
fixtures for creating and injecting versions of the asyncio event
loop
fixtures for injecting unused tcp/udp ports
pytest markers for treating tests as asyncio coroutines
easy testing with non-default event loops
support for [async def]{.title-ref} fixtures and async generator
fixtures
support auto mode to handle all async fixtures and tests
automatically by asyncio; provide strict mode if a test suite
should work with different async frameworks simultaneously, e.g.
asyncio
and trio
.
Installation
... (truncated)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1631/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1160407071,I_kwDOBm6k_c5FKmgf,1647,Test failures with SQLite 3.37.0+ due to column affinity case,9599,simonw,closed,0,,,,,5,2022-03-05T17:37:46Z,2022-03-05T19:56:28Z,2022-03-05T19:47:04Z,OWNER,,"These three tests are failing on my local machine:
```
FAILED tests/test_internals_database.py::test_table_column_details[facetable-expected0] - AssertionError: assert [Column(cid=0, name='pk', type='INTEGER', no...
FAILED tests/test_internals_database.py::test_table_column_details[sortable-expected1] - AssertionError: assert [Column(cid=0, name='pk1', type='varchar(30)'...
FAILED tests/test_table_html.py::test_sort_links - AssertionError: assert [{'a_href': None,\n 'attrs': {'class': ['col-Link'],\n 'data-column': '...
```
I ran `pytest --lf -vv` and the output had things like this in it:
```
E - Column(cid=1, name='created', type='text', notnull=0, default_value=None, is_pk=0, hidden=0),
E ? ^^^^
E + Column(cid=1, name='created', type='TEXT', notnull=0, default_value=None, is_pk=0, hidden=0),
...
E {'a_href': '/fixtures/sortable?_sort=sortable_with_nulls_2',
E 'attrs': {'class': ['col-sortable_with_nulls_2'],
E 'data-column': 'sortable_with_nulls_2',
E 'data-column-not-null': '0',
E - 'data-column-type': 'real',
E ? ^^^^
E + 'data-column-type': 'REAL',
E ? ^^^^
```
Something is causing column types to come back in uppercase where previously they were lowercase.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1647/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1148725876,I_kwDOBm6k_c5EeCp0,1640,"Support static assets where file length may change, e.g. logs",57859326,broccolihighkicks,open,0,,,,,2,2022-02-24T00:34:42Z,2022-03-05T01:19:25Z,,NONE,,"This is a bit of an oxymoron.
I am serving a log.txt file for a background process using the Datasette --static CLI. This is useful as I can observe a background process from the web UI to see any errors that occur (instead of spelunking the logs via docker exec/ssh etc).
I get this error, which I think is because Datasette assumes that the size of the content does not change (but appending new log lines means the content length changes).
```python
Traceback (most recent call last):
File ""/usr/local/lib/python3.9/site-packages/datasette/app.py"", line 1181, in route_path
response = await view(request, send)
File ""/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py"", line 305, in inner_static
await asgi_send_file(send, full_path, chunk_size=chunk_size)
File ""/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py"", line 280, in asgi_send_file
await send(
File ""/usr/local/lib/python3.9/site-packages/asgi_csrf.py"", line 104, in wrapped_send
await send(event)
File ""/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py"", line 460, in send
output = self.conn.send(event)
File ""/usr/local/lib/python3.9/site-packages/h11/_connection.py"", line 468, in send
data_list = self.send_with_data_passthrough(event)
File ""/usr/local/lib/python3.9/site-packages/h11/_connection.py"", line 501, in send_with_data_passthrough
writer(event, data_list.append)
File ""/usr/local/lib/python3.9/site-packages/h11/_writers.py"", line 58, in __call__
self.send_data(event.data, write)
File ""/usr/local/lib/python3.9/site-packages/h11/_writers.py"", line 78, in send_data
raise LocalProtocolError(""Too much data for declared Content-Length"")
h11._util.LocalProtocolError: Too much data for declared Content-Length
ERROR: Exception in ASGI application
Traceback (most recent call last):
File ""/usr/local/lib/python3.9/site-packages/datasette/app.py"", line 1181, in route_path
response = await view(request, send)
File ""/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py"", line 305, in inner_static
await asgi_send_file(send, full_path, chunk_size=chunk_size)
File ""/usr/local/lib/python3.9/site-packages/datasette/utils/asgi.py"", line 280, in asgi_send_file
await send(
File ""/usr/local/lib/python3.9/site-packages/asgi_csrf.py"", line 104, in wrapped_send
await send(event)
File ""/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py"", line 460, in send
output = self.conn.send(event)
File ""/usr/local/lib/python3.9/site-packages/h11/_connection.py"", line 468, in send
data_list = self.send_with_data_passthrough(event)
File ""/usr/local/lib/python3.9/site-packages/h11/_connection.py"", line 501, in send_with_data_passthrough
writer(event, data_list.append)
File ""/usr/local/lib/python3.9/site-packages/h11/_writers.py"", line 58, in __call__
self.send_data(event.data, write)
File ""/usr/local/lib/python3.9/site-packages/h11/_writers.py"", line 78, in send_data
raise LocalProtocolError(""Too much data for declared Content-Length"")
h11._util.LocalProtocolError: Too much data for declared Content-Length
```
Thanks, I am finding Datasette very useful.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1640/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1152072027,I_kwDOBm6k_c5Eqzlb,1642,Dependency issue with asgiref and uvicorn,9599,simonw,closed,0,,,,,1,2022-02-26T18:00:35Z,2022-03-05T01:11:27Z,2022-03-05T01:11:17Z,OWNER,,"```
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
datasette 0.60.2 requires asgiref<3.5.0,>=3.2.10, but you'll have asgiref 3.5.0 which is incompatible.
```
That's after I forced an upgrade of `uvicorn` due to this warning:
```
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
uvicorn 0.13.1 requires click==7.*, but you'll have click 8.0.4 which is incompatible.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1642/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1149310456,I_kwDOBm6k_c5EgRX4,1641,Tweak mobile keyboard settings,9599,simonw,open,0,,,,,1,2022-02-24T13:47:10Z,2022-02-24T13:49:26Z,,OWNER,,"https://developer.apple.com/library/archive/documentation/StringsTextFonts/Conceptual/TextAndWebiPhoneOS/KeyboardManagement/KeyboardManagement.html#//apple_ref/doc/uid/TP40009542-CH5-SW12
`autocorrect=""off""` is worth experimenting with.
Twitter: https://twitter.com/forestgregg/status/1496842959563726852",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1641/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1148638868,I_kwDOBm6k_c5EdtaU,1639,Make datasette-redirect-forbidden unneccessary,9599,simonw,open,0,,,,,0,2022-02-23T22:18:46Z,2022-02-23T22:18:46Z,,OWNER,,"I wrote `datasette-redirect-forbidden` today because I needed 403 errors to redirect to `/-/login` and it was the quickest way to solve that problem.
This should be a feature of Datasette core.
- https://github.com/simonw/datasette-redirect-forbidden/issues/2",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1639/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
677272618,MDU6SXNzdWU2NzcyNzI2MTg=,928,Test failures caused by failed attempts to mock pip,9599,simonw,closed,0,,,,,4,2020-08-11T23:53:18Z,2022-02-23T16:19:47Z,2020-08-12T00:07:49Z,OWNER,,"Errors like this one:
https://github.com/simonw/datasette/pull/927/checks?check_run_id=973559696
```
2020-08-11T23:36:39.8801334Z =================================== FAILURES ===================================
2020-08-11T23:36:39.8802411Z _________________________________ test_install _________________________________
2020-08-11T23:36:39.8803242Z
2020-08-11T23:36:39.8804935Z thing =
2020-08-11T23:36:39.8806663Z comp = 'main', import_path = 'pip._internal.cli.main'
2020-08-11T23:36:39.8807696Z
2020-08-11T23:36:39.8808728Z def _dot_lookup(thing, comp, import_path):
2020-08-11T23:36:39.8810573Z try:
2020-08-11T23:36:39.8812262Z > return getattr(thing, comp)
2020-08-11T23:36:39.8817136Z E AttributeError: module 'pip._internal.cli' has no attribute 'main'
2020-08-11T23:36:39.8843043Z
2020-08-11T23:36:39.8855951Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/unittest/mock.py:1215: AttributeError
2020-08-11T23:36:39.8873372Z
2020-08-11T23:36:39.8877803Z During handling of the above exception, another exception occurred:
2020-08-11T23:36:39.8906532Z
2020-08-11T23:36:39.8925767Z def get_src_prefix():
2020-08-11T23:36:39.8928277Z # type: () -> str
2020-08-11T23:36:39.8930068Z if running_under_virtualenv():
2020-08-11T23:36:39.8949721Z src_prefix = os.path.join(sys.prefix, 'src')
2020-08-11T23:36:39.8951813Z else:
2020-08-11T23:36:39.8969014Z # FIXME: keep src in cwd for now (it is not a temporary folder)
2020-08-11T23:36:39.9012110Z try:
2020-08-11T23:36:39.9013489Z > src_prefix = os.path.join(os.getcwd(), 'src')
2020-08-11T23:36:39.9014538Z E FileNotFoundError: [Errno 2] No such file or directory
2020-08-11T23:36:39.9016122Z
2020-08-11T23:36:39.9017617Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/pip/_internal/locations.py:50: FileNotFoundError
2020-08-11T23:36:39.9018802Z
2020-08-11T23:36:39.9020070Z During handling of the above exception, another exception occurred:
2020-08-11T23:36:39.9020930Z
2020-08-11T23:36:39.9022275Z args = (), keywargs = {}
2020-08-11T23:36:39.9023183Z
2020-08-11T23:36:39.9024077Z @wraps(func)
2020-08-11T23:36:39.9024984Z def patched(*args, **keywargs):
2020-08-11T23:36:39.9028770Z > with self.decoration_helper(patched,
2020-08-11T23:36:39.9031861Z args,
2020-08-11T23:36:39.9038358Z keywargs) as (newargs, newkeywargs):
2020-08-11T23:36:39.9039654Z
2020-08-11T23:36:39.9040566Z /opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/unittest/mock.py:1322:
2020-08-11T23:36:39.9041492Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/928/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1142107925,I_kwDOBm6k_c5EEy8V,1638,`filters_from_request` plugin hook docs should mention that returning an async function is allowed,9599,simonw,open,0,,,,,0,2022-02-18T00:08:26Z,2022-02-18T00:08:26Z,,OWNER,,"https://docs.datasette.io/en/stable/plugin_hooks.html#filters-from-request-request-database-table-datasette doesn't mention that you can return an `async` function - but you can, and in fact Datasette itself uses that here: https://github.com/simonw/datasette/blob/aa7f0037a46eb76ae6fe9bf2a1f616c58738ecdf/datasette/filters.py#L43-L47",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1638/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
335200136,MDU6SXNzdWUzMzUyMDAxMzY=,327,Explore if SquashFS can be used to shrink size of packaged Docker containers,9599,simonw,open,0,,,,,4,2018-06-24T18:15:16Z,2022-02-17T23:37:24Z,,OWNER,,"Inspired by this article: https://cldellow.com/2018/06/22/sqlite-parquet-vtable.html#sqlite-database-indexed--squashed
https://en.wikipedia.org/wiki/SquashFS is ""a compressed read-only file system for Linux"" - which means it could be a really nice fit for Datasette and its read-only SQLite databases.
It would be interesting to explore a Dockerfile recipe that used SquashFS to compress the SQLite database file that was bundled up by `datasette package` and friends.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/327/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
764059235,MDU6SXNzdWU3NjQwNTkyMzU=,1143,"More flexible CORS support in core, to encourage good security practices",114388,yurivish,open,0,,,3268330,Datasette 1.0,6,2020-12-12T17:06:35Z,2022-02-13T17:41:17Z,,NONE,,"It would be nice if the `--cors` option accepted an origin regex to more securely allow secure local development.
As an example, Observable notebooks namespace every user's notebooks by their username and user content is served from username.observableusercontent.com, so you would set `--cors-origin username.observableusercontent.com` to restrict access to a local development Datasette instance to only your own notebooks, rather than exposing the data to any website that makes a request.
Thank you for all of your work on Datasette!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1143/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1109783030,I_kwDOBm6k_c5CJfH2,1607,More detailed information about installed SpatiaLite version,9599,simonw,closed,0,,,3268330,Datasette 1.0,2,2022-01-20T21:28:03Z,2022-02-09T06:42:02Z,2022-02-09T06:32:28Z,OWNER,,"https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.0.0.html#version has a whole bunch of interesting functions for things like `freexl_version()` and `geos_version()` and `HasMathSQL()` and suchlike.
These could be shown on the `/-/versions` page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1607/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1101705012,PR_kwDOBm6k_c4w7eqc,1593,"Update pytest-asyncio requirement from <0.17,>=0.10 to >=0.10,<0.18",49699333,dependabot[bot],closed,0,,,,,2,2022-01-13T13:11:50Z,2022-02-07T13:13:24Z,2022-02-07T13:13:23Z,CONTRIBUTOR,simonw/datasette/pulls/1593,"Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
Release notes
Sourced from pytest-asyncio's releases .
pytest-asyncio 0.17.0
title: 'pytest-asyncio: pytest support for asyncio'
pytest-asyncio is an Apache2 licensed library, written in Python, for
testing asyncio code with pytest.
asyncio code is usually written in the form of coroutines, which makes
it slightly more difficult to test using normal testing tools.
pytest-asyncio provides useful fixtures and markers to make testing
easier.
@pytest.mark.asyncio
async def test_some_asyncio_code():
res = await library.do_something()
assert b"expected result" == res
pytest-asyncio has been strongly influenced by
pytest-tornado .
Features
fixtures for creating and injecting versions of the asyncio event
loop
fixtures for injecting unused tcp/udp ports
pytest markers for treating tests as asyncio coroutines
easy testing with non-default event loops
support for [async def]{.title-ref} fixtures and async generator
fixtures
support auto mode to handle all async fixtures and tests
automatically by asyncio; provide strict mode if a test suite
should work with different async frameworks simultaneously, e.g.
asyncio
and trio
.
Installation
... (truncated)
Commits
2e2d5d2
Bump to 0.17 release
90436c9
Fix pandoc installation procedure
d291c66
Convert README.rst to Markdown for making githun release
141937b
Fix release artifacts
696cf7d
Fix trove classifier for asyncio
8ccb306
Build on tag
cd84987
Release process automation (#252 )
2eb12a7
Setup GitHub Workflows linter and yaml-reformatter (#253 )
d28b826
Bump codecov/codecov-action from 1 to 2.1.0 (#251 )
2f523ba
Configure dependabot version updater (#250 )
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1593/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
779691739,MDU6SXNzdWU3Nzk2OTE3Mzk=,1176,"Policy on documenting ""public"" datasette.utils functions",9599,simonw,closed,0,,,3268330,Datasette 1.0,13,2021-01-05T22:55:25Z,2022-02-07T06:43:32Z,2022-02-07T06:42:58Z,OWNER,,"https://github.com/simonw/datasette-css-properties starts [like this](https://github.com/simonw/datasette-css-properties/blob/0.1/datasette_css_properties/__init__.py#L1-L3):
```python
from datasette import hookimpl
from datasette.utils.asgi import Response
from datasette.utils import escape_css_string, to_css_class
```
`escape_css_string` and `to_css_class` are not documented, which means relying on them is risky since there's no promise that they won't change.
Would be good to figure out a policy on this, and maybe promote some of them to ""documented"" status.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1176/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1125576543,I_kwDOBm6k_c5DFu9f,1630,Review datasette.utils and decide which functions should be documented for 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2022-02-07T06:39:52Z,2022-02-07T06:39:52Z,,OWNER,,"Follows:
- #1176",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1630/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
688622148,MDU6SXNzdWU2ODg2MjIxNDg=,957,Simplify imports of common classes,9599,simonw,closed,0,,,3268330,Datasette 1.0,7,2020-08-29T23:44:04Z,2022-02-06T06:36:41Z,2022-02-06T06:34:37Z,OWNER,,"There are only a few classes that plugins need to import. It would be nice if these imports were as short and memorable as possible.
For example:
```python
from datasette.app import Datasette
from datasette.utils.asgi import Response
```
Could both become:
```python
from datasette import Datasette
from datasette import Response
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/957/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1087181951,I_kwDOBm6k_c5AzRR_,1576,Traces should include SQL executed by subtasks created with `asyncio.gather`,9599,simonw,closed,0,,,3268330,Datasette 1.0,12,2021-12-22T20:52:02Z,2022-02-05T05:21:35Z,2022-02-05T05:19:53Z,OWNER,,"I tried running some parallel SQL queries using `asyncio.gather()` but the SQL that was executed didn't show up in the trace rendered by https://datasette.io/plugins/datasette-pretty-traces
I realized that was because traces are keyed against the current task ID, which changes when a sub-task is run using `asyncio.gather` or similar.
The faceting and suggest faceting queries are missing from this trace:
![image](https://user-images.githubusercontent.com/9599/147153855-2d611f07-922a-4d18-9e6e-4be89e010dc4.png)
> The reason they aren't showing up in the traces is that traces are stored just for the currently executing `asyncio` task ID: https://github.com/simonw/datasette/blob/ace86566b28280091b3844cf5fbecd20158e9004/datasette/tracer.py#L13-L25
>
> This is so traces for other incoming requests don't end up mixed together. But there's no current mechanism to track async tasks that are effectively ""child tasks"" of the current request, and hence should be tracked the same.
>
> https://stackoverflow.com/a/69349501/6083 suggests that you pass the task ID as an argument to the child tasks that are executed using `asyncio.gather()` to work around this kind of problem.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1518#issuecomment-999870993_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1576/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
734777631,MDU6SXNzdWU3MzQ3Nzc2MzE=,1080,"""View all"" option for facets, to provide a (paginated) list of ALL of the facet counts plus a link to view them",9599,simonw,open,0,,,3268330,Datasette 1.0,7,2020-11-02T19:55:06Z,2022-02-04T06:25:18Z,,OWNER,,Can use `/database/-/...` namespace from #296,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1080/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1075893249,I_kwDOBm6k_c5AINQB,1545,Custom pages don't work on windows,559711,ryascott,closed,0,,,,,3,2021-12-09T18:53:05Z,2022-02-03T02:08:31Z,2022-02-03T01:58:35Z,NONE,,"It seems that custom pages don't work when put in templates/pages
To reproduce on datasette version 0.59.4 using PowerShell on WIndows 10 with Python 3.10.0
mkdir -p templates/pages
echo ""hello world"" >> templates/pages/about.html
Start datasette
datasette --template-dir templates/
Navigate to [http://127.0.0.1:8001/about](url) and receive:
Error 404:
Database not found: about
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1545/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1120990806,PR_kwDOBm6k_c4x6zZ5,1617,"Ensure template_path always uses ""/"" to match jinja",3526913,cb160,closed,0,,,,,3,2022-02-01T17:20:30Z,2022-02-03T01:58:35Z,2022-02-03T01:58:35Z,CONTRIBUTOR,simonw/datasette/pulls/1617,"This PR shoudl fix #1545
The existing code substituted / for \, assuming this was the right behaviour for windows. But on Windows, Jinja still uses / for the template list - See https://github.com/pallets/jinja/blob/896a62135bcc151f2997e028c5125bec2cb2431f/src/jinja2/loaders.py#L225",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1617/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1122557010,I_kwDOBm6k_c5C6NxS,1627,Get the tests passing against Windows,9599,simonw,open,0,,,,,0,2022-02-03T01:23:06Z,2022-02-03T01:23:32Z,,OWNER,,"> OK, the tests do NOT pass against Windows! https://github.com/simonw/datasette/runs/5044105941
>
>
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1626#issuecomment-1028515161_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1627/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1122451096,PR_kwDOBm6k_c4x_mXy,1626,Try test suite against macOS and Windows,9599,simonw,open,0,,,,,3,2022-02-02T22:26:51Z,2022-02-03T01:22:44Z,,OWNER,simonw/datasette/pulls/1626,Refs #1625,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1626/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1122450452,I_kwDOBm6k_c5C5zwU,1625,Try running tests against macOS and Windows in addition to Ubuntu,9599,simonw,open,0,,,,,0,2022-02-02T22:25:57Z,2022-02-02T22:25:57Z,,OWNER,,"I already do this for `sqlite-utils`: https://github.com/simonw/sqlite-utils/blob/3.22.1/.github/workflows/test.yml
Related:
- #1617
- #1545",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1625/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1119413338,PR_kwDOBm6k_c4x1kCu,1616,Bump black from 21.12b0 to 22.1.0,49699333,dependabot[bot],closed,0,,,,,2,2022-01-31T13:13:46Z,2022-02-02T22:23:52Z,2022-02-02T22:23:51Z,CONTRIBUTOR,simonw/datasette/pulls/1616,"Bumps [black](https://github.com/psf/black) from 21.12b0 to 22.1.0.
Release notes
Sourced from black's releases .
22.1.0
At long last, Black is no longer a beta product! This is the first non-beta release and the first release covered by our new stability policy.
Highlights
Remove Python 2 support (#2740 )
Introduce the --preview
flag (#2752 )
Style
Deprecate --experimental-string-processing
and move the functionality under --preview
(#2789 )
For stubs, one blank line between class attributes and methods is now kept if there's at least one pre-existing blank line (#2736 )
Black now normalizes string prefix order (#2297 )
Remove spaces around power operators if both operands are simple (#2726 )
Work around bug that causes unstable formatting in some cases in the presence of the magic trailing comma (#2807 )
Use parentheses for attribute access on decimal float and int literals (#2799 )
Don't add whitespace for attribute access on hexadecimal, binary, octal, and complex literals (#2799 )
Treat blank lines in stubs the same inside top-level if statements (#2820 )
Fix unstable formatting with semicolons and arithmetic expressions (#2817 )
Fix unstable formatting around magic trailing comma (#2572 )
Parser
Fix mapping cases that contain as-expressions, like case {"key": 1 | 2 as password}
(#2686 )
Fix cases that contain multiple top-level as-expressions, like case 1 as a, 2 as b
(#2716 )
Fix call patterns that contain as-expressions with keyword arguments, like case Foo(bar=baz as quux)
(#2749 )
Tuple unpacking on return
and yield
constructs now implies 3.8+ (#2700 )
Unparenthesized tuples on annotated assignments (e.g values: Tuple[int, ...] = 1, 2, 3
) now implies 3.8+ (#2708 )
Fix handling of standalone match()
or case()
when there is a trailing newline or a comment inside of the parentheses. (#2760 )
from __future__ import annotations
statement now implies Python 3.7+ (#2690 )
Performance
Speed-up the new backtracking parser about 4X in general (enabled when --target-version
is set to 3.10 and higher). (#2728 )
Black is now compiled with mypyc for an overall 2x speed-up. 64-bit Windows, MacOS, and Linux (not including musl) are supported. (#1009 , #2431 )
Configuration
Do not accept bare carriage return line endings in pyproject.toml (#2408 )
Add configuration option (python-cell-magics
) to format cells with custom magics in Jupyter Notebooks (#2744 )
Allow setting custom cache directory on all platforms with environment variable BLACK_CACHE_DIR
(#2739 ).
Enable Python 3.10+ by default, without any extra need to specify --target-version=py310
. (#2758 )
Make passing SRC
or --code
mandatory and mutually exclusive (#2804 )
Output
Improve error message for invalid regular expression (#2678 )
Improve error message when parsing fails during AST safety check by embedding the underlying SyntaxError (#2693 )
No longer color diff headers white as it's unreadable in light themed terminals (#2691 )
Text coloring added in the final statistics (#2712 )
Verbose mode also now describes how a project root was discovered and which paths will be formatted. (#2526 )
Packaging
All upper version bounds on dependencies have been removed (#2718 )
typing-extensions
is no longer a required dependency in Python 3.10+ (#2772 )
Set click
lower bound to 8.0.0
as Black crashes on 7.1.2
(#2791 )
... (truncated)
Changelog
Sourced from black's changelog .
22.1.0
At long last, Black is no longer a beta product! This is the first non-beta release
and the first release covered by our new stability policy.
Highlights
Remove Python 2 support (#2740 )
Introduce the --preview
flag (#2752 )
Style
Deprecate --experimental-string-processing
and move the functionality under
--preview
(#2789 )
For stubs, one blank line between class attributes and methods is now kept if there's
at least one pre-existing blank line (#2736 )
Black now normalizes string prefix order (#2297 )
Remove spaces around power operators if both operands are simple (#2726 )
Work around bug that causes unstable formatting in some cases in the presence of the
magic trailing comma (#2807 )
Use parentheses for attribute access on decimal float and int literals (#2799 )
Don't add whitespace for attribute access on hexadecimal, binary, octal, and complex
literals (#2799 )
Treat blank lines in stubs the same inside top-level if
statements (#2820 )
Fix unstable formatting with semicolons and arithmetic expressions (#2817 )
Fix unstable formatting around magic trailing comma (#2572 )
Parser
Fix mapping cases that contain as-expressions, like case {"key": 1 | 2 as password}
(#2686 )
Fix cases that contain multiple top-level as-expressions, like case 1 as a, 2 as b
(#2716 )
Fix call patterns that contain as-expressions with keyword arguments, like
case Foo(bar=baz as quux)
(#2749 )
Tuple unpacking on return
and yield
constructs now implies 3.8+ (#2700 )
Unparenthesized tuples on annotated assignments (e.g
values: Tuple[int, ...] = 1, 2, 3
) now implies 3.8+ (#2708 )
Fix handling of standalone match()
or case()
when there is a trailing newline or a
comment inside of the parentheses. (#2760 )
from __future__ import annotations
statement now implies Python 3.7+ (#2690 )
Performance
Speed-up the new backtracking parser about 4X in general (enabled when
--target-version
is set to 3.10 and higher). (#2728 )
Black is now compiled with mypyc for an overall 2x
speed-up. 64-bit Windows, MacOS, and Linux (not including musl) are supported. (#1009 ,
#2431 )
... (truncated)
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.12b0&new-version=22.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1616/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1122414274,PR_kwDOBm6k_c4x_evE,1622,Test against Python 3.11-dev,9599,simonw,closed,0,,,,,1,2022-02-02T21:39:38Z,2022-02-02T21:58:53Z,2022-02-02T21:58:53Z,OWNER,simonw/datasette/pulls/1622,Refs #1621,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1622/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1121618041,I_kwDOBm6k_c5C2oh5,1620,"Link: rel=""alternate"" to JSON for queries too",9599,simonw,closed,0,,,3268330,Datasette 1.0,3,2022-02-02T08:02:42Z,2022-02-02T21:53:02Z,2022-02-02T21:33:00Z,OWNER,,"Following:
- #1533
I implemented it for tables and rows but I should have done queries as well.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1620/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1121121305,I_kwDOBm6k_c5C0vQZ,1618,"Reconsider policy on blocking queries containing the string ""pragma""",770231,strada,open,0,,,,,6,2022-02-01T19:39:46Z,2022-02-02T19:42:03Z,,NONE,,"First of all, thanks for creating this cool project, and also supporting publishing to various hosting services out of the box.
While testing out, I noticed legitimate queries such as
```
select * from books where title like 'Pragmatic%'
```
or
```
select * from books where title = 'The Pragmatic Programmer'
```
are blocked, due to the regular expression check here:
https://github.com/simonw/datasette/blob/main/datasette/utils/__init__.py#L185
Example as seen from a Datasette instance:
https://fivethirtyeight.datasettes.com/polls?sql=select+*+from+books+where+title+like+%27Pragmatic%25%27%0D%0A
I'd propose a regular expression like
```
re.compile(f""pragma_(?!({'|'.join(allowed_pragmas)}))""),
```
instead of
```
re.compile(f""pragma(?!_({'|'.join(allowed_pragmas)}))""),
```
I can create a pull request with this change, unless the maintainers think it would allow unwanted queries to be executed.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1618/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1065431383,I_kwDOBm6k_c4_gTFX,1533,"Add `Link: rel=""alternate""` header pointing to JSON for a table/query",9599,simonw,closed,0,,,3268330,Datasette 1.0,4,2021-11-28T20:43:25Z,2022-02-02T07:56:51Z,2022-02-02T07:49:33Z,OWNER,,"Originally explored in https://github.com/simonw/datasette-notebook/issues/2#issuecomment-980789406 - I wanted an efficient way to scan a list of URLs and figure out which if any of those corresponded to Datasette tables, canned queries or SQL output that could be represented as a table on a page.
It looks like a neat way to do that is with ` Link:` header like this:
`Link: http://127.0.0.1:8058/fixtures/compound_three_primary_keys.json; rel=""alternate""; type=""application/datasette+json""`
I can put a `
Looks like running `count(*)` against KNN took 83s! It ignored the time limit. And still only returned a count of 0.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1611/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1114628238,I_kwDOBm6k_c5Cb-CO,1613,Improvements to help make Datasette a better tool for learning SQL,9599,simonw,open,0,,,,,5,2022-01-26T04:56:07Z,2022-01-26T16:41:46Z,,OWNER,,Tracking issue for the general goal of making Datasette a better tool for learning SQL.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1613/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1087913724,I_kwDOBm6k_c5A2D78,1577,Drop support for Python 3.6,9599,simonw,closed,0,,,3268330,Datasette 1.0,6,2021-12-23T18:17:03Z,2022-01-25T23:30:03Z,2022-01-20T04:31:41Z,OWNER,,"*Original title: Decide when to drop support for Python 3.6*
> `context_vars` can solve this but they were introduced in Python 3.7: https://www.python.org/dev/peps/pep-0567/
>
> Python 3.6 support ends in a few days time, and it looks like Glitch has updated to 3.7 now - so maybe I can get away with Datasette needing 3.7 these days?
>
> Tweeted about that here: https://twitter.com/simonw/status/1473761478155010048
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1576#issuecomment-999878907_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1577/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1109884720,I_kwDOBm6k_c5CJ38w,1609,"Ensure ""pip install datasette"" still works with Python 3.6",9599,simonw,closed,0,,,,,12,2022-01-21T00:08:10Z,2022-01-24T19:20:09Z,2022-01-21T02:24:13Z,OWNER,,"## Original title: Can I keep ""pip install datasette"" working on Python 3.6?
I dropped support for 3.6 in:
- #1577
I'm getting reports that `pip3 install datasette` throws an error on that Python, even though I haven't made that new release yet - presumably due to lack of pinning of Uvicorn: https://twitter.com/ldodds/status/1484289475195080706
Is it possible to get `pip` on that version of Python to install the highest possible version of the packages that are still known to support Python 3.6?
If so, how?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1609/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1105916061,I_kwDOBm6k_c5B6vCd,1601,Add KNN and data_licenses to hidden tables list,25778,eyeseast,closed,0,,,,,5,2022-01-17T14:19:57Z,2022-01-20T21:29:44Z,2022-01-20T04:38:54Z,CONTRIBUTOR,,"They're generated by Spatialite and not very interesting in most cases.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1601/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
824064069,MDU6SXNzdWU4MjQwNjQwNjk=,1249,Updated Dockerfile with SpatiaLite version 5.0,9599,simonw,closed,0,,,,,45,2021-03-08T00:17:36Z,2022-01-20T21:29:43Z,2021-03-29T00:57:13Z,OWNER,,"The version bundled in Datasette's Docker image right now is 4.4.0-RC0
https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/Dockerfile#L16-L17
5 has been out for a couple of months and has a bunch of big improvements, most notable stable KNN support.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
837308703,MDU6SXNzdWU4MzczMDg3MDM=,1268,Figure out why SpatiaLite 5.0 hangs the database page on Linux,9599,simonw,closed,0,,,,,18,2021-03-22T04:44:16Z,2022-01-20T21:29:43Z,2021-03-22T17:41:12Z,OWNER,,"See detailed notes in https://github.com/simonw/datasette/issues/1249 - for some reason SpatiaLite 5.0 hangs the `/dbname` page on Linux (inside Docker containers, both with a custom compiled SpatiaLite and one installed from the Ubuntu 20.10 package repository). This doesn't happen on macOS with SpatiaLite 5 installed using Homebrew.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1268/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
842416110,MDU6SXNzdWU4NDI0MTYxMTA=,1278,SpatiaLite timezones demo is broken,9599,simonw,closed,0,,,,,2,2021-03-27T04:45:27Z,2022-01-20T21:29:43Z,2021-03-27T16:17:13Z,OWNER,,https://github.com/simonw/datasette/blob/5fd02890650db790b2ffdb90eb9f78f8e0639c37/docs/spatialite.rst#L96,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1278/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083581011,I_kwDOBm6k_c5AliJT,1564,_prepare_connection not called on write connections,9599,simonw,closed,0,,,7571612,Datasette 0.60,1,2021-12-17T20:06:47Z,2022-01-20T21:29:43Z,2021-12-18T01:58:44Z,OWNER,,"I was trying to initalize SpatiaLite in a write connection:
```pycon
>>> from datasette.app import Datasette
>>> ds = Datasette(memory=True, files=[], sqlite_extensions=[""spatialite""])
>>> db = ds.add_memory_database('geo')
>>> await db.execute_write(""select InitSpatialMetadata(1)"")
UUID('3f143baa-4e3d-5842-a36f-4fa2f683b72f')
no such function: InitSpatialMetadata
```
It looks like the code that loads additional modules only works on read-only connections, not on write connections:
https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L146-L153
Compared to:
https://github.com/simonw/datasette/blob/92a5280d2e75c39424a75ad6226fc74400ae984f/datasette/database.py#L124-L132",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1564/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
752966476,MDU6SXNzdWU3NTI5NjY0NzY=,1114,--load-extension=spatialite not working with datasetteproject/datasette docker image,2182,danp,closed,0,,,,,4,2020-11-29T17:35:20Z,2022-01-20T21:29:42Z,2020-11-29T17:37:45Z,CONTRIBUTOR,,"https://github.com/simonw/datasette/commit/6aa5886379dd9017215904fb28567b80018902f9 added the `--load-extension=spatialite` shortcut looking for the extension in these places:
https://github.com/simonw/datasette/blob/12877d7a48e2aa28bb5e780f929a218f7265d849/datasette/utils/__init__.py#L56-L60
However, in the datasetteproject/datasette docker image the file is at `/usr/local/lib/mod_spatialite.so`.
This results in the example command [here](https://docs.datasette.io/en/stable/installation.html#loading-spatialite) failing:
```
% docker run --rm -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/data.db --load-extension=spatialite
Error: Could not find SpatiaLite extension
```
But it does work when given an explicit path:
```
% docker run --rm -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/data.db --load-extension=/usr/local/lib/mod_spatialite.so
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
...
```
Perhaps `SPATIALITE_PATHS` should include `/usr/local/lib/mod_spatialite.so`?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1114/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
752995227,MDU6SXNzdWU3NTI5OTUyMjc=,1115,SpatiaLite error could suggest --load-extension=spatialite,9599,simonw,closed,0,,,,,1,2020-11-29T20:05:07Z,2022-01-20T21:29:42Z,2020-11-29T20:13:22Z,OWNER,,"https://github.com/simonw/datasette/blob/242bc89fdf2e775e340d69a4e851b3a9accb31c6/datasette/cli.py#L533-L548
This could use the `find_spatialite()` function and, if it finds something, suggest the user use `--load-extension=spatialite`
https://github.com/simonw/datasette/blob/242bc89fdf2e775e340d69a4e851b3a9accb31c6/datasette/utils/__init__.py#L1015-L1019",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1115/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
443040665,MDU6SXNzdWU0NDMwNDA2NjU=,466,"Move ""no such module: VirtualSpatialIndex"" code elsewhere",9599,simonw,closed,0,,,4305096,0.28,2,2019-05-11T22:09:00Z,2022-01-20T21:29:41Z,2019-05-11T22:57:22Z,OWNER,,"We currently show a useful warning (from #331) when the user tries to open a spatialite database without first loading the module:
https://github.com/simonw/datasette/blob/c692cd291111050483a32bea1ee08e994a0b781b/datasette/app.py#L547-L554
This code is part of `.inspect()` which is going away - see #462 - so I need to find somewhere else for it to live.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/466/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
563347679,MDU6SXNzdWU1NjMzNDc2Nzk=,668,Make it easier to load SpatiaLite,9599,simonw,closed,0,,,,,2,2020-02-11T17:03:43Z,2022-01-20T21:29:41Z,2021-01-04T20:18:39Z,OWNER,,"```
$ datasette spatial.db
Serve! files=('spatial.db',) (immutables=()) on port 8001
ERROR: conn=, sql = 'PRAGMA table_info(SpatialIndex);', params = None: no such module: VirtualSpatialIndex
Usage: datasette serve [OPTIONS] [FILES]...
Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module.
Read more: https://datasette.readthedocs.io/en/latest/spatialite.html
```
This error message could sniff around in the common locations for the SpatiaLite module and output the CLI command you should use to enable it:
```
datasette spatial.db --load-extension=/usr/local/lib/mod_spatialite.dylib
```
Even better: if Datasette had a `--spatialite` option which automatically loads the extension from common locations, if it can find it.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/668/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
723803777,MDU6SXNzdWU3MjM4MDM3Nzc=,1028,--load-extension=spatialite shortcut,9599,simonw,closed,0,,,6026070,0.51,1,2020-10-17T17:02:08Z,2022-01-20T21:29:41Z,2020-10-19T22:37:55Z,OWNER,,I added this to `sqlite-utils` in https://github.com/simonw/sqlite-utils/issues/136 and I really like it: pass a special value of `spatialite` and Datasette should attempt to load it from known likely installation locations.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1028/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
336936010,MDU6SXNzdWUzMzY5MzYwMTA=,331,Datasette throws error when loading spatialite db without extension loaded,82988,psychemedia,closed,0,,,,,2,2018-06-29T09:51:14Z,2022-01-20T21:29:40Z,2018-07-10T15:13:36Z,CONTRIBUTOR,,"When starting datasette on a SpatialLite database *without* loading the SpatiaLite extension (using eg `--load-extension=/usr/local/lib/mod_spatialite.dylib`) an error is thrown and the server fails to start:
```
datasette -p 8003 adminboundaries.db
Serve! files=('adminboundaries.db',) on port 8003
Traceback (most recent call last):
File ""/Users/ajh59/anaconda3/bin/datasette"", line 11, in
sys.exit(cli())
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 722, in __call__
return self.main(*args, **kwargs)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 697, in main
rv = self.invoke(ctx)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 535, in invoke
return callback(*args, **kwargs)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/cli.py"", line 552, in serve
ds.inspect()
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/app.py"", line 273, in inspect
""tables"": inspect_tables(conn, self.metadata.get(""databases"", {}).get(name, {}))
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/inspect.py"", line 79, in inspect_tables
""PRAGMA table_info({});"".format(escape_sqlite(table))
sqlite3.OperationalError: no such module: VirtualSpatialIndex
```
It would be nice to trap this and return a message saying something like:
```
It looks like you're trying to load a SpatiaLite database? Make sure you load in the SpatiaLite extension when starting datasette.
Read more: https://datasette.readthedocs.io/en/latest/spatialite.html
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/331/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
328155946,MDU6SXNzdWUzMjgxNTU5NDY=,301,"--spatialite option for ""datasette publish heroku""",9599,simonw,open,0,,,,,1,2018-05-31T14:13:09Z,2022-01-20T21:28:50Z,,OWNER,,Split off from #243. Need to figure out how to install and configure SpatiaLite on Heroku.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/301/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
838382890,MDU6SXNzdWU4MzgzODI4OTA=,1273,Refresh SpatiaLite documentation,9599,simonw,open,0,,,,,4,2021-03-23T06:05:55Z,2022-01-20T21:28:50Z,,OWNER,,https://docs.datasette.io/en/0.55/spatialite.html was written before I had tools like [geojson-to-sqlite](https://datasette.io/tools/geojson-to-sqlite) and [shapefile-to-sqlite](https://datasette.io/tools/shapefile-to-sqlite).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1273/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
849975810,MDU6SXNzdWU4NDk5NzU4MTA=,1292,Research ctypes.util.find_library('spatialite'),9599,simonw,open,0,,,,,1,2021-04-04T22:36:59Z,2022-01-20T21:28:50Z,,OWNER,,"Spotted this in the Django SpatiaLite backend: https://github.com/django/django/blob/8f6a7a0e9e7c5404af6520ae606927e32415eb00/django/contrib/gis/db/backends/spatialite/base.py#L24-L36
```python
ctypes.util.find_library('spatialite')
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1292/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1104691662,I_kwDOBm6k_c5B2EHO,1600,plugins --all example should use cog,9599,simonw,closed,0,,,,,1,2022-01-15T11:47:49Z,2022-01-20T05:06:21Z,2022-01-20T05:04:16Z,OWNER,,The example output for `datasette plugins --all`on this page has got out of date: https://docs.datasette.io/en/stable/plugins.html#seeing-what-plugins-are-installed,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1600/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1108846067,I_kwDOBm6k_c5CF6Xz,1606,Tests failing against Python 3.6,9599,simonw,closed,0,,,,,3,2022-01-20T04:22:44Z,2022-01-20T04:36:42Z,2022-01-20T04:36:42Z,OWNER,,"https://github.com/simonw/datasette/runs/4877484366
```
E File ""/opt/hostedtoolcache/Python/3.6.15/x64/lib/python3.6/site-packages/uvicorn/server.py"", line 67, in run
E return asyncio.run(self.serve(sockets=sockets))
E AttributeError: module 'asyncio' has no attribute 'run'
```
I think this may mean `uvicorn` has dropped support for Python 3.6.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1097040427,I_kwDOBm6k_c5BY4Ir,1587,Add `sqlite_stat1`(-4) tables to hidden table list,9599,simonw,closed,0,,,,,2,2022-01-08T21:28:20Z,2022-01-20T04:12:59Z,2022-01-20T04:12:59Z,OWNER,,"> Running `ANALYZE` creates a new visible table called `sqlite_stat1`: https://www.sqlite.org/fileformat.html#the_sqlite_stat1_table
>
> This should be added to the default list of hidden tables in Datasette.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1587/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
910092577,MDU6SXNzdWU5MTAwOTI1Nzc=,1356,"Research: syntactic sugar for using --get with SQL queries, maybe ""datasette query""",9599,simonw,open,0,,,,,10,2021-06-03T04:49:42Z,2022-01-20T01:06:37Z,,OWNER,,"Inspired by https://github.com/simonw/sqlite-utils/issues/264 - in particular this example:
```
datasette covid.db --get='/covid.yaml?sql=select * from ny_times_us_counties limit 1'
- date: '2020-01-21'
county: Snohomish
state: Washington
fips: 53061
cases: 1
deaths: 0
```
Having to construct that URL - including potentially URL escaping the SQL query - isn't a great developer experience.
Imagine if you could do this instead:
datasette covid.db --query ""select * from ny_times_us_counties limit 1"" --format yaml
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1356/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1108300685,I_kwDOBm6k_c5CD1ON,1604,Option to assign a domain/subdomain using `datasette publish cloudrun`,9599,simonw,open,0,,,,,1,2022-01-19T16:21:17Z,2022-01-19T16:23:54Z,,OWNER,,Looks like this API should be able to do that: https://twitter.com/steren/status/1483835859191304192 - https://cloud.google.com/run/docs/reference/rest/v1/namespaces.domainmappings/create,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1604/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1100015398,I_kwDOBm6k_c5BkOcm,1591,Maybe let plugins define custom serve options?,9599,simonw,open,0,,,,,7,2022-01-12T08:18:47Z,2022-01-15T11:56:59Z,,OWNER,,"https://twitter.com/psychemedia/status/1481171650934714370
> can extensions be passed their own cli args? eg `--ext-tiddlywiki-dbname tiddlywiki2.sqlite` ?
I've thought something like this might be useful for other plugins in the past, too.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1591/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1102966378,I_kwDOBm6k_c5Bve5q,1599,Add architecture documentation,9599,simonw,open,0,,,,,0,2022-01-14T04:55:38Z,2022-01-14T04:56:03Z,,OWNER,,"Inspired by https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html
Good example: https://github.com/rust-analyzer/rust-analyzer/blob/d7c99931d05e3723d878bea5dc26766791fa4e69/docs/dev/architecture.md",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1599/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1083669410,I_kwDOBm6k_c5Al3ui,1566,Release Datasette 0.60,9599,simonw,closed,0,,,7571612,Datasette 0.60,6,2021-12-17T22:58:12Z,2022-01-14T01:59:55Z,2022-01-14T01:59:55Z,OWNER,,Using this as a tracking issue. I'm hoping to get the bulk of the JSON redesign work from the refactor in #1554 in for this release.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1566/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1102484126,I_kwDOBm6k_c5BtpKe,1595,Release notes for 0.60,9599,simonw,closed,0,,,7571612,Datasette 0.60,4,2022-01-13T22:23:14Z,2022-01-14T01:37:39Z,2022-01-14T01:37:39Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1595/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1099723916,I_kwDOBm6k_c5BjHSM,1590,Table+query JSON and CSV links broken when using `base_url` setting,1001306,eelkevdbos,closed,0,,,7571612,Datasette 0.60,11,2022-01-11T23:46:39Z,2022-01-14T01:16:34Z,2022-01-14T01:16:08Z,NONE,,"Datasette appends the prefix found in the `base_url` setting twice if a `base_url` is set.
In the follow asgi example, I'm hosting a custom Datasette instance:
```python
# asgi.py
import pathlib
from asgi_cors import asgi_cors
from channels.routing import URLRouter
from django.urls import re_path
from datasette.app import Datasette
datasette_ = Datasette(
files=[],
settings={
""base_url"": ""/datasettes/"",
""plugins"": {}
},
config_dir=pathlib.Path('.'),
)
application = URLRouter([
re_path(r""^datasettes/.*"", asgi_cors(datasette_.app(), allow_all=True)),
])
```
Running it with:
```shell
$ daphne -p 8002 asgi:application
```
Using a simple query on the `_memory` table:
```sql
select sqlite_version()
```
http://localhost:8002/datasettes/_memory?sql=select+sqlite_version%28%29
It renders the following upon inspection:
![image](https://user-images.githubusercontent.com/1001306/149038851-aa842950-126a-467c-9a86-fae13bce6221.png)
I am using datasette version `0.59.4`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1590/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1059555791,I_kwDOBm6k_c4_J4nP,1527,Columns starting with an underscore behave poorly in filters,9599,simonw,closed,0,,,7571612,Datasette 0.60,7,2021-11-22T01:01:36Z,2022-01-14T00:57:08Z,2022-01-14T00:57:08Z,OWNER,,"Similar bug to #1525 (and #1506 before it). Start on https://latest.datasette.io/fixtures/facetable?_facet=_neighborhood - then select a neighborhood - then try to remove that filter using the little ""x"" and submitting the form again.
![filter-bug](https://user-images.githubusercontent.com/9599/142786754-31d265a2-944d-4ea2-af6f-305d445a2ccb.gif)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1527/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1102637351,I_kwDOBm6k_c5BuOkn,1598,Replace update-docs-help.py script with cog,9599,simonw,closed,0,,,,,1,2022-01-14T00:33:27Z,2022-01-14T00:47:57Z,2022-01-14T00:47:57Z,OWNER,,"I introduced `cog` in #1594 - I can use this to replace the older `update-docs-help.py` mechanism:
https://github.com/simonw/datasette/blob/76d66d5b2bf10249c0beaac0999b93ac8d757f48/tests/test_docs.py#L36-L53",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1598/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1102612922,I_kwDOBm6k_c5BuIm6,1597,"""datasette inspect"" has no help summary",9599,simonw,closed,0,,,,,1,2022-01-14T00:02:16Z,2022-01-14T00:07:36Z,2022-01-14T00:07:36Z,OWNER,,"Made obvious by the new CLI reference page added in #1594. https://docs.datasette.io/en/latest/cli-reference.html#datasette-inspect-help
```
Commands:
serve* Serve up specified SQLite database files with a web UI
inspect
install Install Python packages - e.g.
```
```
Usage: datasette inspect [OPTIONS] [FILES]...
Options:
--inspect-file TEXT
--load-extension TEXT Path to a SQLite extension to load
--help Show this message and exit.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1597/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1102568047,I_kwDOBm6k_c5Bt9pv,1596,Documentation page warning of changes coming in 1.0,9599,simonw,open,0,,,,,0,2022-01-13T23:26:04Z,2022-01-13T23:26:04Z,,OWNER,,I should start this relatively soon.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1596/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1082746149,I_kwDOBm6k_c5AiWUl,1560,"Table page title has ""where where"" in it",9599,simonw,closed,0,,,7571612,Datasette 0.60,0,2021-12-17T00:05:48Z,2022-01-13T22:28:35Z,2022-01-13T22:20:15Z,OWNER,,"Just noticed this while working on #1518.
```
% curl -s 'https://latest.datasette.io/fixtures/facetable?_sort=pk&on_earth__exact=1' | grep -C 1 ''
fixtures: facetable: 14 rows
where where on_earth = 1 sorted by pk
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1560/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
991467558,MDU6SXNzdWU5OTE0Njc1NTg=,1466,Add Datasette Desktop to installation documentation,9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-09-08T19:41:27Z,2022-01-13T22:28:28Z,2022-01-13T21:55:18Z,OWNER,,See https://datasette.io/desktop and https://simonwillison.net/2021/Sep/8/datasette-desktop/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1466/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1102359726,I_kwDOBm6k_c5BtKyu,1594,"Add a CLI reference page to the docs, inspired by sqlite-utils",9599,simonw,closed,0,,,7571612,Datasette 0.60,3,2022-01-13T20:55:08Z,2022-01-13T22:28:22Z,2022-01-13T21:38:48Z,OWNER,,"Thought of this while posting this comment: https://github.com/simonw/datasette/issues/1591#issuecomment-1012506595
I added https://sqlite-utils.datasette.io/en/stable/cli-reference.html to `sqlite-utils` in https://github.com/simonw/sqlite-utils/issues/383 and I _really_ like it - it's a page showing the `--help` output of every CLI command for that tool.
It's maintained using `cog`. One of the benefits is that I get a free commit history of changes to `--help` at https://github.com/simonw/sqlite-utils/commits/main/docs/cli-reference.rst",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1594/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1097101917,I_kwDOBm6k_c5BZHJd,1588,`explain query plan select` is too strict about whitespace,9599,simonw,closed,0,,,7571612,Datasette 0.60,3,2022-01-09T04:22:42Z,2022-01-13T22:28:19Z,2022-01-13T20:35:05Z,OWNER,,"`explain query plan select * from facetable` is allowed: https://latest.datasette.io/fixtures?sql=explain+query+plan+select+*+from+facetable
But... `explain query plan select * from facetable` (with two spaces before the `select`) returns a ""Statement must be a SELECT"" error: https://latest.datasette.io/fixtures?sql=explain+query+plan++select+*+from+facetable",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1588/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1087931918,I_kwDOBm6k_c5A2IYO,1579,`.execute_write(... block=True)` should be the default behaviour,9599,simonw,closed,0,,,7571612,Datasette 0.60,7,2021-12-23T18:54:28Z,2022-01-13T22:28:08Z,2021-12-23T19:18:26Z,OWNER,,"Every single piece of code I've written against the write APIs has used the `block=True` option to wait for the result.
Without that, it instead fires the write into the queue but then continues even before it has finished executing.
`block=True` should clearly be the default behaviour here!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1579/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1076388044,I_kwDOBm6k_c5AKGDM,1547,Writable canned queries fail to load custom templates,127565,wragge,closed,0,,,7571612,Datasette 0.60,6,2021-12-10T03:31:48Z,2022-01-13T22:27:59Z,2021-12-19T21:12:00Z,CONTRIBUTOR,,"I've created a canned query with `""write"": true` set. I've also created a custom template for it, but the template doesn't seem to be found. If I look in the HTML I see (`stock_exchange` is the db name):
``
My non-writeable canned queries pick up custom templates as expected, and if I look at their HTML I see the canned query name added to the templates considered (the canned query here is `date_search`):
``
So it seems like the writeable canned query is behaving differently for some reason. Is it an authentication thing? I'm using the built in `--root` authentication.
Thanks!
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1547/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083927147,I_kwDOBm6k_c5Am2pr,1571,Track number of executions for execute_write_many() in traces,9599,simonw,closed,0,,,7571612,Datasette 0.60,0,2021-12-18T19:16:17Z,2022-01-13T22:27:49Z,2021-12-19T20:30:40Z,OWNER,,"Spotted while working on #1555
There's no indication there of how many times `execute_write_many()` executed the SQL.
Solving this is a tiny bit tricky because `params_seq` is an iterator that we don't want to exhaust before passing it to `conn.executemany()` - so we need to instead wrap it in something that counts how many times it was called.
But then we need a way to attach that to the trace here: https://github.com/simonw/datasette/blob/d637ed46762fdbbd8e32b86f258cd9a53c1cfdc7/datasette/database.py#L115-L122
So probably need to redesign the `trace()` decorator to allow extra pairs to be attached to it within the `with` statement.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1571/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1084007781,I_kwDOBm6k_c5AnKVl,1572,"""Query took"" should be ""Queries took""",9599,simonw,closed,0,,,7571612,Datasette 0.60,0,2021-12-19T04:03:00Z,2022-01-13T22:27:43Z,2021-12-19T04:03:24Z,OWNER,,"This is misleading, since usually there have been more than one query executed:
![CleanShot 2021-12-18 at 20 02 35@2x](https://user-images.githubusercontent.com/9599/146663457-9c4c2900-5cc0-4650-a565-bb1ff0b8a725.png)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1572/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083921371,I_kwDOBm6k_c5Am1Pb,1570,Separate db.execute_write() into three methods,9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-12-18T18:45:54Z,2022-01-13T22:27:38Z,2021-12-18T18:57:25Z,OWNER,,"> Rather than adding a `executemany=True` parameter, I'm now thinking a better design might be to have three methods:
>
> - `db.execute_write(sql, params=None, block=False)`
> - `db.execute_write_script(sql, block=False)`
> - `db.execute_write_many(sql, params_seq, block=False)`
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997267416_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1570/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1079149656,I_kwDOBm6k_c5AUoRY,1555,Optimize all those calls to index_list and foreign_key_list,9599,simonw,closed,0,,,7571612,Datasette 0.60,27,2021-12-13T23:50:56Z,2022-01-13T22:27:32Z,2021-12-19T20:55:59Z,OWNER,,"On the first hit to a restarted index I'm seeing this in the SQL traces: https://latest-with-plugins.datasette.io/github/commits?_trace=1
I imagine this could be sped up a lot using tricks like this one from the SQLite documentation: https://sqlite.org/pragma.html#pragfunc
```sql
SELECT DISTINCT m.name || '.' || ii.name AS 'indexed-columns'
FROM sqlite_schema AS m,
pragma_index_list(m.name) AS il,
pragma_index_info(il.name) AS ii
WHERE m.type='table'
ORDER BY 1;
```
https://latest-with-plugins.datasette.io/fixtures?sql=SELECT+DISTINCT+m.name+%7C%7C+%27.%27+%7C%7C+ii.name+AS+%27indexed-columns%27%0D%0A++FROM+sqlite_schema+AS+m%2C%0D%0A+++++++pragma_index_list%28m.name%29+AS+il%2C%0D%0A+++++++pragma_index_info%28il.name%29+AS+ii%0D%0A+WHERE+m.type%3D%27table%27%0D%0A+ORDER+BY+1%3B",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1555/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083895395,I_kwDOBm6k_c5Amu5j,1569,"db.execute_write(..., executescript=True) parameter",9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-12-18T18:20:47Z,2022-01-13T22:27:27Z,2021-12-18T18:34:18Z,OWNER,,"> Idea: teach `execute_write` to accept an optional `executescript=True` parameter, like this:
```diff
diff --git a/datasette/database.py b/datasette/database.py
index 468e936..1a424f5 100644
--- a/datasette/database.py
+++ b/datasette/database.py
@@ -94,10 +94,14 @@ class Database:
f""file:{self.path}{qs}"", uri=True, check_same_thread=False
)
- async def execute_write(self, sql, params=None, block=False):
+ async def execute_write(self, sql, params=None, executescript=False, block=False):
+ assert not executescript and params, ""Cannot use params with executescript=True""
def _inner(conn):
with conn:
- return conn.execute(sql, params or [])
+ if executescript:
+ return conn.executescript(sql)
+ else:
+ return conn.execute(sql, params or [])
with trace(""sql"", database=self.name, sql=sql.strip(), params=params):
results = await self.execute_write_fn(_inner, block=block)
```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997248364_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1569/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083726550,I_kwDOBm6k_c5AmFrW,1568,Trace should show queries on the write connection too,9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-12-18T02:34:12Z,2022-01-13T22:27:23Z,2021-12-18T02:42:34Z,OWNER,,"> Here's why - `trace` only applies to read, not write SQL operations: https://github.com/simonw/datasette/blob/7c8f8aa209e4ba7bf83976f8495d67c28fbfca24/datasette/database.py#L209-L211
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997128508_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1568/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083573206,I_kwDOBm6k_c5AlgPW,1563,Datasette(... files=) should not be a required argument,9599,simonw,closed,0,,,7571612,Datasette 0.60,2,2021-12-17T19:54:18Z,2022-01-13T22:27:18Z,2021-12-18T02:19:40Z,OWNER,,"```pycon
>>> ds = Datasette(memory=True)
Traceback (most recent call last):
File """", line 1, in
TypeError: __init__() missing 1 required positional argument: 'files'
>>> ds = Datasette(memory=True, files=[])
```
I wanted to create an in-memory Datasette for running some tests, no point in forcing me to pass `files=[]` to do that.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1563/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083718998,I_kwDOBm6k_c5AmD1W,1567,Remove undocumented sqlite_functions mechanism,9599,simonw,closed,0,,,7571612,Datasette 0.60,0,2021-12-18T01:51:10Z,2022-01-13T22:27:04Z,2021-12-18T01:54:46Z,OWNER,,"I added this in 0b8c1b0a6da9cb8ac0d28cc90dd783de87554036 but it's never been documented and the same thing can now be achieved using the `prepare_connection` plugin hook.
https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L262
https://github.com/simonw/datasette/blob/0c91e59d2bbfc08884cfcf5d1b902a2f4968b7ff/datasette/app.py#L551-L552
It's used here in the tests:
https://github.com/simonw/datasette/blob/69244a617b1118dcbd04a8f102173f04680cf08c/tests/fixtures.py#L156",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1567/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
520740741,MDU6SXNzdWU1MjA3NDA3NDE=,625,If you apply ?_facet_array=tags then &_facet=tags does nothing,9599,simonw,closed,0,,,7571612,Datasette 0.60,13,2019-11-11T04:59:29Z,2022-01-13T22:26:58Z,2021-12-16T20:12:22Z,OWNER,,"Start here: https://v0-30-2.datasette.io/fixtures/facetable?_facet_array=tags
Note that `tags` is offered as a suggested facet. But if you click that you get this:
https://v0-30-2.datasette.io/fixtures/facetable?_facet_array=tags&_facet=tags
The `_facet=tags` is added to the URL and it's removed from the list of suggested tags... but the facet itself is not displayed:
The `_facet=tags` facet should look like this:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/625/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1082564912,I_kwDOBm6k_c5AhqEw,1557,`?_nosuggest=1` parameter for disabling facet suggestions on table view,9599,simonw,closed,0,,,7571612,Datasette 0.60,1,2021-12-16T19:21:42Z,2022-01-13T22:26:48Z,2021-12-16T19:24:59Z,OWNER,,"Found I wanted this while I was debugging #625 just to clean up the debug traces, but it makes sense as a partner to `?_nofacet=1` and `?_nocount=1` from #1350 and #1353.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1557/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1078702875,I_kwDOBm6k_c5AS7Mb,1552,Allow to set `facets_array` in metadata (like current `facets`),3556,davidbgk,closed,0,,,7571612,Datasette 0.60,9,2021-12-13T16:00:44Z,2022-01-13T22:26:15Z,2021-12-16T18:47:48Z,CONTRIBUTOR,,"For now, you can set a `facets` value (array) in your metadata file but I couldn't find a way to set a `facets_array` in order to provide default facets for arrays (like tags). My use-case is to access to [that kind of view](https://latest.datasette.io/fixtures/facetable?_facet_array=tags) by default without URL's parameters as with other default facets.
_I'm new to datasette, and I'm willing to help with a PR if that is not already implemented and I missed it!_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1552/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1081318247,I_kwDOBm6k_c5Ac5tn,1556,"Show count of facet values always, not just for `?_facet_size=max`",9599,simonw,closed,0,,,7571612,Datasette 0.60,1,2021-12-15T17:49:01Z,2022-01-13T22:26:07Z,2021-12-15T17:58:06Z,OWNER,,"> You've caused me to rethink this feature - I no longer think there's value in only showing these numbers if `?_facet_size=max` as opposed to all of the time.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1423#issuecomment-995023410_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1556/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,completed
1077893013,I_kwDOBm6k_c5AP1eV,1551,`keep_blank_values=True` when parsing `request.args`,9599,simonw,closed,0,,,7571612,Datasette 0.60,3,2021-12-12T19:53:07Z,2022-01-13T22:26:04Z,2021-12-12T20:02:01Z,OWNER,,"This code in `TableView` wouldn't be necessary: https://github.com/simonw/datasette/blob/492f9835aa7e90540dd0c6324282b109f73df71b/datasette/views/table.py#L396-L399
If that happened here instead: https://github.com/simonw/datasette/blob/492f9835aa7e90540dd0c6324282b109f73df71b/datasette/utils/asgi.py#L98-L100
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1518#issuecomment-991827468_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1551/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
777140799,MDU6SXNzdWU3NzcxNDA3OTk=,1166,Adopt Prettier for JavaScript code formatting,9599,simonw,open,0,,,,,10,2020-12-31T21:25:27Z,2022-01-13T22:22:18Z,,OWNER,,https://prettier.io/ - I'm going to go with 2 spaces.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1166/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
520655983,MDU6SXNzdWU1MjA2NTU5ODM=,619,"""Invalid SQL"" page should let you edit the SQL",9599,simonw,closed,0,,,,,14,2019-11-10T20:54:12Z,2022-01-13T22:21:42Z,2021-06-02T04:15:54Z,OWNER,,"https://latest.datasette.io/fixtures?sql=select%0D%0A++*%0D%0Afrom%0D%0A++%5Bfoo%5D
Would be useful if this page showed you the invalid SQL you entered so you can edit it and try again.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/619/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
642651572,MDU6SXNzdWU2NDI2NTE1NzI=,860,Plugin hook for instance/database/table metadata,9599,simonw,closed,0,,,,,10,2020-06-21T22:20:25Z,2022-01-13T22:21:42Z,2021-06-26T22:56:28Z,OWNER,,"I'm not happy with how `metadata.(json|yaml)` keeps growing new features. Rather than having a single plugin hook for all of `metadata.json` I'm going to split out the feature that shows actual real metadata for tables and databases - `source`, `license` etc - into its own plugin-powered mechanism.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/357#issuecomment-647189045_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/860/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
681334912,MDU6SXNzdWU2ODEzMzQ5MTI=,942,Support column descriptions in metadata.json,9599,simonw,closed,0,,,,,18,2020-08-18T20:52:00Z,2022-01-13T22:21:42Z,2021-08-12T23:53:24Z,OWNER,,"Could look something like this:
```json
{
""title"": ""Five Thirty Eight"",
""license"": ""CC Attribution 4.0 License"",
""license_url"": ""https://creativecommons.org/licenses/by/4.0/"",
""source"": ""fivethirtyeight/data on GitHub"",
""source_url"": ""https://github.com/fivethirtyeight/data"",
""databases"": {
""fivethirtyeight"": {
""tables"": {
""mueller-polls/mueller-approval-polls"": {
""description_html"": ""....
"",
""columns"": {
""name_of_column"": ""column_description goes here""
}
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/942/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
712202333,MDU6SXNzdWU3MTIyMDIzMzM=,982,"SQL editor should allow execution of write queries, if you have permission",9599,simonw,open,0,,,,,2,2020-09-30T19:04:35Z,2022-01-13T22:21:29Z,,OWNER,,"The `datasette-write` plugin provides this at the moment https://github.com/simonw/datasette-write - but it feels like it should be a built-in capability, protected by a default permission.
UI concept: if you have write permission then the existing SQL editor gets an ""execute write"" checkbox underneath it.
JavaScript can spot if you appear to be trying to execute an UPDATE or INSERT or DELETE query and check that checkbox for you.
If you link to a query page with a non-SELECT then that query will be displayed in the box ready for you to POST submit it. The page will also then get ""cannot be embedded"" headers to protect against clickjacking.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/982/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
733999615,MDU6SXNzdWU3MzM5OTk2MTU=,1079,Handle long breadcrumbs better with new menu,9599,simonw,open,0,,,,,1,2020-11-01T15:57:41Z,2022-01-13T22:21:29Z,,OWNER,,"On this page when signed in as root: https://latest.datasette.io/fixtures/roadside_attraction_characteristics/1
![EF921CB1-625F-4D04-A850-490B812A72B3](https://user-images.githubusercontent.com/9599/97807807-db0fbf80-1c17-11eb-9c77-ae5169b12c3d.jpeg)
![A49D8B76-5ACF-4F71-A8B4-21A44F5C8D51](https://user-images.githubusercontent.com/9599/97807809-dea34680-1c17-11eb-9511-a49af56a4bd2.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1079/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
776634318,MDU6SXNzdWU3NzY2MzQzMTg=,1164,Mechanism for minifying JavaScript that ships with Datasette,9599,simonw,open,0,,,,,9,2020-12-30T20:59:06Z,2022-01-13T22:21:29Z,,OWNER,,"> If I'm going to minify it I'll need to figure out a build step in Datasette itself so that I can easily work on that minified version.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752748496_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1164/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
776635426,MDU6SXNzdWU3NzY2MzU0MjY=,1165,Mechanism for executing JavaScript unit tests,9599,simonw,open,0,,,,,9,2020-12-30T21:02:34Z,2022-01-13T22:21:29Z,,OWNER,,"> I'm going to need to add JavaScript unit tests for this new plugin system.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/983#issuecomment-752757289_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1165/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
323718842,MDU6SXNzdWUzMjM3MTg4NDI=,268,Mechanism for ranking results from SQLite full-text search,9599,simonw,open,0,,,,,12,2018-05-16T17:36:40Z,2022-01-13T22:21:28Z,,OWNER,,This isn't particularly straight-forward - all the more reason for Datasette to implement it for you. This article is helpful: http://charlesleifer.com/blog/using-sqlite-full-text-search-with-python/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/268/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
707849175,MDU6SXNzdWU3MDc4NDkxNzU=,974,static assets and favicon aren't cached by the browser,45416,obra,open,0,,,,,1,2020-09-24T04:44:55Z,2022-01-13T22:21:28Z,,NONE,,"Using datasette to solve some frustrating problems with our fulfillment provider today, I was surprised to see repeated requests for assets under /-/static and the favicon. While it won't likely be a huge performance bottleneck, I bet datasette would feel a bit zippier if you had Uvicorn serving up some caching-related headers telling the browser it was safe to cache static assets.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/974/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
534629631,MDU6SXNzdWU1MzQ2Mjk2MzE=,650,Add a glossary to the documentation,9599,simonw,open,0,,,,,3,2019-12-09T00:23:45Z,2022-01-13T22:04:56Z,,OWNER,,"Call it `glossary.rst` - it can use a definition list something like this:
```rst
.. _glossary:
Glossary
========
Term
A definition of the term.
Another term
Another definition.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/650/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
895686039,MDU6SXNzdWU4OTU2ODYwMzk=,1336,Document turning on WAL for live served SQLite databases,9599,simonw,open,0,,,,,1,2021-05-19T17:08:58Z,2022-01-13T21:55:59Z,,OWNER,,"Datasette docs don't talk about WAL yet, which allows you to safely serve reads from a database file while it is accepting writes.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1336/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1100499619,I_kwDOBm6k_c5BmEqj,1592,Row pages should show links to foreign keys,9599,simonw,open,0,,,,,1,2022-01-12T15:50:20Z,2022-01-12T15:52:17Z,,OWNER,,Refs #1518 refactor.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1592/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1091838742,I_kwDOBm6k_c5BFCMW,1585,Fire base caching for `publish cloudrun`,9599,simonw,open,0,,,,,1,2022-01-01T15:38:15Z,2022-01-01T15:40:38Z,,OWNER,,"https://gist.github.com/steren/03d3e58c58c9a53fd49bb78f58541872 has a recipe for this, via https://twitter.com/steren/status/1477038411114446848
Could this enable easier vanity URLs of the format `https://$project_id.web.app/`? How about CDN caching?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1585/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1091257796,I_kwDOBm6k_c5BC0XE,1584,give error with recursive sql,58088336,tunguyenatwork,open,0,,,,,0,2021-12-30T18:53:16Z,2021-12-30T18:53:16Z,,NONE,,"I got an error ""near ""WITH"": syntax error"" after I upgraded to version 0.59 from 0.52.4. This error is related to recursive sql. It works great on the previous version but it failed after upgraded. Below is an example of sql:
WITH RECURSIVE manager_of(position, super_position) AS (SELECT position, case ifnull(INDIRECT_SUPER_POSITION,'') when '' then super_position else INDIRECT_SUPER_POSITION end as SUPER_POSITION FROM position where super_position<>'SGV000000001' and super_position!='' and position <> super_position),chain_manager_of_position(position, level) AS (SELECT super_position, 1 as level FROM manager_of WHERE super_position!='' and (position=:pos or position in (Select position from employee where employee=:ein)) UNION ALL SELECT super_position, level+1 as level FROM manager_of JOIN chain_manager_of_position USING(position)) SELECT * FROM chain_manager_of_position left join employee using(position) where employee is not NULL order by level limit 1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1584/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1090810196,I_kwDOBm6k_c5BBHFU,1583,consider adding deletion step of cloudbuild artifacts to gcloud publish,536941,fgregg,open,0,,,,,1,2021-12-30T00:33:23Z,2021-12-30T00:34:16Z,,CONTRIBUTOR,,"right now, as part of the the publish process images and other artifacts are stored to gcloud's cloud storage before being deployed to cloudrun.
after successfully deploying, it would be nice if the the script deleted these artifacts. otherwise, if you have regularly scheduled build process, you can end up paying to store lots of out of date artifacts.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1583/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
770598024,MDU6SXNzdWU3NzA1OTgwMjQ=,1152,Efficiently calculate list of databases/tables a user can view,9599,simonw,open,0,,,,,12,2020-12-18T06:13:01Z,2021-12-27T23:04:31Z,,OWNER,,"> The homepage currently performs a massive flurry of permission checks - one for each, database, table and view: https://github.com/simonw/datasette/blob/0.53/datasette/views/index.py#L21-L75
>
> A paginated version of this is a little daunting as the permission checks would have to be carried out in every single table just to calculate the count that will be paginated.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747864831_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1152/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1087919372,I_kwDOBm6k_c5A2FUM,1578,Confirm if documented nginx proxy config works for row pages with escaped characters in their primary key,9599,simonw,open,0,,,,,4,2021-12-23T18:27:59Z,2021-12-24T21:33:19Z,,OWNER,,"Found this while working on https://github.com/simonw/datasette-tiddlywiki
Then clicking on `/tiddlywiki/tiddlers/%24%3A%2FDefaultTiddlers` returns a 404.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1578/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
781262510,MDU6SXNzdWU3ODEyNjI1MTA=,1181,"Certain database names results in 404: ""Database not found: None""",1470389,jieter,closed,0,,,6346396,Datasette 0.54,4,2021-01-07T12:01:16Z,2021-12-21T18:25:15Z,2021-01-25T05:13:19Z,NONE,,"I have a file named `test-database (1).sqlite`. When requesting the home route `/`, I see datasette is able to read it correctly:
However, if I click any of the links, datasette replies with: `Error 404 Database not found: None`
It seems the hash is crucial, as renaming the file to `database (1).sqlite` makes the error go away.
This lines checks for a single dash:
https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/views/base.py#L184
```
$ datasette test-database\ \(1\).sqlite
INFO: Started server process [68314]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
INFO: 127.0.0.1:54043 - ""GET /favicon.ico HTTP/1.1"" 200 OK
INFO: 127.0.0.1:54043 - ""GET / HTTP/1.1"" 200 OK
...
INFO: 127.0.0.1:54044 - ""GET /favicon.ico HTTP/1.1"" 200 OK
INFO: 127.0.0.1:54044 - ""GET /test-database (1) HTTP/1.1"" 404 Not Found
```
Version:
```
$ datasette --version
datasette, version 0.53
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1181/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1079129258,PR_kwDOBm6k_c4vynly,1554,TableView refactor,9599,simonw,closed,0,,,,,3,2021-12-13T23:16:04Z,2021-12-20T23:52:11Z,2021-12-20T23:52:04Z,OWNER,simonw/datasette/pulls/1554,"I'm starting a PR with almost nothing in it so I can use the GitHub code commenting feature to add a bunch of comments to the code I intend to refactor.
Related issues:
- #617
- #715
- #870
- #1518",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1554/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1084257842,I_kwDOBm6k_c5AoHYy,1575,__call__() got an unexpected keyword argument 'specname',9599,simonw,closed,0,,,,,1,2021-12-20T01:24:04Z,2021-12-20T01:48:03Z,2021-12-20T01:47:57Z,OWNER,,"> I've installed the alpha version but get an error when starting up Datasette:
```
Traceback (most recent call last):
File ""/Users/tim/.pyenv/versions/stock-exchange/bin/datasette"", line 5, in
from datasette.cli import cli
File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/cli.py"", line 15, in
from .app import Datasette, DEFAULT_SETTINGS, SETTINGS, SQLITE_LIMIT_ATTACHED, pm
File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/app.py"", line 31, in
from .views.database import DatabaseDownload, DatabaseView
File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/views/database.py"", line 25, in
from datasette.plugins import pm
File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/plugins.py"", line 29, in
mod = importlib.import_module(plugin)
File ""/Users/tim/.pyenv/versions/3.8.5/lib/python3.8/importlib/__init__.py"", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File ""/Users/tim/.pyenv/versions/3.8.5/envs/stock-exchange/lib/python3.8/site-packages/datasette/filters.py"", line 9, in
@hookimpl(specname=""filters_from_request"")
TypeError: __call__() got an unexpected keyword argument 'specname'
```
_Originally posted by @wragge in https://github.com/simonw/datasette/issues/1547#issuecomment-997511968_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1575/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083657868,I_kwDOBm6k_c5Al06M,1565,Documented JavaScript variables on different templates made available for plugins,9599,simonw,open,0,,,,,8,2021-12-17T22:30:51Z,2021-12-19T22:37:29Z,,OWNER,,"While working on https://github.com/simonw/datasette-leaflet-freedraw/issues/10 I found myself writing this atrocity to figure out the SQL query used for a specific table page:
```javascript
let innerSql = Array.from(document.getElementsByTagName(""span"")).filter(
el => el.innerText == ""View and edit SQL""
)[0].parentElement.getAttribute(""title"")
```
This is obviously bad - it's very brittle, and will break if I ever change the text on that link (like localizing it for example).
Instead, I think pages like that one should have a block of script at the bottom something like this:
```javascript
window.datasette = window.datasette || {};
datasette.view_name = 'table';
datasette.table_sql = 'select * from ...';
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1565/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1084185188,I_kwDOBm6k_c5An1pk,1573,Make trace() a documented internal API,9599,simonw,open,0,,,,,1,2021-12-19T20:32:56Z,2021-12-19T21:13:13Z,,OWNER,,"This should be documented so plugin authors can use it to add their own custom traces: https://github.com/simonw/datasette/blob/8f311d6c1d9f73f4ec643009767749c17b5ca5dd/datasette/tracer.py#L28-L52
Including the new `kwargs` pattern I added in #1571: https://github.com/simonw/datasette/blob/f65817000fdf87ce8a0c23edc40784ebe33b5842/datasette/database.py#L128-L132",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1573/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1076057610,I_kwDOBm6k_c5AI1YK,1546,validating the sql,50336793,jadsongmatos,closed,0,,,,,1,2021-12-09T21:35:57Z,2021-12-18T02:05:17Z,2021-12-18T02:05:16Z,NONE,,Could someone tell me that part of the code is responsible for validating the sql that guarantees that only a table can be read,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1546/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
520681725,MDU6SXNzdWU1MjA2ODE3MjU=,621,Syntax for ?_through= that works as a form field,9599,simonw,open,0,,,,,7,2019-11-11T00:19:03Z,2021-12-18T01:42:33Z,,OWNER,,"The current syntax for `?_through=` uses JSON to avoid any risk of confusion with table or column names that contain special characters.
This means you can't target a form field at it.
We should be able to support both - `?x.y.z=value` for tables and columns with ""regular"" names, falling back to the current JSON syntax for columns or tables that won't work with the key/value syntax.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/621/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
519613116,MDU6SXNzdWU1MTk2MTMxMTY=,617,Refactor TableView.data() method,9599,simonw,closed,0,,,,,9,2019-11-08T01:55:41Z,2021-12-18T01:41:47Z,2021-12-11T19:17:11Z,OWNER,,"This is by far the most complex piece of Datasette - the `TableView.data()` method is over 500 lines long and is increasingly getting in the way of cleanly implementing new features (e.g. #615 and #613).
Need to break it up into smaller, cleaner pieces.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/617/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
445850934,MDU6SXNzdWU0NDU4NTA5MzQ=,473,Plugin hook: filters_from_request,9599,simonw,closed,0,,,,,13,2019-05-19T18:44:33Z,2021-12-17T23:11:30Z,2021-12-17T19:02:17Z,OWNER,,"I meant to add this as part of the facets plugin mechanism but didn't quite get to it. Original idea was to allow plugins to register extra filters, as seen in `datasette/filters.py`:
https://github.com/simonw/datasette/blob/260085838887ee343f4d3b177c422e7aef5ade9d/datasette/filters.py#L83-L98",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/473/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1082743068,PR_kwDOBm6k_c4v-izc,1559,"filters_from_request plugin hook, now used in TableView",9599,simonw,closed,0,,,,,6,2021-12-16T23:59:33Z,2021-12-17T23:09:41Z,2021-12-17T19:02:15Z,OWNER,simonw/datasette/pulls/1559,"New plugin hook, refs #473
Used it to extract the logic from TableView that handles _search and
_through and _where - refs #1518",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1559/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1083246400,PR_kwDOBm6k_c4wAMK8,1562,"Update janus requirement from <0.8,>=0.6.2 to >=0.6.2,<1.1",49699333,dependabot[bot],closed,0,,,,,4,2021-12-17T13:11:10Z,2021-12-17T23:08:29Z,2021-12-17T23:08:28Z,CONTRIBUTOR,simonw/datasette/pulls/1562,"Updates the requirements on [janus](https://github.com/aio-libs/janus) to permit the latest version.
Release notes
Sourced from janus's releases .
janus 1.0.0 release
Dropped Python 3.6 support
Janus is marked as stable, no API changes was made for years
Changelog
Sourced from janus's changelog .
1.0.0 (2021-12-17)
0.7.0 (2021-11-24)
Add SyncQueue and AsyncQueue Protocols to provide type hints for sync and async queues #374
0.6.2 (2021-10-24)
Fix Python 3.10 compatibility #358
0.6.1 (2020-10-26)
0.6.0 (2020-10-10)
0.5.0 (2020-04-23)
Remove explicit loop arguments and forbid creating queues outside event loops #246
0.4.0 (2018-07-28)
0.3.2 (2018-07-06)
Fixed python 3.7 support #97
... (truncated)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1562/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
793002853,MDExOlB1bGxSZXF1ZXN0NTYwNzYwMTQ1,1204,WIP: Plugin includes,9599,simonw,open,0,,,,,3,2021-01-25T03:59:06Z,2021-12-17T07:10:49Z,,OWNER,simonw/datasette/pulls/1204,"Refs #1191
Next steps:
- [ ] Get comfortable that this pattern is the right way to go
- [ ] Implement it for all of the other pages, not just the table page
- [ ] Add a new set of plugin tests that exercise ALL of these new hook locations
- [ ] Document, then ship",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1204/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1,
636511683,MDU6SXNzdWU2MzY1MTE2ODM=,830,Redesign register_facet_classes plugin hook,9599,simonw,open,0,,,3268330,Datasette 1.0,3,2020-06-10T20:03:27Z,2021-12-16T19:58:22Z,,OWNER,,"Nothing uses this plugin hook yet, so the design is not yet proven.
I'm going to build a real plugin against it and use that process to inform any design changes that may need to be made.
I'll add a warning about this to the documentation.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/830/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1079111498,I_kwDOBm6k_c5AUe9K,1553,if csv export is truncated in non streaming mode set informative response header,536941,fgregg,open,0,,,,,3,2021-12-13T22:50:44Z,2021-12-16T19:17:28Z,,CONTRIBUTOR,,"streaming mode is currently not enabled for custom queries, so the queries will be truncated to max row limit.
it would be great if a response is truncated that an header signalling that was set in the header.
i need to write some pagination code for getting full results back for a custom query and it would make the code much better if i could reliably known when there is nothing more to limit/offset ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1553/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
741231849,MDU6SXNzdWU3NDEyMzE4NDk=,1087,Idea: ?_extra=urls for getting back URLs to useful things,9599,simonw,open,0,,,,,0,2020-11-12T02:55:41Z,2021-12-15T18:06:16Z,,OWNER,,"Working on https://github.com/simonw/datasette-search-all/issues/10 made me realize that sometimes it can be difficult to calculate the URL for a database, table or row within Datasette.
It would be useful to have an optional extra JSON extension (using `?_extra=` from #262) that can help with this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1087/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
705840673,MDU6SXNzdWU3MDU4NDA2NzM=,972,Support faceting against arbitrary SQL queries,9599,simonw,open,0,,,,,1,2020-09-21T19:00:43Z,2021-12-15T18:02:20Z,,OWNER,,"> ... support for running facets against arbitrary custom SQL queries is half-done in that facets now execute against wrapped subqueries as-of ea66c45df96479ef66a89caa71fff1a97a862646
>
> https://github.com/simonw/datasette/blob/ea66c45df96479ef66a89caa71fff1a97a862646/datasette/facets.py#L192-L200
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/971#issuecomment-696307922_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/972/reactions"", ""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 3, ""rocket"": 0, ""eyes"": 0}",,
962391325,MDU6SXNzdWU5NjIzOTEzMjU=,1423,Show count of facet values if ?_facet_size=max,9599,simonw,closed,0,,,,,9,2021-08-06T04:42:20Z,2021-12-15T17:48:40Z,2021-08-16T18:56:43Z,OWNER,,"I sometimes want to get a count of the values in a facet - if it's a facet of US states for example I want to know if all 50 are represented.
Idea: if `?_facet_size=max` is present, add a count to the facet heading. So on:
https://latest.datasette.io/fixtures/compound_three_primary_keys?_facet=content&_facet_size=max&_facet=pk1&_facet=pk2#facet-pk2
It could have something like this:
Note that the first column shows >1000 - because in that case we've truncated the facet calculation since the maximum allowed returned rows is 1000.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1423/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1072106103,I_kwDOBm6k_c4_5wp3,1542,feature request: order and dependency of plugins (that use js),33631,fs111,open,0,,,,,1,2021-12-06T12:40:45Z,2021-12-15T17:47:08Z,,NONE,,"I have been playing with datasette for the last couple of weeks and it is great! I am a big fan of `datasette-cluster-map` and wanted to enhance it a bit with a what I would call a sub-plugin. I basically want to add more controls to the map that cluster map provides. I have been looking into its code and how the plugin management works, but it seems what I am trying to do is not doable without hacks in js.
Basically what would like to have is a way to say load my plugin after the plugins I depend on have been loaded and rendered. There seems to be no prior art where plugins have these dependencies on the js level so I was wondering if that could be added or if it exists how to do it.
Basically what I want to do is:
my-awesome-plugin has a dependency on datastte-cluster-map. Whenever datasette cluster map has finished rendering on page load, call my plugin, but no earlier. To make that work datasette probably needs some total order in which way plugins are loaded intialized.
Since I am new to datastte, I may be missing something obvious, so please let me know if the above makes no sense.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1542/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1072135269,PR_kwDOBm6k_c4vb__Y,1543,Bump black from 21.11b1 to 21.12b0,49699333,dependabot[bot],closed,0,,,,,1,2021-12-06T13:11:16Z,2021-12-13T23:22:29Z,2021-12-13T23:22:29Z,CONTRIBUTOR,simonw/datasette/pulls/1543,"Bumps [black](https://github.com/psf/black) from 21.11b1 to 21.12b0.
Release notes
Sourced from black's releases .
21.12b0
Black
Fix determination of f-string expression spans (#2654 )
Fix bad formatting of error messages about EOF in multi-line statements (#2343 )
Functions and classes in blocks now have more consistent surrounding spacing (#2472 )
Jupyter Notebook support
Cell magics are now only processed if they are known Python cell magics. Earlier, all cell magics were tokenized, leading to possible indentation errors e.g. with %%writefile
. (#2630 )
Fix assignment to environment variables in Jupyter Notebooks (#2642 )
Python 3.10 support
Point users to using --target-version py310
if we detect 3.10-only syntax (#2668 )
Fix match
statements with open sequence subjects, like match a, b:
or match a, *b:
(#2639 ) (#2659 )
Fix match
/case
statements that contain match
/case
soft keywords multiple times, like match re.match()
(#2661 )
Fix case
statements with an inline body (#2665 )
Fix styling of starred expressions inside match
subject (#2667 )
Fix parser error location on invalid syntax in a match
statement (#2649 )
Fix Python 3.10 support on platforms without ProcessPoolExecutor (#2631 )
Improve parsing performance on code that uses match
under --target-version py310
up to ~50% (#2670 )
Packaging
Thank you!
@isidentical
for the polishing up 3.10 syntax support (which they contributed in the first place!)
@MarcoGorelli
for their ever-continuing work on Black's jupyter support
@jalaziz
for cleaning up our Pyinstaller CD workflow
@hauntsaninja
for helping us drop the regex
dependency
And also congrats to first contributors!
Changelog
Sourced from black's changelog .
21.12b0
Black
Fix determination of f-string expression spans (#2654 )
Fix bad formatting of error messages about EOF in multi-line statements (#2343 )
Functions and classes in blocks now have more consistent surrounding spacing (#2472 )
Jupyter Notebook support
Cell magics are now only processed if they are known Python cell magics. Earlier, all
cell magics were tokenized, leading to possible indentation errors e.g. with
%%writefile
. (#2630 )
Fix assignment to environment variables in Jupyter Notebooks (#2642 )
Python 3.10 support
Point users to using --target-version py310
if we detect 3.10-only syntax (#2668 )
Fix match
statements with open sequence subjects, like match a, b:
or
match a, *b:
(#2639 ) (#2659 )
Fix match
/case
statements that contain match
/case
soft keywords multiple
times, like match re.match()
(#2661 )
Fix case
statements with an inline body (#2665 )
Fix styling of starred expressions inside match
subject (#2667 )
Fix parser error location on invalid syntax in a match
statement (#2649 )
Fix Python 3.10 support on platforms without ProcessPoolExecutor (#2631 )
Improve parsing performance on code that uses match
under --target-version py310
up to ~50% (#2670 )
Packaging
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.11b1&new-version=21.12b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1543/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1076834768,PR_kwDOBm6k_c4vrZxV,1548,"Update pytest-xdist requirement from <2.5,>=2.2.1 to >=2.2.1,<2.6",49699333,dependabot[bot],closed,0,,,,,1,2021-12-10T13:12:06Z,2021-12-13T23:22:22Z,2021-12-13T23:22:21Z,CONTRIBUTOR,simonw/datasette/pulls/1548,"Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version.
Changelog
Sourced from pytest-xdist's changelog .
pytest-xdist 2.5.0 (2021-12-10)
Features
[#722](https://github.com/pytest-dev/pytest-xdist/issues/722) <https://github.com/pytest-dev/pytest-xdist/issues/722>
_: Full compatibility with pytest 7 - no deprecation warnings or use of legacy features.
[#733](https://github.com/pytest-dev/pytest-xdist/issues/733) <https://github.com/pytest-dev/pytest-xdist/issues/733>
_: New --dist=loadgroup
option, which ensures all tests marked with @pytest.mark.xdist_group
run in the same session/worker. Other tests run distributed as in --dist=load
.
Trivial Changes
[#708](https://github.com/pytest-dev/pytest-xdist/issues/708) <https://github.com/pytest-dev/pytest-xdist/issues/708>
_: Use @pytest.hookspec
decorator to declare hook options in newhooks.py
to avoid warnings in pytest 7.0
.
[#719](https://github.com/pytest-dev/pytest-xdist/issues/719) <https://github.com/pytest-dev/pytest-xdist/issues/719>
_: Use up-to-date setup.cfg
/pyproject.toml
packaging setup.
[#720](https://github.com/pytest-dev/pytest-xdist/issues/720) <https://github.com/pytest-dev/pytest-xdist/issues/720>
_: Require pytest>=6.2.0.
[#721](https://github.com/pytest-dev/pytest-xdist/issues/721) <https://github.com/pytest-dev/pytest-xdist/issues/721>
_: Started using type annotations and mypy checking internally. The types are incomplete and not published.
pytest-xdist 2.4.0 (2021-09-20)
Features
[#696](https://github.com/pytest-dev/pytest-xdist/issues/696) <https://github.com/pytest-dev/pytest-xdist/issues/696>
_: On Linux, the process title now changes to indicate the current worker state (running/idle).
Depends on the setproctitle <https://pypi.org/project/setproctitle/>
__ package, which can be installed with pip install pytest-xdist[setproctitle]
.
[#704](https://github.com/pytest-dev/pytest-xdist/issues/704) <https://github.com/pytest-dev/pytest-xdist/issues/704>
_: Add support for Python 3.10.
pytest-xdist 2.3.0 (2021-06-16)
Deprecations and Removals
[#654](https://github.com/pytest-dev/pytest-xdist/issues/654) <https://github.com/pytest-dev/pytest-xdist/issues/654>
_: Python 3.5 is no longer supported.
Features
[#646](https://github.com/pytest-dev/pytest-xdist/issues/646) <https://github.com/pytest-dev/pytest-xdist/issues/646>
_: Add --numprocesses=logical
flag, which automatically uses the number of logical CPUs available, instead of physical CPUs with auto
.
... (truncated)
Commits
13f3934
Remove unnecessary skip from test_logfinish_hook as we require pytest>=6.2
c76d562
Skip test_warning_captured_deprecated_in_pytest_6 in pytest>=7.1
5f78c71
Fix CHANGELOG header
c8bbc03
Release 2.5.0
8dbf367
Merge pull request #738 from pytest-dev/pre-commit-ci-update-config
a25c14b
[pre-commit.ci] pre-commit autoupdate
110c114
Merge pull request #734 from nicoddemus/revamp-readme
83bdbf4
Revamp README
630c1eb
Merge pull request #733 from baekdohyeop/feature-loadgroup
62e50d0
Address review
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1548/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1077628073,I_kwDOBm6k_c5AO0yp,1550,Research option for returning all rows from arbitrary query,9599,simonw,open,0,,,,,2,2021-12-11T19:31:11Z,2021-12-11T23:43:24Z,,OWNER,,"Inspired by thinking about #1549 - returning ALL rows from an arbitrary query is a lot easier if you just run that query and keep iterating over the cursor.
I've avoided doing that in the past because it could tie up a connection for a long time - but in private instances this wouldn't be such a problem.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1550/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1068791148,I_kwDOBm6k_c4_tHVs,1540,Idea: hover to reveal details of linked row,9599,simonw,open,0,,,,,6,2021-12-01T19:28:07Z,2021-12-09T23:38:39Z,,OWNER,,"
Hovering over that could work a little bit like GitHub issue links:
![hover](https://user-images.githubusercontent.com/9599/144300537-9cd9e9af-ac16-42db-842f-37661bc94063.gif)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1540/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
863884805,MDU6SXNzdWU4NjM4ODQ4MDU=,1304,"Document how to send multiple values for ""Named parameters"" ",9308268,rayvoelker,open,0,,,,,4,2021-04-21T13:19:06Z,2021-12-08T03:23:14Z,,NONE,,"https://docs.datasette.io/en/stable/sql_queries.html#named-parameters
I thought that I had seen an example of how to do this example below, but I can't seem to find it
```sql
select
*
from
bib
where
bib.bib_record_num in (1008088,1008092)
```
```sql
select
*
from
bib
where
bib.bib_record_num in (:bib_record_numbers)
```
![image](https://user-images.githubusercontent.com/9308268/115558839-2333a480-a281-11eb-85e6-ce3bada79140.png)
https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-204d100?sql=select%0D%0A++*%0D%0Afrom%0D%0A++bib%0D%0Awhere%0D%0A++bib.bib_record_num+in+%28%3Abib_record_numbers%29&bib_record_numbers=1008088%2C1008092
Or, maybe this isn't a fully supported feature.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1304/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1073712378,I_kwDOBm6k_c4__4z6,1544,Code that detects the label column for a table is case-sensitive,9599,simonw,closed,0,,,,,2,2021-12-07T20:01:25Z,2021-12-07T20:03:43Z,2021-12-07T20:03:43Z,OWNER,,I just noticed that a column called `Name` is not being picked up as the label column for a table.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1544/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
964322136,MDU6SXNzdWU5NjQzMjIxMzY=,1426,"Manage /robots.txt in Datasette core, block robots by default",9599,simonw,open,0,,,,,9,2021-08-09T19:56:56Z,2021-12-04T07:11:29Z,,OWNER,,"See accompanying Twitter thread: https://twitter.com/simonw/status/1424820203603431439
> Datasette currently has a plugin for configuring robots.txt, but I'm beginning to think it should be part of core and crawlers should be blocked by default - having people explicitly opt-in to having their sites crawled and indexed feels a lot safer https://datasette.io/plugins/datasette-block-robots
I have a lot of Datasettes deployed now, and tailing logs shows that they are being *hammered* by search engine crawlers even though many of them are not interesting enough to warrant indexing.
I'm starting to think blocking crawlers would actually be a better default for most people, provided it was well documented and easy to understand how to allow them.
Default-deny is usually a better policy than default-allow!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1426/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1069881276,I_kwDOBm6k_c4_xRe8,1541,Different default layout for row page,9599,simonw,open,0,,,,,1,2021-12-02T18:56:36Z,2021-12-02T18:56:54Z,,OWNER,,"The row page displays as a table even though it only has one table row.
maybe default to the same display as the narrow page version, even for wide pages?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1541/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1058790545,I_kwDOBm6k_c4_G9yR,1519,base_url is omitted in JSON and CSV views,157158,phubbard,closed,0,,,,,22,2021-11-19T18:10:45Z,2021-12-01T17:50:09Z,2021-11-20T19:11:21Z,NONE,,"I have a datasette deployment, using Apache2 to reverse proxy:
ProxyPass /ged http://thor.phfactor.net:8001
ProxyPreserveHost On
In settings.json I have
```json
{
""base_url"": ""/ged/"",
""trace_debug"": 1,
""template_debug"": 1
}
```
and datasette works correctly. However, if you view a query and then click on the 'This data as json, CSV' both links omit the base_url prefix and are therefore 404.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1519/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1067775061,I_kwDOBm6k_c4_pPRV,1539,Research PRAGMA query_only,9599,simonw,open,0,,,,,0,2021-11-30T23:30:24Z,2021-11-30T23:30:24Z,,OWNER,,"https://www.sqlite.org/pragma.html#pragma_query_only
> The query_only pragma prevents data changes on database files when enabled. When this pragma is enabled, any attempt to CREATE, DELETE, DROP, INSERT, or UPDATE will result in an [SQLITE_READONLY](https://www.sqlite.org/rescode.html#readonly) error. However, the database is not truly read-only. You can still run a [checkpoint](https://www.sqlite.org/wal.html#ckpt) or a [COMMIT](https://www.sqlite.org/lang_transaction.html) and the return value of the [sqlite3_db_readonly()](https://www.sqlite.org/c3ref/db_readonly.html) routine is not affected.
Would it be worth adding this as an extra protection against accidental writes to a DB file over a read-only connection?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1539/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1059509927,I_kwDOBm6k_c4_Jtan,1525,"""Links from other tables"" broken for columns starting with underscore",9599,simonw,closed,0,,,,,3,2021-11-21T22:55:08Z,2021-11-30T06:39:01Z,2021-11-30T06:34:35Z,OWNER,,"Same bug as #1506, this time it's this link or the row page:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1525/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1062414013,PR_kwDOBm6k_c4u9wQq,1529,"Update janus requirement from <0.7,>=0.6.2 to >=0.6.2,<0.8",49699333,dependabot[bot],closed,0,,,,,1,2021-11-24T13:12:42Z,2021-11-30T02:37:13Z,2021-11-30T02:37:13Z,CONTRIBUTOR,simonw/datasette/pulls/1529,"Updates the requirements on [janus](https://github.com/aio-libs/janus) to permit the latest version.
Changelog
Sourced from janus's changelog .
0.7.0 (2021-11-24)
Add SyncQueue and AsyncQueue Protocols to provide type hints for sync and async queues #374
0.6.2 (2021-10-24)
Fix Python 3.10 compatibility #358
0.6.1 (2020-10-26)
0.6.0 (2020-10-10)
0.5.0 (2020-04-23)
Remove explicit loop arguments and forbid creating queues outside event loops #246
0.4.0 (2018-07-28)
0.3.2 (2018-07-06)
Fixed python 3.7 support #97
0.3.1 (2018-01-30)
Fixed bug with join() in case tasks are added by sync_q.put() #75
... (truncated)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1529/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1057340779,PR_kwDOBm6k_c4utsKs,1516,Bump black from 21.9b0 to 21.11b1,49699333,dependabot[bot],closed,0,,,,,1,2021-11-18T13:11:12Z,2021-11-30T02:35:29Z,2021-11-30T02:35:29Z,CONTRIBUTOR,simonw/datasette/pulls/1516,"Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.11b1.
Release notes
Sourced from black's releases .
21.11b1
Black
Bumped regex version minimum to 2021.4.4 to fix Pattern class usage (#2621 )
21.11b0
Black
Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
detection (#2592 )
Add experimental PyPy support (#2559 )
Add partial support for the match statement. As it's experimental, it's only enabled
when --target-version py310
is explicitly specified (#2586 )
Add support for parenthesized with (#2586 )
Declare support for Python 3.10 for running Black (#2562 )
Integrations
Fixed vim plugin with Python 3.10 by removing deprecated distutils import (#2610 )
The vim plugin now parses skip_magic_trailing_comma
from pyproject.toml (#2613 )
21.10b0
Black
Document stability policy, that will apply for non-beta releases (#2529 )
Add new --workers
parameter (#2514 )
Fixed feature detection for positional-only arguments in lambdas (#2532 )
Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (#2519 )
Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
even though it has been removed (#2580 )
Deprecate Python 2 formatting support (#2523 )
Blackd
Remove dependency on aiohttp-cors (#2500 )
Bump required aiohttp version to 3.7.4 (#2509 )
Black-Primer
Add primer support for --projects (#2555 )
Print primer summary after individual failures (#2570 )
Integrations
Allow to pass target_version
in the vim plugin (#1319 )
Install build tools in docker file and use multi-stage build to keep the image size
down (#2582 )
Changelog
Sourced from black's changelog .
21.11b1
Black
Bumped regex version minimum to 2021.4.4 to fix Pattern class usage (#2621 )
21.11b0
Black
Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
detection (#2592 )
Add experimental PyPy support (#2559 )
Add partial support for the match statement. As it's experimental, it's only enabled
when --target-version py310
is explicitly specified (#2586 )
Add support for parenthesized with (#2586 )
Declare support for Python 3.10 for running Black (#2562 )
Integrations
Fixed vim plugin with Python 3.10 by removing deprecated distutils import (#2610 )
The vim plugin now parses skip_magic_trailing_comma
from pyproject.toml (#2613 )
21.10b0
Black
Document stability policy, that will apply for non-beta releases (#2529 )
Add new --workers
parameter (#2514 )
Fixed feature detection for positional-only arguments in lambdas (#2532 )
Bumped typed-ast version minimum to 1.4.3 for 3.10 compatibility (#2519 )
Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
even though it has been removed (#2580 )
Deprecate Python 2 formatting support (#2523 )
Blackd
Remove dependency on aiohttp-cors (#2500 )
Bump required aiohttp version to 3.7.4 (#2509 )
Black-Primer
Add primer support for --projects (#2555 )
Print primer summary after individual failures (#2570 )
Integrations
Allow to pass target_version
in the vim plugin (#1319 )
Install build tools in docker file and use multi-stage build to keep the image size
down (#2582 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.9b0&new-version=21.11b1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1516/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1053655062,PR_kwDOBm6k_c4uiE0n,1508,Update docutils requirement from <0.18 to <0.19,49699333,dependabot[bot],closed,0,,,,,1,2021-11-15T13:15:47Z,2021-11-30T02:35:19Z,2021-11-30T02:35:19Z,CONTRIBUTOR,simonw/datasette/pulls/1508,"Updates the requirements on [docutils](http://docutils.sourceforge.net/) to permit the latest version.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1508/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1066023866,PR_kwDOBm6k_c4vIJqi,1537,"Update aiofiles requirement from <0.8,>=0.4 to >=0.4,<0.9",49699333,dependabot[bot],closed,0,,,,,1,2021-11-29T13:13:52Z,2021-11-30T02:29:55Z,2021-11-30T02:29:54Z,CONTRIBUTOR,simonw/datasette/pulls/1537,"Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version.
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1537/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1066288689,I_kwDOBm6k_c4_jkYx,1538,Research pattern for re-registering existing Click tools with register_commands,9599,simonw,closed,0,,,,,3,2021-11-29T17:09:47Z,2021-11-29T17:32:44Z,2021-11-29T17:27:16Z,OWNER,,"Building a Datasette plugin that imports an existing Click CLI tool and re-registers it is proving hard - Click doesn't really want you to do that. I tried this:
```python
from datasette import hookimpl
from git_history.cli import file as git_history_file
@hookimpl
def register_commands(cli):
cli.command(name=""git-history"")(git_history_file.callback)
```
But when I run this:
```
% datasette git-history --help
Usage: datasette git-history [OPTIONS]
Analyze the history of a specific file and write it to SQLite
Options:
--help Show this message and exit.
```
The options are all missing - which means that the command doesn't actually work. Will need to research this pattern separately.
_Originally posted by @simonw in https://github.com/simonw/git-history/issues/21#issuecomment-981835305_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1538/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1058896236,I_kwDOBm6k_c4_HXls,1522,Deploy a live instance of demos/apache-proxy,9599,simonw,closed,0,,,,,34,2021-11-19T20:32:55Z,2021-11-23T03:00:34Z,2021-11-20T18:51:56Z,OWNER,,"> I'll get this working on my laptop first, but then I want to get it up and running on Cloud Run - maybe with a GitHub Actions workflow in this repo that re-deploys it on manual execution.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1521#issuecomment-974322178_
I started by following https://ahmet.im/blog/cloud-run-multiple-processes-easy-way/ - see example in https://github.com/ahmetb/multi-process-container-lazy-solution",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1522/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1059549523,I_kwDOBm6k_c4_J3FT,1526,"Add to vercel.json, rather than overwriting it.",192568,mroswell,closed,0,,,,,2,2021-11-22T00:47:12Z,2021-11-22T04:49:45Z,2021-11-22T04:13:47Z,CONTRIBUTOR,,"I'd like to be able to add to vercel.json. But Datasette overwrites whatever I put in that file. I originally reported this here:
https://github.com/simonw/datasette-publish-vercel/issues/51
In that case, I wanted to do a rewrite... and now I need to do 301 redirects (because we had to rename our site).
Can this be addressed?
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1526/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
273944952,MDU6SXNzdWUyNzM5NDQ5NTI=,93,Package as standalone binary,67420,atomotic,closed,0,,,,,18,2017-11-14T21:14:07Z,2021-11-21T07:00:23Z,2021-11-21T07:00:23Z,NONE,,"hint: more than the docker image a standalone and multiplatform binary (containing the app and the database) could be simpler to distribute.
i would like to investigate the possibility to package everything with [pyinstaller](http://www.pyinstaller.org/) adding the database as a [data file](https://pythonhosted.org/PyInstaller/spec-files.html#adding-data-files)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/93/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1059219106,I_kwDOBm6k_c4_Imai,1524,"Improve Apache proxy documentation, link to demo",9599,simonw,closed,0,,,,,4,2021-11-20T20:03:14Z,2021-11-20T23:34:03Z,2021-11-20T23:34:03Z,OWNER,,"> The latest demo is now live at https://datasette-apache-proxy-demo.fly.dev/prefix/fixtures/sortable?_facet=pk2
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974697824_
I'm going to put out 0.59.3 bugfix release with this, but I'd like to first improve the documentation on https://docs.datasette.io/en/stable/deploying.html#apache-proxy-configuration to highlight the new demo.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1524/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
637395097,MDU6SXNzdWU2MzczOTUwOTc=,838,Incorrect URLs when served behind a proxy with base_url set,79913,tsibley,closed,0,,,6026070,0.51,14,2020-06-11T23:58:55Z,2021-11-20T19:35:48Z,2021-11-20T19:35:48Z,NONE,,"I'm running `datasette serve --config base_url:/foo/ …`, proxying to it with this Apache config:
ProxyPass /foo/ http://localhost:8001/
ProxyPassReverse /foo/ http://localhost:8001/
and then accessing it via `https://example.com/foo/`.
Although many of the URLs in the pages are correct (presumably because they either use absolute paths which include `base_url` or relative paths), the faceting and pagination links still use fully-qualified URLs pointing at `http://localhost:8001`.
I looked into this a little in the source code, and it seems to be an issue anywhere `request.url` or `request.path` is used, as these contain the values for the request between the frontend (Apache) and backend (Datasette) server. Those properties are primarily used via the `path_with_…` family of utility functions and the `Datasette.absolute_url` method.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/838/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1059209412,I_kwDOBm6k_c4_IkDE,1523,Come up with a more elegant solution for base_url than ds.urls.path(),9599,simonw,open,0,,,,,0,2021-11-20T19:05:22Z,2021-11-20T19:05:22Z,,OWNER,,"While fixing #1519 I added a lot of ugly code that looks like this: https://github.com/simonw/datasette/blob/08947fa76433d18988aa1ee1d929bd8320c75fe2/datasette/facets.py#L228-L230
See these two commits in particular: fe687fd0207c4c56c4778d3e92e3505fc4b18172 and 08947fa76433d18988aa1ee1d929bd8320c75fe2
It would be great to come up with a less verbose and error-prone way of handling this problem.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1523/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1058815557,I_kwDOBm6k_c4_HD5F,1521,Docker configuration for exercising Datasette behind Apache mod_proxy,9599,simonw,closed,0,,,,,10,2021-11-19T18:46:18Z,2021-11-19T20:32:29Z,2021-11-19T20:32:29Z,OWNER,,"> Having a live demo running on Cloud Run that proxies through Apache and uses `base_url` would be incredibly useful for replicating and debugging this kind of thing. I wonder how hard it is to run Apache and `mod_proxy` in the same Docker container as Datasette?
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1519#issuecomment-974310208_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1521/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1058803238,I_kwDOBm6k_c4_HA4m,1520,Pattern for avoiding accidental URL over-rides,9599,simonw,open,0,,,,,1,2021-11-19T18:28:05Z,2021-11-19T18:29:26Z,,OWNER,,"Following #1517 I'm experimenting with a plugin that does this:
```python
@hookimpl
def register_routes():
return [
(r""/(?P[^/]+)/(?P[^/]+?)$"", Table().view),
]
```
This is supposed to replace the default table page with new code... but there's a problem: `/-/versions` on that instance now returns 404 `Database '-' does not exist`!
Need to figure out a pattern to avoid that happening. Plugins get to add their routes before Datasette's default routes, which is why this is happening here.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1520/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1033678984,PR_kwDOBm6k_c4tjgJ8,1495,Allow routes to have extra options,536941,fgregg,open,0,,,,,5,2021-10-22T15:00:45Z,2021-11-19T15:36:27Z,,CONTRIBUTOR,simonw/datasette/pulls/1495,"Right now, datasette routes can only be a 2-tuple of `(regex, view_fn)`.
If it was possible for datasette to handle extra options, like [standard Django does](https://docs.djangoproject.com/en/3.2/topics/http/urls/#passing-extra-options-to-view-functions), it would add flexibility for plugin authors.
For example, if extra options were enabled, then it would be easy to make a single table the home page (#1284). This plugin would accomplish it.
```python
from datasette import hookimpl
from datasette.views.table import TableView
@hookimpl
def register_routes(datasette):
return [
(r""^/$"", TableView.as_view(datasette), {'db_name': 'DB_NAME',
'table': 'TABLE_NAME'})
]
```
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1495/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1057996111,I_kwDOBm6k_c4_D71P,1517,Let `register_routes()` over-ride default routes within Datasette,9599,simonw,closed,0,,,3268330,Datasette 1.0,2,2021-11-19T00:22:15Z,2021-11-19T03:20:00Z,2021-11-19T03:07:27Z,OWNER,,"See https://github.com/simonw/datasette/issues/878#issuecomment-973554024_ - right now `register_routes()` can't replace default Datasette routes.
It would be neat if plugins could do this - especially if there was a neat documented way for them to then re-dispatch to the original route code after making some kind of modification.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1517/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1056117435,PR_kwDOBm6k_c4up0R0,1514,Bump black from 21.9b0 to 21.11b0,49699333,dependabot[bot],closed,0,,,,,2,2021-11-17T13:13:55Z,2021-11-18T13:11:17Z,2021-11-18T13:11:15Z,CONTRIBUTOR,simonw/datasette/pulls/1514,"Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.11b0.
Release notes
Sourced from black's releases .
21.11b0
Black
Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
detection (#2592 )
Add experimental PyPy support (#2559 )
Add partial support for the match statement. As it's experimental, it's only enabled
when --target-version py310
is explicitly specified (#2586 )
Add support for parenthesized with (#2586 )
Declare support for Python 3.10 for running Black (#2562 )
Integrations
Fixed vim plugin with Python 3.10 by removing deprecated distutils import (#2610 )
The vim plugin now parses skip_magic_trailing_comma
from pyproject.toml (#2613 )
21.10b0
Black
Document stability policy, that will apply for non-beta releases (#2529 )
Add new --workers
parameter (#2514 )
Fixed feature detection for positional-only arguments in lambdas (#2532 )
Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (#2519 )
Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
even though it has been removed (#2580 )
Deprecate Python 2 formatting support (#2523 )
Blackd
Remove dependency on aiohttp-cors (#2500 )
Bump required aiohttp version to 3.7.4 (#2509 )
Black-Primer
Add primer support for --projects (#2555 )
Print primer summary after individual failures (#2570 )
Integrations
Allow to pass target_version
in the vim plugin (#1319 )
Install build tools in docker file and use multi-stage build to keep the image size
down (#2582 )
Changelog
Sourced from black's changelog .
21.11b0
Black
Warn about Python 2 deprecation in more cases by improving Python 2 only syntax
detection (#2592 )
Add experimental PyPy support (#2559 )
Add partial support for the match statement. As it's experimental, it's only enabled
when --target-version py310
is explicitly specified (#2586 )
Add support for parenthesized with (#2586 )
Declare support for Python 3.10 for running Black (#2562 )
Integrations
Fixed vim plugin with Python 3.10 by removing deprecated distutils import (#2610 )
The vim plugin now parses skip_magic_trailing_comma
from pyproject.toml (#2613 )
21.10b0
Black
Document stability policy, that will apply for non-beta releases (#2529 )
Add new --workers
parameter (#2514 )
Fixed feature detection for positional-only arguments in lambdas (#2532 )
Bumped typed-ast version minimum to 1.4.3 for 3.10 compatibility (#2519 )
Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
even though it has been removed (#2580 )
Deprecate Python 2 formatting support (#2523 )
Blackd
Remove dependency on aiohttp-cors (#2500 )
Bump required aiohttp version to 3.7.4 (#2509 )
Black-Primer
Add primer support for --projects (#2555 )
Print primer summary after individual failures (#2570 )
Integrations
Allow to pass target_version
in the vim plugin (#1319 )
Install build tools in docker file and use multi-stage build to keep the image size
down (#2582 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.9b0&new-version=21.11b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1514/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1056746091,I_kwDOBm6k_c4-_Kpr,1515,Handle foreign keys that point to a non-existent table,9599,simonw,open,0,,,,,0,2021-11-17T23:40:13Z,2021-11-18T01:31:56Z,,OWNER,,"Spotted in https://github.com/simonw/datasette-graphql/issues/79
Demo: https://datasette-graphql-demo.datasette.io/fixtures/bad_foreign_key
The foreign key links to a 404 page.
![B87009C7-CFCA-4DF9-8FBA-FA3E6CA28EC2](https://user-images.githubusercontent.com/9599/142334788-4d1a4acd-bc87-4426-b333-d46b221afcec.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1515/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1041158024,PR_kwDOBm6k_c4t7RKr,1500,Bump black from 21.9b0 to 21.10b0,49699333,dependabot[bot],closed,0,,,,,2,2021-11-01T13:11:23Z,2021-11-17T13:14:00Z,2021-11-17T13:13:58Z,CONTRIBUTOR,simonw/datasette/pulls/1500,"Bumps [black](https://github.com/psf/black) from 21.9b0 to 21.10b0.
Release notes
Sourced from black's releases .
21.10b0
Black
Document stability policy, that will apply for non-beta releases (#2529 )
Add new --workers
parameter (#2514 )
Fixed feature detection for positional-only arguments in lambdas (#2532 )
Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (#2519 )
Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
even though it has been removed (#2580 )
Deprecate Python 2 formatting support (#2523 )
Blackd
Remove dependency on aiohttp-cors (#2500 )
Bump required aiohttp version to 3.7.4 (#2509 )
Black-Primer
Add primer support for --projects (#2555 )
Print primer summary after individual failures (#2570 )
Integrations
Allow to pass target_version
in the vim plugin (#1319 )
Install build tools in docker file and use multi-stage build to keep the image size
down (#2582 )
Changelog
Sourced from black's changelog .
21.10b0
Black
Document stability policy, that will apply for non-beta releases (#2529 )
Add new --workers
parameter (#2514 )
Fixed feature detection for positional-only arguments in lambdas (#2532 )
Bumped typed-ast version minimum to 1.4.3 for 3.10 compatiblity (#2519 )
Fixed a Python 3.10 compatibility issue where the loop argument was still being passed
even though it has been removed (#2580 )
Deprecate Python 2 formatting support (#2523 )
Blackd
Remove dependency on aiohttp-cors (#2500 )
Bump required aiohttp version to 3.7.4 (#2509 )
Black-Primer
Add primer support for --projects (#2555 )
Print primer summary after individual failures (#2570 )
Integrations
Allow to pass target_version
in the vim plugin (#1319 )
Install build tools in docker file and use multi-stage build to keep the image size
down (#2582 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.9b0&new-version=21.10b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1500/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1055402144,PR_kwDOBm6k_c4unfnq,1512,New pattern for async view classes,9599,simonw,closed,0,,,,,7,2021-11-16T21:55:44Z,2021-11-17T01:39:54Z,2021-11-17T01:39:44Z,OWNER,simonw/datasette/pulls/1512,Refs #878 - starting out with the new `AsyncBase` class implementing a pytest-inspired `asyncio` parallel execution mechanism.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1512/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1,
1055469073,I_kwDOBm6k_c4-6S4R,1513,Research: CTEs and union all to calculate facets AND query at the same time,9599,simonw,closed,0,,,,,12,2021-11-16T22:26:45Z,2021-11-16T23:41:46Z,2021-11-16T23:41:46Z,OWNER,,"Consider this page: https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_search=plant&_facet=owner&_facet=country_long&_facet=primary_fuel
Datasette needs to run the main query for the rows on that page, a count query for the total query, then a separate query for each of those three specified facets.
This is a `_search=` query, so it needs to execute the FTS code once for the rows, again for the count, and then three more times for each of the facets.
Could running that query as a CTE and doing the other queries as part of the same large query produce significant speed improvements?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1513/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
812704869,MDU6SXNzdWU4MTI3MDQ4Njk=,1237,?_pretty=1 option for pretty-printing JSON output,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2021-02-20T20:54:40Z,2021-11-16T18:28:33Z,,OWNER,,Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1237/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
718540751,MDU6SXNzdWU3MTg1NDA3NTE=,1012,For 1.0 update trove classifier in setup.py,9599,simonw,open,0,,,3268330,Datasette 1.0,5,2020-10-10T05:52:08Z,2021-11-16T13:18:36Z,,OWNER,, Development Status :: 5 - Production/Stable,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1012/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1052247023,I_kwDOBm6k_c4-uAPv,1505,Datasette should have an option to output CSV with semicolons,9599,simonw,open,0,,,,,1,2021-11-12T18:02:21Z,2021-11-16T11:40:52Z,,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1505/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
440222719,MDU6SXNzdWU0NDAyMjI3MTk=,448,_facet_array should work against views,9599,simonw,closed,0,,,3268330,Datasette 1.0,12,2019-05-03T21:08:04Z,2021-11-16T01:32:05Z,2021-11-16T01:19:40Z,OWNER,,"I created this view: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads-8dbda00/ads_with_targets
```
CREATE VIEW ads_with_targets as select ads.*, json_group_array(targets.name) as target_names from ads
join ad_targets on ad_targets.ad_id = ads.id
join targets on ad_targets.target_id = targets.id
group by ad_targets.ad_id
```
When I try to apply faceting by array it appears to work at first: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names
But actually it's doing the wrong thing - the SQL for the facets uses rowid, but rowid is not present on views at all! These results are incorrect, and clicking to select a facet will fail to produce any rows: https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads/ads_with_targets?_facet_array=target_names&target_names__arraycontains=people_who_match%3Ainterests%3AAfrican-American+Civil+Rights+Movement+%281954%E2%80%9468%29
Here's the SQL it should be using when you select a facet (note that it does not use a rowid):
https://json-view-facet-bug-demo-j7hipcg4aq-uc.a.run.app/russian-ads?sql=select+*+from+ads_with_targets+where+id+in+%28%0D%0A++++++++++++select+ads_with_targets.id+from+ads_with_targets%2C+json_each%28ads_with_targets.target_names%29+j%0D%0A++++++++++++where+j.value+%3D+%3Ap0%0D%0A++++++++%29+limit+101&p0=people_who_match%3Ainterests%3ABlack+%28Color%29
So we need to do something a lot smarter here. I'm not sure what the fix will look like, or even if it's feasible given that views don't have a rowid to hook into so the JSON faceting SQL may have to be completely rewritten.
```
datasette publish cloudrun \
russian-ads.db \
--name json-view-facet-bug-demo \
--branch master \
--extra-options ""--config sql_time_limit_ms:5000 --config facet_time_limit_ms:5000""
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/448/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1054246919,I_kwDOBm6k_c4-1ogH,1511,Review plugin hooks for Datasette 1.0,9599,simonw,open,0,,,3268330,Datasette 1.0,1,2021-11-15T23:26:05Z,2021-11-16T01:20:14Z,,OWNER,,I need to perform a detailed review of the plugin interface - especially the plugin hooks like [register_facet_classes()](https://docs.datasette.io/en/stable/plugin_hooks.html#register-facet-classes) which I don't yet have complete confidence in.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1511/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
459590021,MDU6SXNzdWU0NTk1OTAwMjE=,519,Decide what goes into Datasette 1.0,9599,simonw,closed,0,,,3268330,Datasette 1.0,4,2019-06-23T15:47:41Z,2021-11-15T23:26:11Z,2021-11-15T23:26:11Z,OWNER,,Datasette ASGI #272 is a big part of it... but 1.0 will generally be an indicator that Datasette is a stable platform for developers to write plugins and custom templates against. So lots to think about.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/519/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1052851176,I_kwDOBm6k_c4-wTvo,1507,ReadTheDocs build failed for 0.59.2 release,9599,simonw,closed,0,,,,,6,2021-11-14T05:24:34Z,2021-11-14T05:41:55Z,2021-11-14T05:41:55Z,OWNER,,"I had to cancel the 0.59.2 release because ReadTheDocs was failing to build the documentation.
https://readthedocs.org/projects/datasette/builds/15268454/
```
/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m sphinx -T -b html -d _build/doctrees -D language=en . _build/html
Running Sphinx v1.8.5
loading translations [en]... done
making output directory...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 27 source files that are out of date
updating environment: 27 added, 0 changed, 0 removed
reading sources... [ 3%] authentication
Traceback (most recent call last):
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/cmd/build.py"", line 304, in build_main
app.build(args.force_all, filenames)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/application.py"", line 341, in build
self.builder.build_update()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 347, in build_update
len(to_build))
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 360, in build
updated_docnames = set(self.read())
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 468, in read
self._read_serial(docnames)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 490, in _read_serial
self.read_doc(docname)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/builders/__init__.py"", line 534, in read_doc
doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/io.py"", line 318, in read_doc
pub.publish()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py"", line 219, in publish
self.apply_transforms()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/core.py"", line 200, in apply_transforms
self.document.transformer.apply_transforms()
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py"", line 90, in apply_transforms
Transformer.apply_transforms(self)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/docutils/transforms/__init__.py"", line 171, in apply_transforms
transform.apply(**kwargs)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/transforms/__init__.py"", line 245, in apply
apply_source_workaround(n)
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py"", line 94, in apply_source_workaround
for classifier in reversed(node.parent.traverse(nodes.classifier)):
TypeError: argument to reversed() must be a sequence
Exception occurred:
File ""/home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/lib/python2.7/site-packages/sphinx/util/nodes.py"", line 94, in apply_source_workaround
for classifier in reversed(node.parent.traverse(nodes.classifier)):
TypeError: argument to reversed() must be a sequence
The full traceback has been saved in /tmp/sphinx-err-vkl0oE.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at . Thanks!
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1507/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1050163432,I_kwDOBm6k_c4-mDjo,1503,`?_nocol=` removes that column from the filter interface,9599,simonw,closed,0,,,,,1,2021-11-10T18:22:50Z,2021-11-14T05:08:27Z,2021-11-14T04:53:07Z,OWNER,,"e.g. on https://latest.datasette.io/fixtures/sortable?_nocol=sortable
This causes weird behaviour when you e.g. facet by a hidden column, since selecting facets and then re-submitting the form will clear the selected filter.
![nocol-bug](https://user-images.githubusercontent.com/9599/141171135-aded71d1-a4cb-4b7f-a4ea-26828fa98906.gif)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1503/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1052826038,I_kwDOBm6k_c4-wNm2,1506,Columns beginning with an underscore do not facet correctly,9599,simonw,closed,0,,,,,1,2021-11-14T02:20:32Z,2021-11-14T04:45:21Z,2021-11-14T04:45:21Z,OWNER,,"Datasette treats columns that start with an underscore as querystring parameters it should ignore!
Discovered in https://github.com/simonw/git-history/issues/14#issuecomment-968192464",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1506/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
924748955,MDU6SXNzdWU5MjQ3NDg5NTU=,1380,Serve all db files in a folder,193463,stratosgear,open,0,,,,,5,2021-06-18T10:03:32Z,2021-11-13T08:09:11Z,,NONE,,"I tried to get the `serve` command to serve all the .db files in the `/mnt` folder but is seems that the server does not refresh the list of files.
In more detail:
* Starting datasette as a docker container with:
```
docker run -p 8001:8001 -v `pwd`:/mnt \
datasetteproject/datasette \
datasette -p 8001 -h 0.0.0.0 /mnt
```
* Datasette correctly serves all the *.db files found in the /mnt folder
* When the server is running, if I copy a new file in the $PWD folder, Datasette does not seem to see the new files, forcing me to restart Docker.
Is there an option/setting that I overlooked, or is this something missing?
BTW, the `--reload` setting, although at first glance is what you think you need, does not seem to do anything in regards of seeing all *.db files.
Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1380/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1051277222,I_kwDOBm6k_c4-qTem,1504,Link to ?_size=max at bottom of table page,9599,simonw,open,0,,,,,0,2021-11-11T19:06:33Z,2021-11-11T19:06:33Z,,OWNER,,"This can have text such as ""Show 1,000 rows per page"", based on the max size limit setting. Would make it easier for people to see more data at once without having to know how to hack the URL, similar to the `...` for facet sizes I added in #1337.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1504/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1049946823,I_kwDOBm6k_c4-lOrH,1502,"Full-text search: No support to unary ""-"" operator",516827,gustavorps,open,0,,,,,0,2021-11-10T15:11:19Z,2021-11-10T15:11:19Z,,NONE,,"Reference: https://www.sqlite.org/fts3.html#set_operations_using_the_standard_query_syntax
Test: https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort+-freedman&_sort=rowid",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1502/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
845794436,MDU6SXNzdWU4NDU3OTQ0MzY=,1284,Feature or Documentation Request: Individual table as home page template,192568,mroswell,open,0,,,,,4,2021-03-31T03:56:17Z,2021-11-04T03:15:01Z,,CONTRIBUTOR,,"It would be great to have a sample showing how to move a single database that has a single table, to the index page. I'm trying it now, and find there is a real depth of Datasette and Python understanding that's required to be successful.
I've got all the basic jinja concepts down... variables, template control structures, template inheritance, template overrides, css, html, the --template-dir and --static arguments, etc.
But copying the table.html file to index.html doesn't work. There are undocumented functions and filters... I can figure some of them out (yay, url_builder.py and utils/__init__.py!) but it's a slog better handled by a much stronger Python developer.
One sample would make a world of difference. The ideal form of this documentation would be a diff between the default table.html and how that would look if essentially moved to index.html. The use case is for everyone who wants to create a public-facing website to explore a single table at the root directory. (Maybe a second bit of documentation for people who have a single database with multiple tables.)
(Hmm... might be cool to have a setting for that, where it happens automagically! If only one table, then home page is at the table level. if only one database, then home page is at the database level.... as an option.)
I suppose I could ignore this, and somehow do this in the DNS settings once I hook up Vercel to a domain name, maybe.. and remove the breadcrumbs in table.html... but for now, a documentation request in the form of a diff... for viewing a single table (or a single database) at the root.
(Actually, there's probably room for a whole expanded section on templates. Noticed some nice table metadata in one of the datasette examples, for instance... Hmm... maybe a whole library of solutions in one place... maybe a documentation hackathon! If that's of interest, of course it's a separate issue. )
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1284/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
826064552,MDU6SXNzdWU4MjYwNjQ1NTI=,1253,"Capture ""Ctrl + Enter"" or ""⌘ + Enter"" to send SQL query?",9308268,rayvoelker,open,0,,,,,1,2021-03-09T15:00:50Z,2021-10-30T16:00:42Z,,NONE,,"It appears as though ""Shift + Enter"" triggers the form submit action to submit SQL, but could that action be bound to the ""Ctrl + Enter"" or ""⌘ + Enter"" action?
I feel like that pattern already exists in a number of similar tools and could improve usability of the editor.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1253/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1029100823,PR_kwDOBm6k_c4tU5cz,1494,"Update pytest-asyncio requirement from <0.16,>=0.10 to >=0.10,<0.17",49699333,dependabot[bot],closed,0,,,,,1,2021-10-18T13:14:17Z,2021-10-24T22:22:40Z,2021-10-24T22:22:39Z,CONTRIBUTOR,simonw/datasette/pulls/1494,"Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
Commits
f2fe98e
0.16.0
4e1df31
Remove obsolete test, add make test
6ec7647
feat: Add support for Python 3.10.
42ff5d1
ci: Include Python 3.10 in the CI test run.
be3b327
build: Include Python 3.10 in Tox test runs.
1c283bd
refactor: test_async_fixtures_with_finalizer no longer trigger a DeprecationW...
2751982
refactor: Replaced tests asserting that the event loop is properly closed.
70989fd
refactor: Grouped test cases together that are related to the use of the asyn...
b27abe8
refactor: Removed TestUnexistingLoop.remove_loop fixture, because it has no e...
e3ec312
Adjusted Hypothesis integration test to use the same event loop initializatio...
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1494/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1021550542,I_kwDOBm6k_c4845_O,1482,Support Python 3.10,9599,simonw,closed,0,,,,,2,2021-10-09T00:30:52Z,2021-10-24T22:21:40Z,2021-10-24T22:19:55Z,OWNER,,"I started work on this in #1481 where I found a Python 3.10 bug that needs a workaround in Janus, see:
- https://github.com/aio-libs/janus/issues/358
This is a tracking issue for anything else that shows up.
This is also needed for the Homebrew package to upgrade to 3.10:
- https://github.com/Homebrew/homebrew-core/pull/86932",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1482/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1020436713,PR_kwDOBm6k_c4s6bJm,1481,Fix compatibility with Python 3.10,9599,simonw,closed,0,,,,,13,2021-10-07T20:34:23Z,2021-10-24T22:19:55Z,2021-10-24T22:19:54Z,OWNER,simonw/datasette/pulls/1481,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1481/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
950664971,MDU6SXNzdWU5NTA2NjQ5NzE=,1401,unordered list is not rendering bullet points in description_html on database page,536941,fgregg,open,0,,,,,2,2021-07-22T13:24:18Z,2021-10-23T13:09:10Z,,CONTRIBUTOR,,"Thanks for this tremendous package, @simonw!
In the `description_html` for a database, I [have an unordered list](https://github.com/labordata/warehouse/blob/fcea4502e5b615b0eb3e0bdcb45ec634abe20bb6/warehouse_metadata.yml#L19-L22).
However, on the database page on the deployed site, it is not rendering this as a bulleted list.
![Screenshot 2021-07-22 at 09-21-51 nlrb](https://user-images.githubusercontent.com/536941/126645923-2777b7f1-fd4c-4d2d-af70-a35e49a07675.png)
Page here: https://labordata-warehouse.herokuapp.com/nlrb-9da4ae5
The documentation gives an [example of using an unordered list](https://docs.datasette.io/en/stable/metadata.html#using-yaml-for-metadata) in a `description_html`, so I expected this will work.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1401/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1033864602,I_kwDOBm6k_c49n4Wa,1496,Named parameters docs should include an example of a cast,9599,simonw,closed,0,,,,,1,2021-10-22T18:56:04Z,2021-10-22T19:38:23Z,2021-10-22T19:34:27Z,OWNER,,"https://docs.datasette.io/en/stable/sql_queries.html#named-parameters
It's not obvious that the values from parameters are always SQLite strings, which means that you can't do e.g. integer comparisons on them without casting them first. The documentation here should include an example of this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1496/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
944903881,MDU6SXNzdWU5NDQ5MDM4ODE=,1396,"""invalid reference format"" publishing Docker image",9599,simonw,closed,0,,,,,9,2021-07-15T01:02:07Z,2021-10-19T08:10:26Z,2021-07-15T19:47:25Z,OWNER,,"Error ocurred at the end of the publish flow for Datasette 0.58: https://github.com/simonw/datasette/runs/3072216421
```
Removing intermediate container cf32b9440907
---> dfd6985b2afc
Successfully built dfd6985b2afc
Successfully tagged ***/datasette:0.58
invalid reference format
Error: Process completed with exit code 1.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1396/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1028115674,I_kwDOBm6k_c49R8za,1493,`--get '/:memory:.json?sql=select+3*5'` error with datasette 0.59,1580956,chenrui333,closed,0,,,,,1,2021-10-16T18:22:22Z,2021-10-19T04:39:11Z,2021-10-19T04:39:11Z,NONE,,"👋 trying to upgrade the formula to use the latest release, but runs into some regression test issue with `--get` command.
My QQ is does this `datasette --get '/:memory:.json?sql=select+3*5'` supposed to return 15? Thanks!
relates to https://github.com/Homebrew/homebrew-core/pull/87369",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1493/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
969855774,MDU6SXNzdWU5Njk4NTU3NzQ=,1432,Rename Datasette.__init__(config=) parameter to settings=,9599,simonw,open,0,,,,,8,2021-08-13T01:00:27Z,2021-10-19T01:16:41Z,,OWNER,,"> While I'm doing this I should rename this internal variable to avoid confusion in the future:
>
> https://github.com/simonw/datasette/blob/e837095ef35ae155b4c78cc9a8b7133a48c94f03/datasette/app.py#L203
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1431#issuecomment-898072940_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1432/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
995098231,MDU6SXNzdWU5OTUwOTgyMzE=,1470,?_sort=rowid with _next= returns error,19851673,eigenfoo,closed,0,,,,,4,2021-09-13T16:36:15Z,2021-10-18T19:30:15Z,2021-10-10T01:15:03Z,NONE,,"For example:
- Go to https://cryptics.eigenfoo.xyz/clues/clues?_next=100 (this is the second page of results in a Datasette site)
- Search anything using the FTS search bar. For example, searching for `hello` will take you to https://cryptics.eigenfoo.xyz/clues/clues?_search=hello&_sort=rowid&_next=100
- A `500 Error: list index out of range` is raised.
This is because the search URL includes the `&_next=100` UTM parameter, carried over from where the FTS search was run. However, there isn't a second page in the search results, so a `list index out of range` error is raised. You can confirm that removing this UTM parameter from the URL returns the appropriate search results.
The FTS search request should strip any `_next` UTM parameter.
---
```bash
datasette, version 0.58.1
sqlite-utils, version 3.17
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1470/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
991575770,MDExOlB1bGxSZXF1ZXN0NzMwMDIwODY3,1467,Add Authorization header when CORS flag is set,3058200,jameslittle230,closed,0,,,,,3,2021-09-08T22:14:41Z,2021-10-17T02:29:07Z,2021-10-14T18:54:18Z,NONE,simonw/datasette/pulls/1467,"This PR adds the [`Access-Control-Allow-Headers`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers) flag when CORS mode is enabled.
This would fix https://github.com/simonw/datasette-auth-tokens/issues/4. When making cross-origin requests, the server must respond with all allowable HTTP headers. A Datasette instance using auth tokens must accept the `Authorization` HTTP header in order for cross-origin authenticated requests to take place.
Please let me know if there's a better way of doing this! I couldn't figure out a way to change the app's response from the plugin itself, so I'm starting here. If you'd rather this logic live in the plugin, I'd love any guidance you're able to give.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1467/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
988555009,MDExOlB1bGxSZXF1ZXN0NzI3NDM2NDA0,1458,Rework the `--static` documentation a bit,51016,ctb,closed,0,,,,,1,2021-09-05T17:08:48Z,2021-10-15T13:24:29Z,2021-10-14T18:39:55Z,CONTRIBUTOR,simonw/datasette/pulls/1458,"Per https://github.com/simonw/datasette/issues/1457, I was confused by the current docs and took a few minutes to sort out what the right combination of locations was.
This is a PR that differentiates the docs to split out `/static/` in URL from `--static` option and `./static/` path.
Not wedded to the details in any way, happy to change to suit.
Fixes #1457.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1458/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
988553806,MDU6SXNzdWU5ODg1NTM4MDY=,1457,suggestion: distinguish names in `--static` documentation,51016,ctb,closed,0,,,,,0,2021-09-05T17:04:27Z,2021-10-14T18:39:55Z,2021-10-14T18:39:55Z,CONTRIBUTOR,,"Over in https://docs.datasette.io/en/stable/custom_templates.html#serving-static-files, there is the slightly comical example command -
```
datasette -m metadata.json --static static:static/ --memory
```
(now, with MORE STATIC!)
It took me a while to sort out all the URLs and paths involved because I wasn't being very clever. But in the interests of simplification and distinction, I might suggest something like
```
datasette -m metadata.json --static loc:static-files/ --memory
```
I will submit a PR for your consideration.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1457/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1026379132,PR_kwDOBm6k_c4tM0JV,1489,"Update pyyaml requirement from ~=5.3 to >=5.3,<7.0",49699333,dependabot[bot],closed,0,,,,,3,2021-10-14T13:09:33Z,2021-10-14T18:10:43Z,2021-10-14T18:10:42Z,CONTRIBUTOR,simonw/datasette/pulls/1489,"Updates the requirements on [pyyaml](https://github.com/yaml/pyyaml) to permit the latest version.
Changelog
Sourced from pyyaml's changelog .
6.0 (2021-10-13)
5.4.1 (2021-01-20)
yaml/pyyaml#480 -- Fix stub compat with older pyyaml versions that may unwittingly load it
5.4 (2021-01-19)
5.3.1 (2020-03-18)
yaml/pyyaml#386 -- Prevents arbitrary code execution during python/object/new constructor
5.3 (2020-01-06)
5.2 (2019-12-02)
... (truncated)
Commits
8cdff2c
6.0 release
a4fb55e
Update Python 3.10 versions for Windows build
e45b964
Add Python 3.10 to tox.ini
4808fba
6.0b1 release
d5aba40
Omnibus CI/artifact build update
a6d384c
Various setup fixes
8f3f979
No longer using appveyor
c274365
The yaml.load{,_all} functions require Loader= now
2f87ac4
Add a basic test file for yaml.load and yaml.dump
7bd92df
Makefile tweaks
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1489/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1025754125,I_kwDOBm6k_c49I8QN,1488,Upgrade to httpx 0.20.0 (request() got an unexpected keyword argument 'allow_redirects'),9599,simonw,closed,0,,,,,5,2021-10-13T22:37:22Z,2021-10-14T18:03:45Z,2021-10-14T18:03:45Z,OWNER,,This is caused by a change made to `httpx` in https://github.com/encode/httpx/releases/tag/0.20.0,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1488/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1026664511,PR_kwDOBm6k_c4tNtoe,1490,Upgrade to httpx 0.20,9599,simonw,closed,0,,,,,0,2021-10-14T17:51:05Z,2021-10-14T18:03:45Z,2021-10-14T18:03:44Z,OWNER,simonw/datasette/pulls/1490,Refs #1488 ,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1490/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
991121619,MDExOlB1bGxSZXF1ZXN0NzI5NjMyNjQz,1463,"Update beautifulsoup4 requirement from <4.10.0,>=4.8.1 to >=4.8.1,<4.11.0",49699333,dependabot[bot],closed,0,,,,,1,2021-09-08T13:09:38Z,2021-10-13T22:35:37Z,2021-10-13T22:35:36Z,CONTRIBUTOR,simonw/datasette/pulls/1463,"Updates the requirements on [beautifulsoup4](http://www.crummy.com/software/BeautifulSoup/bs4/) to permit the latest version.
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1463/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
996002181,PR_kwDOBm6k_c4rutdp,1471,Bump black from 21.7b0 to 21.9b0,49699333,dependabot[bot],closed,0,,,,,1,2021-09-14T13:10:35Z,2021-10-13T21:47:42Z,2021-10-13T21:47:42Z,CONTRIBUTOR,simonw/datasette/pulls/1471,"Bumps [black](https://github.com/psf/black) from 21.7b0 to 21.9b0.
Release notes
Sourced from black's releases .
21.9b0
Packaging
Fix missing modules in self-contained binaries (#2466 )
Fix missing toml extra used during installation (#2475 )
21.8b0
Black
Add support for formatting Jupyter Notebook files (#2357 )
Move from appdirs
dependency to platformdirs
(#2375 )
Present a more user-friendly error if .gitignore is invalid (#2414 )
The failsafe for accidentally added backslashes in f-string expressions has been hardened to handle more edge cases during quote normalization (#2437 )
Avoid changing a function return type annotation's type to a tuple by adding a trailing comma (#2384 )
Parsing support has been added for unparenthesized walruses in set literals, set comprehensions, and indices (#2447 ).
Pin setuptools-scm
build-time dependency version (#2457 )
Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10 (#2460 )
Blackd
Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that scan installed packages (#2440 )
Integrations
The provided pre-commit hooks no longer specify language_version
to avoid overriding default_language_version
(#2430 )
Changelog
Sourced from black's changelog .
21.9b0
Packaging
Fix missing modules in self-contained binaries (#2466 )
Fix missing toml extra used during installation (#2475 )
21.8b0
Black
Add support for formatting Jupyter Notebook files (#2357 )
Move from appdirs
dependency to platformdirs
(#2375 )
Present a more user-friendly error if .gitignore is invalid (#2414 )
The failsafe for accidentally added backslashes in f-string expressions has been
hardened to handle more edge cases during quote normalization (#2437 )
Avoid changing a function return type annotation's type to a tuple by adding a
trailing comma (#2384 )
Parsing support has been added for unparenthesized walruses in set literals, set
comprehensions, and indices (#2447 ).
Pin setuptools-scm
build-time dependency version (#2457 )
Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10
(#2460 )
Blackd
Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that
scan installed packages (#2440 )
Integrations
The provided pre-commit hooks no longer specify language_version
to avoid overriding
default_language_version
(#2430 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.7b0&new-version=21.9b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1471/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
994450961,MDU6SXNzdWU5OTQ0NTA5NjE=,1469,"Column cog shows ""facet by this"" when already default faceted",9599,simonw,closed,0,,,,,2,2021-09-13T04:51:26Z,2021-10-13T21:20:07Z,2021-10-13T21:20:07Z,OWNER,,"e.g. on https://covid-19.datasettes.com/covid/economist_excess_deaths
But if you add `?_facet=country` to the URL that goes away: https://covid-19.datasettes.com/covid/economist_excess_deaths?_facet_size=5&_facet=country
The logic that decides if the ""Facet by this"" item is shown does not take default `metadata.json` facets into account.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1469/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
980228553,MDExOlB1bGxSZXF1ZXN0NzIwNTA2MTM1,1448,"Update pluggy requirement from ~=0.13.0 to >=0.13,<1.1",49699333,dependabot[bot],closed,0,,,,,1,2021-08-26T13:09:52Z,2021-10-13T21:11:01Z,2021-10-13T21:11:00Z,CONTRIBUTOR,simonw/datasette/pulls/1448,"Updates the requirements on [pluggy](https://github.com/pytest-dev/pluggy) to permit the latest version.
Changelog
Sourced from pluggy's changelog .
pluggy 1.0.0 (2021-08-25)
Deprecations and Removals
[#116](https://github.com/pytest-dev/pluggy/issues/116) <https://github.com/pytest-dev/pluggy/issues/116>
_: Remove deprecated implprefix
support.
Decorate hook implementations using an instance of HookimplMarker instead.
The deprecation was announced in release 0.7.0
.
[#120](https://github.com/pytest-dev/pluggy/issues/120) <https://github.com/pytest-dev/pluggy/issues/120>
_: Remove the deprecated proc
argument to call_historic
.
Use result_callback
instead, which has the same behavior.
The deprecation was announced in release 0.7.0
.
[#265](https://github.com/pytest-dev/pluggy/issues/265) <https://github.com/pytest-dev/pluggy/issues/265>
_: Remove the _Result.result
property. Use _Result.get_result()
instead.
Note that unlike result
, get_result()
raises the exception if the hook raised.
The deprecation was announced in release 0.6.0
.
[#267](https://github.com/pytest-dev/pluggy/issues/267) <https://github.com/pytest-dev/pluggy/issues/267>
_: Remove official support for Python 3.4.
[#272](https://github.com/pytest-dev/pluggy/issues/272) <https://github.com/pytest-dev/pluggy/issues/272>
_: Dropped support for Python 2.
Continue to use pluggy 0.13.x for Python 2 support.
[#308](https://github.com/pytest-dev/pluggy/issues/308) <https://github.com/pytest-dev/pluggy/issues/308>
_: Remove official support for Python 3.5.
[#313](https://github.com/pytest-dev/pluggy/issues/313) <https://github.com/pytest-dev/pluggy/issues/313>
_: The internal pluggy.callers
, pluggy.manager
and pluggy.hooks
are now explicitly marked private by a _
prefix (e.g. pluggy._callers
).
Only API exported by the top-level pluggy
module is considered public.
[#59](https://github.com/pytest-dev/pluggy/issues/59) <https://github.com/pytest-dev/pluggy/issues/59>
_: Remove legacy __multicall__
recursive hook calling system.
The deprecation was announced in release 0.5.0
.
Features
[#282](https://github.com/pytest-dev/pluggy/issues/282) <https://github.com/pytest-dev/pluggy/issues/282>
_: When registering a hookimpl which is declared as hookwrapper=True
but whose
function is not a generator function, a PluggyValidationError
exception is
now raised.
Previously this problem would cause an error only later, when calling the hook.
In the unlikely case that you have a hookwrapper that returns a generator
... (truncated)
Commits
4259fdd
Fix CHANGELOG title manually
906abca
Preparing release 1.0.0
56eb23c
Rename HOWTORELEASE to RELEASING to follow pytest
fc6395c
Fix scripts/release.py to use main instead of master
e04816f
Merge pull request #324 from RonnyPfannschmidt/benchmarks
1424ab0
add micro benchmarks for hook calling playing with a the number of callers, w...
5e51864
Merge pull request #323 from RonnyPfannschmidt/switch-to-main
05c3bbd
switch to main as primary branch
6b344fb
Merge pull request #319 from RonnyPfannschmidt/pre-commit-update
71f2d6b
introduce pyupgrade and update black
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1448/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1000779422,PR_kwDOBm6k_c4r9CTw,1474,Update full_text_search.rst,72577720,MichaelTiemannOSC,closed,0,,,,,0,2021-09-20T09:59:45Z,2021-10-13T21:10:23Z,2021-10-13T21:10:23Z,CONTRIBUTOR,simonw/datasette/pulls/1474,"Change ""above"" to ""below"" to correct correspondence of reference to example.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1474/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1002459220,PR_kwDOBm6k_c4sCfBT,1476,"Update pytest-xdist requirement from <2.4,>=2.2.1 to >=2.2.1,<2.5",49699333,dependabot[bot],closed,0,,,,,1,2021-09-21T13:13:01Z,2021-10-13T21:10:03Z,2021-10-13T21:10:03Z,CONTRIBUTOR,simonw/datasette/pulls/1476,"Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version.
Changelog
Sourced from pytest-xdist's changelog .
pytest-xdist 2.4.0 (2021-09-20)
Features
[#696](https://github.com/pytest-dev/pytest-xdist/issues/696) <https://github.com/pytest-dev/pytest-xdist/issues/696>
_: On Linux, the process title now changes to indicate the current worker state (running/idle).
Depends on the setproctitle <https://pypi.org/project/setproctitle/>
__ package, which can be installed with pip install pytest-xdist[setproctitle]
.
[#704](https://github.com/pytest-dev/pytest-xdist/issues/704) <https://github.com/pytest-dev/pytest-xdist/issues/704>
_: Add support for Python 3.10.
pytest-xdist 2.3.0 (2021-06-16)
Deprecations and Removals
[#654](https://github.com/pytest-dev/pytest-xdist/issues/654) <https://github.com/pytest-dev/pytest-xdist/issues/654>
_: Python 3.5 is no longer supported.
Features
[#646](https://github.com/pytest-dev/pytest-xdist/issues/646) <https://github.com/pytest-dev/pytest-xdist/issues/646>
_: Add --numprocesses=logical
flag, which automatically uses the number of logical CPUs available, instead of physical CPUs with auto
.
This is very useful for test suites which are not CPU-bound.
[#650](https://github.com/pytest-dev/pytest-xdist/issues/650) <https://github.com/pytest-dev/pytest-xdist/issues/650>
_: Added new pytest_handlecrashitem
hook to allow handling and rescheduling crashed items.
Bug Fixes
[#421](https://github.com/pytest-dev/pytest-xdist/issues/421) <https://github.com/pytest-dev/pytest-xdist/issues/421>
_: Copy the parent process sys.path into local workers, to work around execnet's python -c adding the current directory to sys.path.
[#638](https://github.com/pytest-dev/pytest-xdist/issues/638) <https://github.com/pytest-dev/pytest-xdist/issues/638>
_: Fix issue caused by changing the branch name of the pytest repository.
Trivial Changes
[#592](https://github.com/pytest-dev/pytest-xdist/issues/592) <https://github.com/pytest-dev/pytest-xdist/issues/592>
_: Replace master with controller where ever possible.
[#643](https://github.com/pytest-dev/pytest-xdist/issues/643) <https://github.com/pytest-dev/pytest-xdist/issues/643>
_: Use 'main' to refer to pytest default branch in tox env names.
pytest-xdist 2.2.1 (2021-02-09)
... (truncated)
Commits
4b487ed
Manually fix changelog title
ecf4d3b
Release 2.4.0
87d8979
Merge pull request #704 from hugovk/add-3.10
b4544c8
Merge pull request #706 from pytest-dev/pre-commit-ci-update-config
66dc390
[pre-commit.ci] pre-commit autoupdate
e0ce1b7
Add news file to add support for Python 3.10
ed47f0e
Add support for Python 3.10
1c8178a
Merge pull request #703 from pytest-dev/pre-commit-ci-update-config
9807064
[pre-commit.ci] pre-commit autoupdate
766e67c
Use setproctitle if available to show state (#696 )
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1476/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1022688960,PR_kwDOBm6k_c4tBIEE,1485,"Update pytest-timeout requirement from <1.5,>=1.4.2 to >=1.4.2,<2.1",49699333,dependabot[bot],closed,0,,,,,1,2021-10-11T13:10:51Z,2021-10-13T21:09:23Z,2021-10-13T21:09:23Z,CONTRIBUTOR,simonw/datasette/pulls/1485,"Updates the requirements on [pytest-timeout](https://github.com/pytest-dev/pytest-timeout) to permit the latest version.
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1485/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1023243105,I_kwDOBm6k_c48_XNh,1486,pipx installation instructions for plugins don't reference pipx inject,41546558,RhetTbull,closed,0,,,,,0,2021-10-12T00:43:42Z,2021-10-13T21:09:11Z,2021-10-13T21:09:11Z,CONTRIBUTOR,,"The datasette [installation instructions](https://github.com/simonw/datasette/blob/main/docs/installation.rst) discuss how to install with pipx, how to upgrade with pipx, and how to upgrade plugins with pipx but do not mention how to install a plugin with pipx. You discussed this on your [blog](https://til.simonwillison.net/python/installing-upgrading-plugins-with-pipx) but looks like this didn't make it in when you updated the docs for pipx (#756).
I'll submit a PR shortly to fix this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1486/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1023245060,PR_kwDOBm6k_c4tC4Lx,1487,"Added instructions for installing plugins via pipx, #1486",41546558,RhetTbull,closed,0,,,,,1,2021-10-12T00:48:30Z,2021-10-13T21:09:11Z,2021-10-13T21:09:10Z,CONTRIBUTOR,simonw/datasette/pulls/1487,Adds missing instructions for installing plugins via pipx,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1487/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1021849766,I_kwDOBm6k_c486DCm,1483,Running a search on page 2 of results should not preserve ?_next=,9599,simonw,closed,0,,,,,0,2021-10-10T01:18:12Z,2021-10-13T21:08:10Z,2021-10-13T21:08:10Z,OWNER,,Reported by @eigenfoo in https://github.com/simonw/datasette/issues/1470,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1483/reactions"", ""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1022294524,PR_kwDOBm6k_c4s_4Cw,1484,GitHub Actions: Add Python 3.10 to the tests,3709715,cclauss,closed,0,,,,,0,2021-10-11T06:03:03Z,2021-10-11T06:03:31Z,2021-10-11T06:03:28Z,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1484,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1484/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
1006781949,I_kwDOBm6k_c48AkX9,1478,Documentation Request: Feature alternative ID instead of default ID,192568,mroswell,open,0,,,,,0,2021-09-24T19:56:13Z,2021-09-25T16:18:54Z,,CONTRIBUTOR,,"My data already has an ID that comes from a federal agency.
Would love to have documentation on how to modify the template to:
- Remove the generated ID from the table
- Link the federal ID to the detail page
- and to ensure that the JSON file uses that as the ID. I'd be happy to include the database ID in the export, but not as a key.
I don't want to remove the ID from the database, though, because my experience with the federal agency is that data often has anomalies. I don't want all hell to break loose if they end up applying the same ID to multiple rows (which they haven't done yet). I just don't want it to display in the table or the data exports.
Perhaps this isn't a template issue, maybe more of a db manipulation...
Margie",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1478/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1006016302,I_kwDOBm6k_c479pcu,1477,Consider adding request to the documented default template context,9599,simonw,open,0,,,,,0,2021-09-24T02:34:09Z,2021-09-24T02:34:09Z,,OWNER,,I made a plugin for this today but I think perhaps it should be a default thing instead: https://datasette.io/plugins/datasette-template-request,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1477/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
274615452,MDU6SXNzdWUyNzQ2MTU0NTI=,111,Add “updated” to metadata,9599,simonw,open,0,,,,,12,2017-11-16T18:22:20Z,2021-09-21T22:48:27Z,,OWNER,,"To give an indication as to when the data was last updated.
This should be a field in the metadata that is then shown on the index page and in the footer, if it is set.
Also support setting it using an option to “datasette publish” and “datasette package” - which can either be a string or can be the magic string “today” to set it to today’s date:
datasette publish file.db --updated=today",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/111/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1001104942,PR_kwDOBm6k_c4r-EVH,1475,feat: allow joins using _through in both directions,5268174,bram2000,open,0,,,,,0,2021-09-20T15:28:20Z,2021-09-20T15:28:20Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1475,"Currently the `_through` clause can only work if the FK relationship is defined in a specific direction. I don't think there is any reason for this limitation, as an FK allows joining in both directions.
This is an admittedly hacky change to implement bidirectional joins using `_through`. It does work for our use-case, but I don't know if there are other implications that I haven't thought of. Also if this change is desirable we probably want to make the code a little nicer.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1475/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
999902754,I_kwDOBm6k_c47mU4i,1473,base logo link visits `undefined` rather than href url,192568,mroswell,open,0,,,,,2,2021-09-18T04:17:04Z,2021-09-19T00:45:32Z,,CONTRIBUTOR,,"I have two connected sites:
http://www.SaferOrToxic.org
(a Hugo website)
and:
http://disinfectants.SaferOrToxic.org/disinfectants/listN
(a datasette table page)
The latter is linked as ""The List"" in the former's menu.
(I'd love a prettier URL, but that's what I've got.)
On:
http://disinfectants.SaferOrToxic.org/disinfectants/listN
... all the other menu links should point back to:
https://www.SaferOrToxic.org
And they do!
But the logo, for some reason--though it has an href pointing to:
https://www.SaferOrToxic.org
Keeps going to this instead:
https://disinfectants.saferortoxic.org/disinfectants/undefined
What is causing that? How can I fix it?
In #1284 back in March, I was doing battle with the index.html template, in a still unresolved issue. (I wanted only a single table page at the root.)
But I thought, well, if I can't resolve that, at least I could just point the main website to the datasette page (""The List,"") and then have the List point back to the home website.
The menu hrefs to https://www.SaferOrToxic.org work just fine, exactly as they should, from the datasette page. Even the Home link works properly.
But the logo link keeps rewriting to: https://disinfectants.saferortoxic.org/disinfectants/undefined
This is the HTML:
```
```
Is this somehow related to cloudflare?
Or something in the datasette code?
I'm starting to think it's a cloudflare issue.
Can I at least rule out it being a datasette issue?
My repository is here:
https://github.com/mroswell/list-N
(BTW, I couldn't figure out how to reference a local image, either, on the datasette side, which is why I'm using the image from the www home page.)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1473/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
982780906,MDExOlB1bGxSZXF1ZXN0NzIyNDgwNTQy,1453,Bump black from 21.7b0 to 21.8b0,49699333,dependabot[bot],closed,0,,,,,2,2021-08-30T13:13:39Z,2021-09-14T13:10:40Z,2021-09-14T13:10:38Z,CONTRIBUTOR,simonw/datasette/pulls/1453,"Bumps [black](https://github.com/psf/black) from 21.7b0 to 21.8b0.
Release notes
Sourced from black's releases .
21.8b0
Black
Add support for formatting Jupyter Notebook files (#2357 )
Move from appdirs
dependency to platformdirs
(#2375 )
Present a more user-friendly error if .gitignore is invalid (#2414 )
The failsafe for accidentally added backslashes in f-string expressions has been hardened to handle more edge cases during quote normalization (#2437 )
Avoid changing a function return type annotation's type to a tuple by adding a trailing comma (#2384 )
Parsing support has been added for unparenthesized walruses in set literals, set comprehensions, and indices (#2447 ).
Pin setuptools-scm
build-time dependency version (#2457 )
Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10 (#2460 )
Blackd
Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that scan installed packages (#2440 )
Integrations
The provided pre-commit hooks no longer specify language_version
to avoid overriding default_language_version
(#2430 )
Changelog
Sourced from black's changelog .
21.8b0
Black
Add support for formatting Jupyter Notebook files (#2357 )
Move from appdirs
dependency to platformdirs
(#2375 )
Present a more user-friendly error if .gitignore is invalid (#2414 )
The failsafe for accidentally added backslashes in f-string expressions has been
hardened to handle more edge cases during quote normalization (#2437 )
Avoid changing a function return type annotation's type to a tuple by adding a
trailing comma (#2384 )
Parsing support has been added for unparenthesized walruses in set literals, set
comprehensions, and indices (#2447 ).
Pin setuptools-scm
build-time dependency version (#2457 )
Exclude typing-extensions version 3.10.0.1 due to it being broken on Python 3.10
(#2460 )
Blackd
Replace sys.exit(-1) with raise ImportError as it plays more nicely with tools that
scan installed packages (#2440 )
Integrations
The provided pre-commit hooks no longer specify language_version
to avoid overriding
default_language_version
(#2430 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.7b0&new-version=21.8b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1453/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
991191951,MDU6SXNzdWU5OTExOTE5NTE=,1464,clean checkout & clean environment has test failures,51016,ctb,open,0,,,,,6,2021-09-08T14:16:23Z,2021-09-13T22:17:17Z,,CONTRIBUTOR,,"I followed the instructions [here](https://docs.datasette.io/en/stable/contributing.html#setting-up-a-development-environment), and even after running `python update-docs-help.py` I get the following failed tests -- any thoughts?
```
FAILED tests/test_api.py::test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]
FAILED tests/test_api.py::test_searchmode[table_metadata1-_search=te*+AND+do*-expected_rows1]
FAILED tests/test_api.py::test_searchmode[table_metadata2-_search=te*+AND+do*&_searchmode=raw-expected_rows2]
```
This is with python 3.9.7 and lots of other packages, as in attached environment listing from `conda list`.
[conda-installed.txt](https://github.com/simonw/datasette/files/7129487/conda-installed.txt)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1464/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
930855052,MDExOlB1bGxSZXF1ZXN0Njc4NDU5NTU0,1385,Fix + improve get_metadata plugin hook docs,2670795,brandonrobertz,closed,0,,,,,1,2021-06-27T05:43:20Z,2021-09-13T18:53:11Z,2021-09-13T18:53:11Z,CONTRIBUTOR,simonw/datasette/pulls/1385,"This fixes documentation inaccuracies and adds a disclaimer about the signature of the `get_metadata` hook.
Addresses the following comments:
- https://github.com/simonw/datasette/issues/1384#issuecomment-869069926
- https://github.com/simonw/datasette/issues/1384#issuecomment-869075368",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1385/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
994390593,MDU6SXNzdWU5OTQzOTA1OTM=,1468,Faceting for custom SQL queries,72577720,MichaelTiemannOSC,closed,0,,,,,2,2021-09-13T02:52:16Z,2021-09-13T04:54:22Z,2021-09-13T04:54:17Z,CONTRIBUTOR,,"Facets are awesome. But not when I need to join to tidy tables together. Or even just running explicitly the default SQL query that simply lists all the rows and columns of a table (up to SIZE). That is to say, when I browse a table, I see facets:
https://latest.datasette.io/fixtures/compound_three_primary_keys
But when I run a custom query, I don't:
https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101
Is there an idiom to cause custom SQL to come back with facet suggestions?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1468/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
991206402,MDExOlB1bGxSZXF1ZXN0NzI5NzA0NTM3,1465,add support for -o --get /path,51016,ctb,open,0,,,,,0,2021-09-08T14:30:42Z,2021-09-08T14:31:45Z,,CONTRIBUTOR,simonw/datasette/pulls/1465,"Fixes https://github.com/simonw/datasette/issues/1459
Adds support for `--open --get /path` to be used in combination.
If `--open` is provided alone, datasette will open a web page to a default URL.
If `--get ` is provided alone, datasette will output the result of doing a GET to that URL and then exit.
If `--open --get ` are provided together, datasette will open a web page to that URL.
TODO items:
- [ ] update documentation
- [ ] print out error message when `--root --open --get ` is used
- [ ] adjust code to require that `` start with a `/` when `-o --get ` is used
- [ ] add test(s)
note, '@CTB' is used in this PR to flag code that needs revisiting.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1465/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1,
990367646,MDU6SXNzdWU5OTAzNjc2NDY=,1462,"Separate out ""debug"" options from ""root"" options",9599,simonw,open,0,,,,,1,2021-09-07T21:27:34Z,2021-09-07T21:34:33Z,,OWNER,,"> I ditched ""root"" for ""admin"" because root by default gives you a whole bunch of stuff which I think could be confusing:
>
>
>
> Maybe the real problem here is that I'm conflating ""root"" permissions with ""debug"" options. Perhaps there should be an extra Datasette mode that unlocks debug tools for the root user?
_Originally posted by @simonw in https://github.com/simonw/datasette-app-support/issues/8#issuecomment-914638998_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1462/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
989986586,MDU6SXNzdWU5ODk5ODY1ODY=,1461,Try blacken-docs,9599,simonw,closed,0,,,,,3,2021-09-07T13:28:50Z,2021-09-07T16:13:59Z,2021-09-07T16:13:59Z,OWNER,,https://github.com/asottile/blacken-docs,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1461/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
989109888,MDU6SXNzdWU5ODkxMDk4ODg=,1460,Override column metadata with metadata from another column,72577720,MichaelTiemannOSC,open,0,,,,,0,2021-09-06T12:13:33Z,2021-09-06T12:13:33Z,,CONTRIBUTOR,,"I have a table from the PUDL project (https://github.com/catalyst-cooperative/pudl) that looks like this:
```
CREATE TABLE fuel_ferc1 (
id INTEGER NOT NULL,
record_id TEXT,
utility_id_ferc1 INTEGER,
report_year INTEGER,
plant_name_ferc1 TEXT,
fuel_type_code_pudl VARCHAR(7),
fuel_unit VARCHAR(7),
fuel_qty_burned FLOAT,
fuel_mmbtu_per_unit FLOAT,
fuel_cost_per_unit_burned FLOAT,
fuel_cost_per_unit_delivered FLOAT,
fuel_cost_per_mmbtu FLOAT,
PRIMARY KEY (id),
FOREIGN KEY(plant_name_ferc1, utility_id_ferc1) REFERENCES plants_ferc1 (plant_name_ferc1, utility_id_ferc1),
CONSTRAINT fuel_ferc1_fuel_type_code_pudl_enum CHECK (fuel_type_code_pudl IN ('coal', 'oil', 'gas', 'solar', 'wind', 'hydro', 'nuclear', 'waste', 'unknown')),
CONSTRAINT fuel_ferc1_fuel_unit_enum CHECK (fuel_unit IN ('ton', 'mcf', 'bbl', 'gal', 'kgal', 'gramsU', 'kgU', 'klbs', 'btu', 'mmbtu', 'mwdth', 'mwhth', 'unknown'))
);
```
Note that `fuel_unit` is a unit that **pint** can understand, and that `fuel_qty_burned` is a column of data that could be expressed in terms of actual units, not merely as a dimensionless number. Ditto the `fuel_cost_per_unit_...` columns. Is there a way to give a column a default metadata unit (such as *tons* or *USD/ton*) and then let that be overridden when the metadata in another column says *barrels* or *USD/gramsU*?
@catalyst-cooperative",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1460/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
988556488,MDU6SXNzdWU5ODg1NTY0ODg=,1459,suggestion: allow `datasette --open` to take a relative URL,51016,ctb,open,0,,,,,1,2021-09-05T17:17:07Z,2021-09-05T19:59:15Z,,CONTRIBUTOR,,"(soft suggestion because I'm not sure I'm using datasette right yet)
Over at https://github.com/ctb/2021-sourmash-datasette, I'm playing around with datasette, and I'm creating some static pages to send people to the right facets. There may well be better ways of achieving this end goal, and I will find out if so, I'm sure!
But regardless I think it might be neat to support an option to allow `-o/--open` to take a relative URL, that then gets appended to the hostname and port. This would let me improve my documentation. I don't see any downsides, either, but 🤷 there may well be some :)
Happy to dig in and provide a PR if it's of interest. I'm not sure off the top of my head how to support an optional value to a parameter in argparse - the current `-o` behavior is kinda nice so it'd be suboptimal to require a url for `-o`. Maybe `--open-url=` or something would work?
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1459/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
988552851,MDU6SXNzdWU5ODg1NTI4NTE=,1456,conda install results in non-functioning `datasette serve` due to out-of-date asgiref,51016,ctb,open,0,,,,,0,2021-09-05T16:59:55Z,2021-09-05T16:59:55Z,,CONTRIBUTOR,,"Over in https://github.com/ctb/2021-sourmash-datasette, I discovered that the following commands fail:
```
conda create -n datasette4 -y datasette=0.58.1
conda activate datasette4
datasette gathertax.db
```
with `ImportError: cannot import name 'WebSocketScope' from 'asgiref.typing'`.
This appears to be because asgiref 3.3.4 doesn't have WebSocketScope, but later versions do - a simple
```
pip install asgiref==3.4.1
```
fixes the problem for me, at least to the point where I can run datasette and poke around as usual.
I note that over in the conda-forge recipe, https://github.com/conda-forge/datasette-feedstock/blob/master/recipe/meta.yaml pins asgiref to < 3.4.0, but I'm not sure why - so I'm not sure how to best resolve this issue :).",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1456/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
988325628,MDExOlB1bGxSZXF1ZXN0NzI3MjY1MDI1,1455,Add scientists to target groups,198537,rgieseke,closed,0,,,,,3,2021-09-04T16:28:58Z,2021-09-04T16:32:21Z,2021-09-04T16:31:38Z,CONTRIBUTOR,simonw/datasette/pulls/1455,"Not sure if you want them mentioned explicitly (it's already a long list), but following up on https://twitter.com/simonw/status/1434176989565382656",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1455/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
978357984,MDU6SXNzdWU5NzgzNTc5ODQ=,1446,Modify base.html template to support optional sticky footer,9599,simonw,closed,0,,,,,3,2021-08-24T18:11:12Z,2021-08-31T01:54:59Z,2021-08-24T20:32:47Z,OWNER,,"The neatest way to have the footer stick to the bottom of the browser window that I've found is to use the flexbox pattern from https://css-tricks.com/couple-takes-sticky-footer/
```html
content
```
```css
html, body {
height: 100%;
}
body {
display: flex;
flex-direction: column;
}
.content {
flex: 1 0 auto;
}
.footer {
flex-shrink: 0;
}
```
I tried this in a custom plugin but it ended up having to duplicate the entire `base.html` template just to get a wrapper around the not-footer content.
I think Datasette's own `base.html` template should have this wrapper element instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1446/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
982803408,MDU6SXNzdWU5ODI4MDM0MDg=,1454,Feature Request: Publish to IPFS,1560788,blitmap,open,0,,,,,0,2021-08-30T13:36:18Z,2021-08-30T13:36:18Z,,NONE,,"Hello,
I am a huge fan of this being used for exploring data. I think it has a lot of flexibility not found in other tools.
I'm not sure if what I'm asking for is possible: Can this be extended to publish to IPFS?
IPFS is an attractive hosting option for decentralized journalism.
Food for thought ~",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1454/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
981676832,MDU6SXNzdWU5ODE2NzY4MzI=,1449,`register_commands()` plugin hook to register extra CLI commands,9599,simonw,closed,0,,,,,13,2021-08-28T00:26:21Z,2021-08-28T01:58:35Z,2021-08-28T01:43:11Z,OWNER,,"The `datasette` CLI tool currently has 7 subcommands: `serve`, `inspect`, `install`, `package`, `plugins`, `publish` and `uninstall`.
A plugin hook could allow plugins to register extra subcommands.
I've avoided this for quite a while because I didn't have good use-cases for it - but the existence of the `datasette install xxx` command for installing packages into the correct virtual environment means that actually there's a good reason to do this: it would allow plugins to provide additional command-line mechanisms without the user having to understand how virtual environments work in order to install those commands into the same environment as the rest of Datasette.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1449/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
981681138,MDU6SXNzdWU5ODE2ODExMzg=,1450,"Datasette --help should show something more useful than ""Datasette!""",9599,simonw,closed,0,,,,,1,2021-08-28T00:44:51Z,2021-08-28T00:49:07Z,2021-08-28T00:49:07Z,OWNER,,"https://github.com/simonw/datasette/blob/a1a33bb5822214be1cebd98cd858b2058d91a4aa/datasette/cli.py#L122-L127
_Originally spotted by @simonw in https://github.com/simonw/datasette/issues/1449#issuecomment-907539668_
```
~ % datasette --help
Usage: datasette [OPTIONS] COMMAND [ARGS]...
Datasette!
Options:
--version Show the version and exit.
--help Show this message and exit.
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1450/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
642572841,MDU6SXNzdWU2NDI1NzI4NDE=,859,Database page loads too slowly with many large tables (due to table counts),3243482,abdusco,open,0,,,,,21,2020-06-21T14:23:17Z,2021-08-25T21:59:55Z,,CONTRIBUTOR,,"Hey,
I have a database that I save in HTML from couple of web scrapers. There are around 200k+, 50+ rows in a couple of tables, with sqlite file weighing around 600MB.
The app runs on a VPS with 2 core CPU, 4GB RAM and refreshing database page regularly takes more than 10 seconds. I was suspecting that counting tables was the culprit, but manually running `select count(*) from table_name` for the largest table finishes under a second.
I've looked at the source code. There's a check for index page for mutable databases larger than 100MB
https://github.com/simonw/datasette/blob/799c5d53570d773203527f19530cf772dc2eeb24/datasette/views/index.py#L15
but this check is not performed for database page.
I've manually crippled `Database::table_counts` method
```py
async def table_counts(self, limit=10):
if not self.is_mutable and self.cached_table_counts is not None:
return self.cached_table_counts
# Try to get counts for each table, $limit timeout for each count
counts = {}
for table in await self.table_names():
try:
# table_count = (
# await self.execute(
# ""select count(*) from [{}]"".format(table),
# custom_time_limit=limit,
# )
# ).rows[0][0]
counts[table] = 10 # table_count
# In some cases I saw ""SQL Logic Error"" here in addition to
# QueryInterrupted - so we catch that too:
except (QueryInterrupted, sqlite3.OperationalError, sqlite3.DatabaseError):
counts[table] = None
if not self.is_mutable:
self.cached_table_counts = counts
return counts
```
now the page loads in <100ms.
Is it possible to apply size check on database page too?
/-/versions output
{
""python"": {
""version"": ""3.8.0"",
""full"": ""3.8.0 (default, Oct 28 2019, 16:14:01) \n[GCC 8.3.0]""
},
""datasette"": {
""version"": ""0.44""
},
""asgi"": ""3.0"",
""uvicorn"": ""0.11.5"",
""sqlite"": {
""version"": ""3.22.0"",
""fts_versions"": [
""FTS5"",
""FTS4"",
""FTS3""
],
""extensions"": {
""json1"": null
},
""compile_options"": [
""COMPILER=gcc-7.4.0"",
""ENABLE_COLUMN_METADATA"",
""ENABLE_DBSTAT_VTAB"",
""ENABLE_FTS3"",
""ENABLE_FTS3_PARENTHESIS"",
""ENABLE_FTS3_TOKENIZER"",
""ENABLE_FTS4"",
""ENABLE_FTS5"",
""ENABLE_JSON1"",
""ENABLE_LOAD_EXTENSION"",
""ENABLE_PREUPDATE_HOOK"",
""ENABLE_RTREE"",
""ENABLE_SESSION"",
""ENABLE_STMTVTAB"",
""ENABLE_UNLOCK_NOTIFY"",
""ENABLE_UPDATE_DELETE_LIMIT"",
""HAVE_ISNAN"",
""LIKE_DOESNT_MATCH_BLOBS"",
""MAX_SCHEMA_RETRY=25"",
""MAX_VARIABLE_NUMBER=250000"",
""OMIT_LOOKASIDE"",
""SECURE_DELETE"",
""SOUNDEX"",
""TEMP_STORE=1"",
""THREADSAFE=1""
]
}
}
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/859/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
947640902,MDExOlB1bGxSZXF1ZXN0NjkyNTk2MDA2,1400,Bump black from 21.6b0 to 21.7b0,49699333,dependabot[bot],closed,0,,,,,1,2021-07-19T13:13:41Z,2021-08-25T01:29:56Z,2021-08-25T01:29:55Z,CONTRIBUTOR,simonw/datasette/pulls/1400,"Bumps [black](https://github.com/psf/black) from 21.6b0 to 21.7b0.
Release notes
Sourced from black's releases .
21.7b0
Black
Configuration files using TOML features higher than spec v0.5.0 are now supported
(#2301 )
Add primer support and test for code piped into black via STDIN (#2315 )
Fix internal error when FORCE_OPTIONAL_PARENTHESES
feature is enabled (#2332 )
Accept empty stdin (#2346 )
Provide a more useful error when parsing fails during AST safety checks (#2304 )
Docker
Add new latest_release
tag automation to follow latest black release on docker
images (#2374 )
Integrations
The vim plugin now searches upwards from the directory containing the current buffer
instead of the current working directory for pyproject.toml. (#1871 )
The vim plugin now reads the correct string normalization option in pyproject.toml
(#1869 )
The vim plugin no longer crashes Black when there's boolean values in pyproject.toml
(#1869 )
Changelog
Sourced from black's changelog .
21.7b0
Black
Configuration files using TOML features higher than spec v0.5.0 are now supported
(#2301 )
Add primer support and test for code piped into black via STDIN (#2315 )
Fix internal error when FORCE_OPTIONAL_PARENTHESES
feature is enabled (#2332 )
Accept empty stdin (#2346 )
Provide a more useful error when parsing fails during AST safety checks (#2304 )
Docker
Add new latest_release
tag automation to follow latest black release on docker
images (#2374 )
Integrations
The vim plugin now searches upwards from the directory containing the current buffer
instead of the current working directory for pyproject.toml. (#1871 )
The vim plugin now reads the correct string normalization option in pyproject.toml
(#1869 )
The vim plugin no longer crashes Black when there's boolean values in pyproject.toml
(#1869 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.6b0&new-version=21.7b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1400/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
970386262,MDExOlB1bGxSZXF1ZXN0NzEyMzQ2MTk5,1433,"Update trustme requirement from <0.9,>=0.7 to >=0.7,<0.10",49699333,dependabot[bot],closed,0,,,,,1,2021-08-13T13:10:24Z,2021-08-25T01:29:27Z,2021-08-25T01:29:26Z,CONTRIBUTOR,simonw/datasette/pulls/1433,"Updates the requirements on [trustme](https://github.com/python-trio/trustme) to permit the latest version.
Commits
8fc5bf9
Bump version to 0.9.0
913e21d
Bump types-cryptography from 3.3.3 to 3.3.5 (#342 )
c66709d
Bump types-pyopenssl from 20.0.4 to 20.0.5 (#343 )
5131f79
Add type annotations (#341 )
a411dad
Bump charset-normalizer from 2.0.3 to 2.0.4 (#340 )
be5ec8a
Bump sphinx from 4.1.1 to 4.1.2
d3b8865
Bump charset-normalizer from 2.0.2 to 2.0.3
4503bef
Merge pull request #334 from python-trio/dependabot/pip/charset-normalizer-2.0.2
ce8099d
Merge pull request #335 from python-trio/dependabot/pip/sphinx-4.1.1
8b6d3c6
Merge pull request #336 from python-trio/dependabot/pip/idna-3.2
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1433/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
978614898,MDExOlB1bGxSZXF1ZXN0NzE5MTc1NTkz,1447,Remove underscore from search mode parameter name,127565,wragge,closed,0,,,,,1,2021-08-25T01:28:04Z,2021-08-25T01:28:58Z,2021-08-25T01:28:58Z,CONTRIBUTOR,simonw/datasette/pulls/1447,The fulltext search documentation refers to the parameter as `searchmode` but the `metadata.json` example uses `search_mode`. The latter doesn't actually seem to work.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1447/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
977323133,MDU6SXNzdWU5NzczMjMxMzM=,1445,Ability to search for text across all columns in a table,9599,simonw,open,0,,,,,5,2021-08-23T18:50:48Z,2021-08-23T19:10:17Z,,OWNER,,"When I'm working with new data I often find myself wanting to run a search for text embedded in ANY of the columns of a table, without having to even fully understand the schema first.
I figured out a trick for doing that using a SQL-generated SQL query here: https://til.simonwillison.net/datasette/search-all-columns-trick
But maybe this should be a core Datasette feature? Or a plugin?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1445/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
657572753,MDU6SXNzdWU2NTc1NzI3NTM=,894,?sort=colname~numeric to sort by by column cast to real,9599,simonw,open,0,,,,,21,2020-07-15T18:47:48Z,2021-08-20T02:07:53Z,,OWNER,,"If a text column actually contains numbers, being able to ""sort by column, treated as numeric"" would be really useful.
Probably depends on column actions enabled by #690",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/894/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
974995592,MDU6SXNzdWU5NzQ5OTU1OTI=,1443,datasette.databases should be a documented property,9599,simonw,closed,0,,,,,1,2021-08-19T19:53:04Z,2021-08-19T21:25:07Z,2021-08-19T21:23:47Z,OWNER,,"https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/app.py#L231
I want to use it in https://github.com/simonw/datasette-block-robots/issues/5",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1443/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
970463436,MDExOlB1bGxSZXF1ZXN0NzEyNDEyODgz,1434,Enrich arbitrary query results with foreign key links and column descriptions,9599,simonw,open,0,,,,,1,2021-08-13T14:43:01Z,2021-08-19T21:18:58Z,,OWNER,simonw/datasette/pulls/1434,"Refs #1293, follows #942.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1434/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
974987856,MDU6SXNzdWU5NzQ5ODc4NTY=,1442,Mechanism to cause specific branches to deploy their own demos,9599,simonw,closed,0,,,,,6,2021-08-19T19:41:39Z,2021-08-19T21:11:45Z,2021-08-19T21:09:40Z,OWNER,,"A useful capability would be if it was super-easy to say ""any pushes to branch X should be deployed to `latest-X.datasette.io`"".
I'd like to use this for the column query information work in #1434",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1442/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
975049826,MDExOlB1bGxSZXF1ZXN0NzE2MjYyODI5,1444,Ability to deploy demos of branches,9599,simonw,closed,0,,,,,0,2021-08-19T21:08:04Z,2021-08-19T21:09:44Z,2021-08-19T21:09:39Z,OWNER,simonw/datasette/pulls/1444,See #1442.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1444/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
972918533,MDU6SXNzdWU5NzI5MTg1MzM=,1438,Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions,9599,simonw,open,0,,,,,7,2021-08-17T17:35:36Z,2021-08-18T00:22:23Z,,OWNER,,"> Confirmed: https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A is a page where the URL correctly encoded `%` as `%25` - but then in the HTML on that page that links to the CSV and JSON versions we get this:
>
> ```html
> This data as
> json ,
> CSV
>
> ```
Those CSV and JSON links are incorrect.
_Originally posted by @simonw in https://github.com/simonw/datasette-publish-vercel/issues/48#issuecomment-900497579_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1438/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
268469569,MDU6SXNzdWUyNjg0Njk1Njk=,39,Protect against malicious SQL that causes damage even though our DB is immutable,9599,simonw,closed,0,,,2857392,Ship first public release,4,2017-10-25T16:44:27Z,2021-08-17T23:52:07Z,2017-11-05T02:53:47Z,OWNER,,"I’m currently operating under the assumption that it’s safe to allow arbitrary SQL statements because we are dealing with an immutable database. But this might not be the case - there are some pretty weird SQLite language extensions (ATTACH, PRAGMA etc) and I’m not certain they cannot be used to break things in a way that would affect future requests to the API.
Solution: provide a “safe mode” option which disables the ?sql= mechanism. This still leaves the URL filter lookups, so I need to make sure that those are “safe”.
In the future I may also implement a whitelist option where datasets can be configured to only allow specific filters against specific columns.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/39/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
970626625,MDU6SXNzdWU5NzA2MjY2MjU=,1435,Turn off suggest facets on tables with large numbers of columns,9599,simonw,open,0,,,,,0,2021-08-13T18:30:48Z,2021-08-13T18:30:48Z,,OWNER,,If a table has 200 columns it will take multiple seconds to try and suggest facets. I should either quit after the first 20 or not suggest facets at all.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1435/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
969548935,MDU6SXNzdWU5Njk1NDg5MzU=,1429,UI for setting `?_size=max` on table page,9599,simonw,open,0,,,,,2,2021-08-12T20:52:09Z,2021-08-13T04:37:41Z,,OWNER,,"It defaults to 100 per page, but you can increase that to 1000 per page using `?_size=max` (or higher if `max_returned_rows` is set higher than that).
But... that's only available to people who know how to hack URLs.
Solution: add a link that sets that option to the pagination block at the bottom of the table:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1429/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
969840302,MDU6SXNzdWU5Njk4NDAzMDI=,1431,`--help-config` should be called `--help-settings`,9599,simonw,closed,0,,,,,1,2021-08-13T00:46:48Z,2021-08-13T01:01:58Z,2021-08-13T01:01:58Z,OWNER,,Follow-on from #1105 rebranding exercise.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1431/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
969758038,MDExOlB1bGxSZXF1ZXN0NzExNzgzNjE2,1430,Column metadata,9599,simonw,closed,0,,,,,1,2021-08-12T23:34:39Z,2021-08-12T23:53:23Z,2021-08-12T23:53:23Z,OWNER,simonw/datasette/pulls/1430,"Refs #942
Still needs:
- [x] Tests
- [x] Documentation",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1430/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
963528457,MDU6SXNzdWU5NjM1Mjg0NTc=,1425,render_cell() hook should support returning an awaitable,9599,simonw,closed,0,,,,,11,2021-08-08T22:32:29Z,2021-08-09T07:14:35Z,2021-08-09T03:00:37Z,OWNER,,"Many of the plugin hooks can return an awaitable - e.g. https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-extra-template-vars - but `render_cell()` doesn't support this.
I recently found myself wanting to execute an additional SQL query from that hook, but it wasn't possible to do that since I couldn't use `await`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959999095,MDU6SXNzdWU5NTk5OTkwOTU=,1421,"""Query parameters"" form shows wrong input fields if query contains ""03:31"" style times",6988,j4mie,closed,0,,,,,11,2021-08-04T07:29:04Z,2021-08-09T03:41:07Z,2021-08-09T03:33:02Z,NONE,,"Datasette version `0.58.1`.
I'm guessing this is a bug in the code that looks for `:param`-style query parameters..
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1421/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
963527045,MDU6SXNzdWU5NjM1MjcwNDU=,1424,Document exceptions that can be raised by db.execute() and friends,9599,simonw,open,0,,,,,4,2021-08-08T22:23:25Z,2021-08-08T22:27:31Z,,OWNER,,Not currently covered here: https://docs.datasette.io/en/stable/internals.html#await-db-execute-sql,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1424/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
961367843,MDU6SXNzdWU5NjEzNjc4NDM=,1422,Ability to default to hiding the SQL for a canned query,9599,simonw,closed,0,,,,,4,2021-08-05T02:51:39Z,2021-08-07T05:32:29Z,2021-08-07T05:32:29Z,OWNER,,"I'm working on a project with some HUGE (400+ lines of SQL) canned queries right now.
Any time you land on the canned query page you have to scroll down a long distance to get to the results!
Would be useful to be able to default to https://latest.datasette.io/fixtures/magic_parameters?_hide_sql=1 without needing the parameter.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1422/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959710008,MDU6SXNzdWU5NTk3MTAwMDg=,1419,`publish cloudrun` should deploy a more recent SQLite version,536941,fgregg,open,0,,,,,3,2021-08-04T00:45:55Z,2021-08-05T03:23:24Z,,CONTRIBUTOR,,"I recently changed from deploying a datasette using `datasette publish heroku` to `datasette publish cloudrun`. [A query that ran on the heroku site](https://odpr.bunkum.us/odpr-6c2f4fc?sql=with+pivot_members+as+%28%0D%0A++select%0D%0A++++f_num%2C%0D%0A++++max%28union_name%29+as+union_name%2C%0D%0A++++max%28aff_abbr%29+as+abbreviation%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2007%0D%0A++++%29+as+%222007%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2008%0D%0A++++%29+as+%222008%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2009%0D%0A++++%29+as+%222009%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2010%0D%0A++++%29+as+%222010%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2011%0D%0A++++%29+as+%222011%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2012%0D%0A++++%29+as+%222012%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2013%0D%0A++++%29+as+%222013%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2014%0D%0A++++%29+as+%222014%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2015%0D%0A++++%29+as+%222015%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2016%0D%0A++++%29+as+%222016%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2017%0D%0A++++%29+as+%222017%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2018%0D%0A++++%29+as+%222018%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2019%0D%0A++++%29+as+%222019%22%0D%0A++from%0D%0A++++lm_data%0D%0A++++left+join+ar_membership+using+%28rpt_id%29%0D%0A++where%0D%0A++++members+%21%3D+%27%27%0D%0A++++and+trim%28desig_name%29+%3D+%27NHQ%27%0D%0A++++and+union_name+not+in+%28%0D%0A++++++%27AFL-CIO%27%2C%0D%0A++++++%27CHANGE+TO+WIN%27%2C%0D%0A++++++%27FOOD+ALLIED+SVC+TRADES+DEPT+AFL-CIO%27%0D%0A++++%29%0D%0A++++and+f_num+not+in+%28387%2C+296%2C+123%2C+347%2C+531897%2C+30410%2C+49%29%0D%0A++++and+lower%28ar_membership.category%29+IN+%28%0D%0A++++++%27active+members%27%2C%0D%0A++++++%27active+professional%27%2C%0D%0A++++++%27regular+members%27%2C%0D%0A++++++%27full+time+member%27%2C%0D%0A++++++%27full+time+members%27%2C%0D%0A++++++%27active+education+support+professional%27%2C%0D%0A++++++%27active%27%2C%0D%0A++++++%27active+member%27%2C%0D%0A++++++%27members%27%2C%0D%0A++++++%27see+item+69%27%2C%0D%0A++++++%27regular%27%2C%0D%0A++++++%27dues+paying+members%27%2C%0D%0A++++++%27building+trades+journeyman%27%2C%0D%0A++++++%27full+per+capita+tax+payers%27%2C%0D%0A++++++%27active+postal%27%2C%0D%0A++++++%27active+membership%27%2C%0D%0A++++++%27full-time+member%27%2C%0D%0A++++++%27regular+active+member%27%2C%0D%0A++++++%27journeyman%27%2C%0D%0A++++++%27member%27%2C%0D%0A++++++%27journeymen%27%2C%0D%0A++++++%27one+half+per+capita+tax+payers%27%2C%0D%0A++++++%27schedule+1%27%2C%0D%0A++++++%27active+memebers%27%2C%0D%0A++++++%27active+members+-+us%27%2C%0D%0A++++++%27dues+paying+membership%27%0D%0A++++%29%0D%0A++GROUP+by%0D%0A++++f_num%0D%0A%29%0D%0Aselect%0D%0A++*%0D%0Afrom%0D%0A++pivot_members%0D%0Aorder+by%0D%0A++%222019%22+desc%3B), now throws a syntax error on the [cloudrun site](https://labordata.bunkum.us/odpr-6c2f4fc?sql=with+pivot_members+as+%28%0D%0A++select%0D%0A++++f_num%2C%0D%0A++++max%28union_name%29+as+union_name%2C%0D%0A++++max%28aff_abbr%29+as+abbreviation%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2007%0D%0A++++%29+as+%222007%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2008%0D%0A++++%29+as+%222008%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2009%0D%0A++++%29+as+%222009%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2010%0D%0A++++%29+as+%222010%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2011%0D%0A++++%29+as+%222011%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2012%0D%0A++++%29+as+%222012%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2013%0D%0A++++%29+as+%222013%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2014%0D%0A++++%29+as+%222014%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2015%0D%0A++++%29+as+%222015%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2016%0D%0A++++%29+as+%222016%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2017%0D%0A++++%29+as+%222017%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2018%0D%0A++++%29+as+%222018%22%2C%0D%0A++++sum%28number%29+filter+%28%0D%0A++++++where%0D%0A++++++++yr_covered+%3D+2019%0D%0A++++%29+as+%222019%22%0D%0A++from%0D%0A++++lm_data%0D%0A++++left+join+ar_membership+using+%28rpt_id%29%0D%0A++where%0D%0A++++members+%21%3D+%27%27%0D%0A++++and+trim%28desig_name%29+%3D+%27NHQ%27%0D%0A++++and+union_name+not+in+%28%0D%0A++++++%27AFL-CIO%27%2C%0D%0A++++++%27CHANGE+TO+WIN%27%2C%0D%0A++++++%27FOOD+ALLIED+SVC+TRADES+DEPT+AFL-CIO%27%0D%0A++++%29%0D%0A++++and+f_num+not+in+%28387%2C+296%2C+123%2C+347%2C+531897%2C+30410%2C+49%29%0D%0A++++and+lower%28ar_membership.category%29+IN+%28%0D%0A++++++%27active+members%27%2C%0D%0A++++++%27active+professional%27%2C%0D%0A++++++%27regular+members%27%2C%0D%0A++++++%27full+time+member%27%2C%0D%0A++++++%27full+time+members%27%2C%0D%0A++++++%27active+education+support+professional%27%2C%0D%0A++++++%27active%27%2C%0D%0A++++++%27active+member%27%2C%0D%0A++++++%27members%27%2C%0D%0A++++++%27see+item+69%27%2C%0D%0A++++++%27regular%27%2C%0D%0A++++++%27dues+paying+members%27%2C%0D%0A++++++%27building+trades+journeyman%27%2C%0D%0A++++++%27full+per+capita+tax+payers%27%2C%0D%0A++++++%27active+postal%27%2C%0D%0A++++++%27active+membership%27%2C%0D%0A++++++%27full-time+member%27%2C%0D%0A++++++%27regular+active+member%27%2C%0D%0A++++++%27journeyman%27%2C%0D%0A++++++%27member%27%2C%0D%0A++++++%27journeymen%27%2C%0D%0A++++++%27one+half+per+capita+tax+payers%27%2C%0D%0A++++++%27schedule+1%27%2C%0D%0A++++++%27active+memebers%27%2C%0D%0A++++++%27active+members+-+us%27%2C%0D%0A++++++%27dues+paying+membership%27%0D%0A++++%29%0D%0A++GROUP+by%0D%0A++++f_num%0D%0A%29%0D%0Aselect%0D%0A++*%0D%0Afrom%0D%0A++pivot_members%0D%0Aorder+by%0D%0A++%222019%22+desc%3B).
I suspect this is because they are running different versions of sqlite3.
- Heroku: sqlite3 3.31.1 ([-/versions](https://odpr.bunkum.us/-/versions))
- Cloudrun: sqlite3 3.27.2 ([-/versions](https://labordata.bunkum.us/-/versions))
If so, it would be great to
1. harmonize the sqlite3 versions across platforms
2. update the docker files so as to update the sqlite3 version for cloudrun",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1419/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
959898166,MDU6SXNzdWU5NTk4OTgxNjY=,1420,`datasette publish cloudrun --cpu X` option,9599,simonw,closed,0,,,,,5,2021-08-04T05:04:31Z,2021-08-05T00:54:59Z,2021-08-04T05:33:48Z,OWNER,,"For setting the number of vCPUs - current valid values are 1, 2 or 4: https://cloud.google.com/run/docs/configuring/cpu
Pass that through to `gcloud run deploy --image IMAGE_URL --cpu CPU`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1420/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959278472,MDU6SXNzdWU5NTkyNzg0NzI=,1417,Use codespell in CI to spot spelling errors,9599,simonw,closed,0,,,,,1,2021-08-03T16:14:15Z,2021-08-03T16:36:40Z,2021-08-03T16:36:40Z,OWNER,,"I noticed Rich is using this: https://github.com/willmcgugan/rich/commit/9c12a4537499797c43725fff5276ef0da62423ef#diff-ce84a1b2c9eb4ab3ea22f610cad7111cb9a2f66365c3b24679901376a2a73ab2
Ran it against the Datasette docs and found a bunch of obvious fixes, surprisingly with no false positives.
```
datasette % codespell docs/*.rst
docs/authentication.rst:63: perfom ==> perform
docs/authentication.rst:76: perfom ==> perform
docs/changelog.rst:429: repsonse ==> response
docs/changelog.rst:503: permissons ==> permissions
docs/changelog.rst:717: compatibilty ==> compatibility
docs/changelog.rst:1172: browseable ==> browsable
docs/deploying.rst:191: similiar ==> similar
docs/internals.rst:434: Respons ==> Response, respond
docs/internals.rst:440: Respons ==> Response, respond
docs/internals.rst:717: tha ==> than, that, the
docs/performance.rst:42: databse ==> database
docs/plugin_hooks.rst:667: utilites ==> utilities
docs/publish.rst:168: countainer ==> container
docs/settings.rst:352: inalid ==> invalid
docs/sql_queries.rst:406: preceeded ==> preceded, proceeded
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1417/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
959284434,MDExOlB1bGxSZXF1ZXN0NzAyNDIyMjYz,1418,Spelling corrections plus CI job for codespell,9599,simonw,closed,0,,,,,2,2021-08-03T16:21:19Z,2021-08-03T16:36:39Z,2021-08-03T16:36:38Z,OWNER,simonw/datasette/pulls/1418,Refs #1417.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1418/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
959276629,MDU6SXNzdWU5NTkyNzY2Mjk=,1416,"Use rich to render tracebacks on errors, if available",9599,simonw,closed,0,,,,,0,2021-08-03T16:12:08Z,2021-08-03T16:12:51Z,2021-08-03T16:12:51Z,OWNER,,"> Now thinking I should try adding Rich as an optional dependency to Datasette - if it's there, show tracebacks using it. Could be really handy for development
> https://twitter.com/simonw/status/1422576091055616003",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1416/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
810394616,MDU6SXNzdWU4MTAzOTQ2MTY=,1227,Configure sphinx.ext.extlinks for issues,9599,simonw,closed,0,,,,,1,2021-02-17T17:38:02Z,2021-08-02T21:38:39Z,2021-02-18T01:20:33Z,OWNER,,"Spotted this in the aspw documentation: https://github.com/rogerbinns/apsw/blob/3.34.0-r1/doc/conf.py#L29-L36
```python
extlinks={
'cvstrac': ('https://sqlite.org/cvstrac/tktview?tn=%s',
'SQLite ticket #'),
'sqliteapi': ('https://sqlite.org/c3ref/%s.html', 'XXYouShouldNotSeeThisXX'),
'issue': ('https://github.com/rogerbinns/apsw/issues/%s',
'APSW issue '),
'source': ('https://github.com/rogerbinns/apsw/blob/master/%s',
''),
}
```
Which lets you link to issues like this:
:issue:`268`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1227/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957345476,MDU6SXNzdWU5NTczNDU0NzY=,1411,Canned query ?sql= is pointlessly echoed in query string starting from hidden mode,9599,simonw,closed,0,,,,,1,2021-08-01T00:17:13Z,2021-08-01T03:27:30Z,2021-08-01T00:58:17Z,OWNER,,"Example: https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1
Submitting that form again results in this:
https://latest.datasette.io/fixtures/neighborhood_search?sql=%0D%0Aselect+neighborhood%2C+facet_cities.name%2C+state%0D%0Afrom+facetable%0D%0A++++join+facet_cities%0D%0A++++++++on+facetable.city_id+%3D+facet_cities.id%0D%0Awhere+neighborhood+like+%27%25%27+%7C%7C+%3Atext+%7C%7C+%27%25%27%0D%0Aorder+by+neighborhood%3B%0D%0A&_hide_sql=1&text=cork
Because the HTML on https://latest.datasette.io/fixtures/neighborhood_search?text=cork&_hide_sql=1 includes this:
```html
Custom SQL query returning 1 row (show )
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1411/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957347860,MDU6SXNzdWU5NTczNDc4NjA=,1412,Mention WAL mode in the documentation (plus backup tips),9599,simonw,open,0,,,,,0,2021-08-01T00:27:11Z,2021-08-01T00:27:11Z,,OWNER,,"This is useful for people who are deploying Datasette with any write functionality, especially if they might be using something like EFS.
I can add a section about this to the bottom of https://docs.datasette.io/en/stable/deploying.html and then link to that section from both https://docs.datasette.io/en/stable/sql_queries.html#writable-canned-queries and https://docs.datasette.io/en/stable/internals.html#await-db-execute-write-sql-params-none-block-false
Also useful: mention that just copying a SQLite database file while it is being written to may not get a consistent file, so tell people to use one of the SQLite backup mechanisms instead.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1412/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
957315684,MDU6SXNzdWU5NTczMTU2ODQ=,1410,Rename settings to `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`,9599,simonw,open,0,,,3268330,Datasette 1.0,0,2021-07-31T20:27:12Z,2021-07-31T20:27:49Z,,OWNER,,"> If I was prone to over-thinking (which I am) I'd note that `allow_facet` and `allow_download` and `allow_csv_stream` are all settings that do NOT have an equivalent in the newer permissions system, which is itself a little weird and inconsistent.
>
> So maybe there's a future task where I introduce those as both permissions and metadata `""allow_x""` blocks, then rename the settings themselves to be called `default_allow_facet` and `default_allow_download` and `default_allow_csv_stream`.
>
> If I was going to do that I should get it in before Datasette 1.0.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1409#issuecomment-890400425_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1410/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
957298475,MDU6SXNzdWU5NTcyOTg0NzU=,1407,OSError: AF_UNIX path too long in ds_unix_domain_socket_server,9599,simonw,closed,0,,,,,2,2021-07-31T18:36:06Z,2021-07-31T19:03:44Z,2021-07-31T19:03:44Z,OWNER,,"Got this exception while working on #1406.
```
@pytest.fixture(scope=""session"")
def ds_unix_domain_socket_server(tmp_path_factory):
socket_folder = tmp_path_factory.mktemp(""uds"")
uds = str(socket_folder / ""datasette.sock"")
ds_proc = subprocess.Popen(
[""datasette"", ""--memory"", ""--uds"", uds],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
cwd=tempfile.gettempdir(),
)
# Give the server time to start
time.sleep(1.5)
# Check it started successfully
> assert not ds_proc.poll(), ds_proc.stdout.read().decode(""utf-8"")
E AssertionError: INFO: Started server process [48453]
E INFO: Waiting for application startup.
E INFO: Application startup complete.
E Traceback (most recent call last):
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/datasette"", line 33, in
E sys.exit(load_entry_point('datasette', 'console_scripts', 'datasette')())
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__
E return self.main(*args, **kwargs)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1062, in main
E rv = self.invoke(ctx)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke
E return _process_result(sub_ctx.command.invoke(sub_ctx))
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke
E return ctx.invoke(self.callback, **ctx.params)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 763, in invoke
E return __callback(*args, **kwargs)
E File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 583, in serve
E uvicorn.run(ds.app(), **uvicorn_kwargs)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/main.py"", line 393, in run
E server.run()
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 50, in run
E loop.run_until_complete(self.serve(sockets=sockets))
E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py"", line 616, in run_until_complete
E return future.result()
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 67, in serve
E await self.startup(sockets=sockets)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 133, in startup
E server = await asyncio.start_unix_server(
E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/streams.py"", line 132, in start_unix_server
E return await loop.create_unix_server(factory, path, **kwds)
E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/unix_events.py"", line 296, in create_unix_server
E sock.bind(path)
E OSError: AF_UNIX path too long
E
E assert not 1
E + where 1 = >()
E + where > = .poll
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1407/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957302085,MDU6SXNzdWU5NTczMDIwODU=,1408,"Review places in codebase that use os.chdir(), in particularly relating to tests",9599,simonw,open,0,,,,,2,2021-07-31T18:57:06Z,2021-07-31T19:00:32Z,,OWNER,,"> To clarify: the core problem here is that an error is thrown any time you call `os.getcwd()` but the directory you are currently in has been deleted.
>
> `runner.isolated_filesystem()` assumes that the current directory in has not been deleted. But the various temporary directory utilities in `pytest` work by creating directories and then deleting them.
>
> Maybe there's a larger problem here that I play a bit fast and loose with `os.chdir()` in both the test suite and in various lines of code in Datasette itself (in particular in the publish commands)?
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1406#issuecomment-890390198_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1408/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
956303470,MDU6SXNzdWU5NTYzMDM0NzA=,1406,Tests failing with FileNotFoundError in runner.isolated_filesystem,9599,simonw,closed,0,,,,,8,2021-07-30T00:39:00Z,2021-07-31T18:56:35Z,2021-07-31T18:56:35Z,OWNER,,"e.g. https://github.com/simonw/datasette/runs/3197141955
I've seen this error before, but I don't yet have a good workaround for it.
```
@contextlib.contextmanager
def isolated_filesystem(
self, temp_dir: t.Optional[t.Union[str, os.PathLike]] = None
) -> t.Iterator[str]:
""""""A context manager that creates a temporary directory and
changes the current working directory to it. This isolates tests
that affect the contents of the CWD to prevent them from
interfering with each other.
:param temp_dir: Create the temporary directory under this
directory. If given, the created directory is not removed
when exiting.
.. versionchanged:: 8.0
Added the ``temp_dir`` parameter.
""""""
> cwd = os.getcwd()
E FileNotFoundError: [Errno 2] No such file or directory
/opt/hostedtoolcache/Python/3.6.14/x64/lib/python3.6/site-packages/click/testing.py:466: FileNotFoundError
=========================== short test summary info ============================
FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_apt_get_install
FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[---setting force_https_urls on]
FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[--setting base_url /foo---setting base_url /foo --setting force_https_urls on]
FAILED tests/test_publish_cloudrun.py::test_publish_cloudrun_extra_options[--setting force_https_urls off---setting force_https_urls off]
FAILED tests/test_publish_heroku.py::test_publish_heroku_requires_heroku - Fi...
FAILED tests/test_publish_heroku.py::test_publish_heroku_installs_plugin - Fi...
FAILED tests/test_publish_heroku.py::test_publish_heroku - FileNotFoundError:...
FAILED tests/test_publish_heroku.py::test_publish_heroku_plugin_secrets - Fil...
================== 8 failed, 920 passed in 188.22s (0:03:08) ===================
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1406/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
947044667,MDU6SXNzdWU5NDcwNDQ2Njc=,1398,Documentation on using Datasette as a library,9599,simonw,open,0,,,,,1,2021-07-18T14:15:27Z,2021-07-30T03:21:49Z,,OWNER,,"Instantiating `Datasette()` directly is an increasingly interesting pattern. I do it in tests all the time, but thanks to `datasette.client` there are plenty of neat things you can do with it in a library context.
Maybe support `from datasette import Datasette` for this.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1398/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
955316250,MDU6SXNzdWU5NTUzMTYyNTA=,1405,utils.parse_metadata() should be a documented internal function,9599,simonw,closed,0,,,,,3,2021-07-28T23:51:39Z,2021-07-29T23:33:30Z,2021-07-29T23:30:24Z,OWNER,,Because it's used by this plugin: https://github.com/simonw/datasette-remote-metadata,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1405/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
530513784,MDExOlB1bGxSZXF1ZXN0MzQ3MTc5MDgx,644,Validate metadata json on startup,6025893,chris48s,closed,0,,,,,1,2019-11-30T00:32:15Z,2021-07-28T17:58:45Z,2021-07-28T17:58:45Z,CONTRIBUTOR,simonw/datasette/pulls/644,"This PR adds a sanity check which builds up a marshmallow schema on-the-fly based on the structure of the database(s) on startup and then validates the metadata json against it.
In case of invalid data, this will raise with a descriptive error e.g:
```
marshmallow.exceptions.ValidationError: {'databases': {'fixtures': {'tables': {'not_a_table': ['Unknown field.']}}}}
```
Closes #260
---
This was intended to be fairly self-contained, but then while I was working on it, I hit some problems getting the tests to pass in the context of the test suite as a whole. My tests passed in isolation, but then failed while doing a full test suite run. That's when the worms started coming out of the can :bug: After some sleuthing, it turned out this was essentially the result of several issues intersecting:
* There are certain events in the application lifecycle where the metadata schema can be modified after it is loaded e.g: https://github.com/simonw/datasette/blob/a562f2965552fb2dbbbd74df245c9965ee23d886/datasette/app.py#L299-L320 This means that sometimes what goes in isn't always exactly what comes out when you call `/-/metadata`.
* Because the test fixtures use session scope for performance reasons if one unit test performs an action which mutates the metadata, that can impact on other unit tests which run after it using the same fixture.
* Because the `self._metadata` property was being set with a simple assignment `self._metadata = metadata`, that created an object reference to the test fixture data, so operating on `self._metadata` was actually modifying the test fixture `METADATA` meaning that depending on when it was loaded in the test suite lifecycle, `METADATA` had different content, which was somewhat unexpected.
As such, I've added some band-aids in 3552024 and 6859fd8:
* Switching the metadata object to a `deepcopy` of the input prevents us directly mutating the input fixture.
* I've switched some of the tests to use a fixture with function scope instead of session scope so we're working on a clean copy that hasn't been mutated by other tests where necessary but keeping session scope in most cases for performance.
* I haven't really addressed the fact that sometimes the metadata object gets mutated in place, so the object that is served from `/-/metadata` isn't necessarily always exactly the same as the file you fed into it on init. I'm not sure how much of a problem that is. The way the tests were written makes me think it was unexpected, but getting into it feels like too much scope creep for this PR so its probably best addressed as another issue.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/644/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
953352015,MDU6SXNzdWU5NTMzNTIwMTU=,1404,`register_routes()` hook should take `datasette` argument,9599,simonw,closed,0,,,,,1,2021-07-26T23:00:33Z,2021-07-26T23:27:07Z,2021-07-26T23:26:00Z,OWNER,,Currently that plugin hook takes no arguments at all. This means it's not possible to conditionally register routes based on Datasette plugin configuration.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1404/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
951185411,MDU6SXNzdWU5NTExODU0MTE=,1402,feature request: social meta tags,536941,fgregg,open,0,,,,,2,2021-07-23T01:57:23Z,2021-07-26T19:31:41Z,,CONTRIBUTOR,,"it would be very nice if the twitter, slack, and other social media could make rich cards when people post a link to a datasette instance
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1402/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
855446829,MDExOlB1bGxSZXF1ZXN0NjEzMTc4OTY4,1296,Dockerfile: use Ubuntu 20.10 as base,82332573,tmcl-it,open,0,,,,,4,2021-04-12T00:23:32Z,2021-07-20T08:52:13Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1296,"This PR changes the main Dockerfile to use ubuntu:20.10 as base image instead of python:3.9.2-slim-buster (itself based on debian:buster-slim).
The Dockerfile is essentially the one from https://github.com/simonw/datasette/issues/1249#issuecomment-803698983 with some additional cleanups to slim it down.
This fixes a couple of issues:
1. The SQLite version in Debian Buster (2.6.0) doesn't support generated columns
2. Installing SpatiaLite from the Debian sid repositories has the side effect of also installing updates to libc and libstdc++ from sid.
As a bonus, the Docker image becomes smaller:
```
$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
datasette 0.56-ubuntu f7aca255140a 5 hours ago 212MB
datasetteproject/datasette 0.56 efb3b282f390 13 days ago 258MB
```
### Reproduction of the first issue
```
$ curl -O https://latest.datasette.io/fixtures.db
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 260k 0 260k 0 0 489k 0 --:--:-- --:--:-- --:--:-- 489k
$ docker run -v `pwd`:/mnt datasetteproject/datasette:0.56 datasette /mnt/fixtures.db
Traceback (most recent call last):
File ""/usr/local/bin/datasette"", line 8, in
sys.exit(cli())
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 829, in __call__
return self.main(*args, **kwargs)
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 782, in main
rv = self.invoke(ctx)
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 610, in invoke
return callback(*args, **kwargs)
File ""/usr/local/lib/python3.9/site-packages/datasette/cli.py"", line 544, in serve
asyncio.get_event_loop().run_until_complete(check_databases(ds))
File ""/usr/local/lib/python3.9/asyncio/base_events.py"", line 642, in run_until_complete
return future.result()
File ""/usr/local/lib/python3.9/site-packages/datasette/cli.py"", line 584, in check_databases
await database.execute_fn(check_connection)
File ""/usr/local/lib/python3.9/site-packages/datasette/database.py"", line 155, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
File ""/usr/local/lib/python3.9/concurrent/futures/thread.py"", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File ""/usr/local/lib/python3.9/site-packages/datasette/database.py"", line 153, in in_thread
return fn(conn)
File ""/usr/local/lib/python3.9/site-packages/datasette/utils/__init__.py"", line 892, in check_connection
for r in conn.execute(
sqlite3.DatabaseError: malformed database schema (generated_columns) - near ""AS"": syntax error
```
Here is the SQLite version:
```
$ docker run -v `pwd`:/mnt -it datasetteproject/datasette:0.56 /bin/bash
root@d9220d3b95dd:/# python3
Python 3.9.2 (default, Mar 27 2021, 02:50:26)
[GCC 8.3.0] on linux
Type ""help"", ""copyright"", ""credits"" or ""license"" for more information.
>>> import sqlite3
>>> sqlite3.version
'2.6.0'
```
### Reproduction of the second issue
```
$ docker build . -t datasette --build-arg VERSION=0.55
[...snip...]
The following packages will be upgraded:
libc-bin libc6 libstdc++6
[...snip...]
Unpacking libc6:amd64 (2.31-11) over (2.28-10) ...
[...snip...]
Unpacking libstdc++6:amd64 (10.2.1-6) over (8.3.0-6) ...
[...snip...]
```
Both libc and libstdc++ are backwards compatible, so the image still works, but it will result in a combination of libraries and Python versions that exists only in the Datasette image, so it's likely untested. In addition, since Debian sid is an always-changing rolling-release, the versions of libc, libstdc++, Spatialite, and their dependencies change frequently, so the library versions in the Datasette image will depend on the day when it was built.
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1296/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
947596222,MDExOlB1bGxSZXF1ZXN0NjkyNTU3Mzgx,1399,Multiple sort,87192257,jgryko5,open,0,,,,,0,2021-07-19T12:20:14Z,2021-07-19T12:20:14Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1399,"Closes #197.
I have added support for sorting by multiple parameters as mentioned in the issue above, and together with that, a suggestion on how to implement such sorting in the user interface.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1399/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
275125561,MDU6SXNzdWUyNzUxMjU1NjE=,123,Datasette serve should accept paths/URLs to CSVs and other file formats,9599,simonw,open,0,,,,,9,2017-11-19T02:05:48Z,2021-07-19T00:04:32Z,,OWNER,,"This would remove the csvs-to-sqlite step which I end up using for almost everything.
I'm hesitant to introduce pandas as a required dependency though since it require compiling numpy. Could build it so this option is only available if you have pandas installed.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/123/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,
792652391,MDU6SXNzdWU3OTI2NTIzOTE=,1199,Experiment with PRAGMA mmap_size=N,9599,simonw,open,0,,,,,2,2021-01-23T21:24:09Z,2021-07-17T17:39:17Z,,OWNER,,"https://sqlite.org/mmap.html - SQLite supports memory-mapped I/O but it's disabled by default. The `PRAGMA mmap_size=N` option can be used to enable it.
It would be very interesting to understand the impact this could have on Datasette performance for various different shapes of data.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1199/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
946553953,MDExOlB1bGxSZXF1ZXN0NjkxNzA3NDA5,1397,"Fix for race condition in refresh_schemas(), closes #1231",9599,simonw,closed,0,,,,,0,2021-07-16T19:44:43Z,2021-07-16T19:45:00Z,2021-07-16T19:44:58Z,OWNER,simonw/datasette/pulls/1397,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1397/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
811367257,MDU6SXNzdWU4MTEzNjcyNTc=,1231,Race condition errors in new refresh_schemas() mechanism,9599,simonw,closed,0,,,,,11,2021-02-18T18:49:54Z,2021-07-16T19:44:59Z,2021-07-16T19:44:59Z,OWNER,,I tried running a Locust load test against Datasette and hit an error message about a failure to create tables because they already existed. I think this means there are race conditions in the new `refresh_schemas()` mechanism added in #1150.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
944870799,MDU6SXNzdWU5NDQ4NzA3OTk=,1394,Big performance boost on faceting: skip the inner order by,9599,simonw,closed,0,,,,,4,2021-07-14T23:32:29Z,2021-07-16T02:23:32Z,2021-07-15T00:05:50Z,OWNER,,"I just noticed something that could make for a huge performance improvement in faceting.
The default query used by Datasette when faceting looks like this:
```sql
select
country_long,
count(*)
from (
select * from [global-power-plants] order by rowid
)
where
country_long is not null
group by
country_long
order by
count(*) desc
```
Here it takes 53ms: https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D+order+by+rowid%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc
Note that there's a `order by rowid` in there which isn't necessary - the order on that inner query doesn't matter since we're grouping and counting.
I had assumed SQLite would optimize this away - but it turns out it doesn't! Consider this version of the query, with that pointless order by removed:
```
select
country_long,
count(*)
from (
select * from [global-power-plants]
)
where
country_long is not null
group by
country_long
order by
count(*) desc
```
https://global-power-plants.datasettes.com/global-power-plants?sql=select%0D%0A++country_long%2C%0D%0A++count%28*%29%0D%0Afrom+%28%0D%0A++select+*+from+%5Bglobal-power-plants%5D%0D%0A%29%0D%0Awhere%0D%0A++country_long+is+not+null%0D%0Agroup+by%0D%0A++country_long%0D%0Aorder+by%0D%0A++count%28*%29+desc runs in 7.2ms!
I tried this optimization on a table with 2.5m rows in it - without the optimization it took 5 seconds, with the optimization it took 450ms. So this is a very significant improvement!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1394/reactions"", ""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
612673948,MDU6SXNzdWU2MTI2NzM5NDg=,759,fts search on a column doesn't work anymore due to escape_fts,133845,Krazybug,closed,0,,,,,3,2020-05-05T15:03:44Z,2021-07-16T02:11:54Z,2020-05-06T17:50:57Z,NONE,,"Hi and first, thank you for this awesome work you make with this projet.
On a db indexed in full text search, I can't query on indexed column anymore.
This request ""cauvin language:ita"": is running smoothly on a old version of datasette but not on the current version.
Compare the current version query
`select uuid, title, authors, year, series, language, formats, publisher, tags, identifiers from summary where rowid in (select rowid from summary_fts where summary_fts match escape_fts(:search)) order by uuid limit 101`
To an older version:
`select title, authors, series, uuid, language, identifiers, tags, publisher, formats, year, links from summary where rowid in (select rowid from summary_fts where summary_fts match :search) order by uuid limit 101`
_language_ is a searchable column but now the search string is known as ""cauvin language:ita"" literally as a search term. columns are not parsed.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/759/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
539590148,MDU6SXNzdWU1Mzk1OTAxNDg=,651,fts5 syntax error when using punctuation,2181410,clausjuhl,closed,0,,,,,3,2019-12-18T10:25:35Z,2021-07-14T19:26:06Z,2019-12-30T06:42:55Z,NONE,,"Hi Simon
I get a syntax error when using punctuation or special characters in a fulltext search (using fts5). I created the virtual table using sqlite-utils' ""enable-fts""-command.
The same error appears on Niche Museums [https://www.niche-museums.com/browse/search?q=park.](https://www.niche-museums.com/browse/search?q=park.), but works fine in most of your other datasette-examples, e.g. register-of-members-interests [https://register-of-members-interests.datasettes.com/regmem-98dc8b7/items?_search=mins.](https://register-of-members-interests.datasettes.com/regmem-98dc8b7/items?_search=mins.)
What am I doing wrong? Many thanks!
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/651/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
941412189,MDExOlB1bGxSZXF1ZXN0Njg3MzA0MjQy,1393,Update deploying.rst,80737,aslakr,closed,0,,,,,1,2021-07-11T09:32:16Z,2021-07-13T18:32:49Z,2021-07-13T18:32:49Z,CONTRIBUTOR,simonw/datasette/pulls/1393,"Example on how to use Unix domain socket option on Apache. Not testet.
(Usually I would have used [`ProxyPassReverse`](https://httpd.apache.org/docs/current/mod/mod_proxy.html#proxypassreverse) in combination with `ProxyPass` , i.e.
```apache
ProxyPass /my-datasette/ http://127.0.0.1:8009/my-datasette/
ProxyPassReverse /my-datasette/ http://127.0.0.1:8009/my-datasette/
```
and
```apache
ProxyPass /my-datasette/ unix:/tmp/datasette.sock|http://localhost/my-datasette/
ProxyPassReverse /my-datasette/ unix:/tmp/datasette.sock|http://localhost/my-datasette/
```
)",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1393/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
941403676,MDExOlB1bGxSZXF1ZXN0Njg3Mjk4MTEy,1392,Update deploying.rst,80737,aslakr,closed,0,,,,,1,2021-07-11T08:43:19Z,2021-07-13T17:42:31Z,2021-07-13T17:42:27Z,CONTRIBUTOR,simonw/datasette/pulls/1392,Use same base url for Apache as in the example,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1392/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
456578474,MDU6SXNzdWU0NTY1Nzg0NzQ=,511,Get Datasette tests passing on Windows in GitHub Actions,9599,simonw,open,0,,,,,13,2019-06-15T21:41:58Z,2021-07-11T17:23:05Z,,OWNER,,"This should almost happen as a side-effect or moving from Sanic to Uvicorn during the port to ASGI: #272
Additional steps:
- test it manually
- update documentation
- set up some form of Windows CI
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/511/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
931557895,MDExOlB1bGxSZXF1ZXN0Njc5MDM1ODQ3,1386,"Update asgiref requirement from <3.4.0,>=3.2.10 to >=3.2.10,<3.5.0",49699333,dependabot[bot],closed,0,,,,,1,2021-06-28T13:13:07Z,2021-07-11T01:36:19Z,2021-07-11T01:36:18Z,CONTRIBUTOR,simonw/datasette/pulls/1386,"Updates the requirements on [asgiref](https://github.com/django/asgiref) to permit the latest version.
Changelog
Sourced from asgiref's changelog .
3.4.0 (2021-06-27)
Calling sync_to_async directly from inside itself (which causes a deadlock
when in the default, thread-sensitive mode) now has deadlock detection.
asyncio usage has been updated to use the new versions of get_event_loop,
ensure_future, wait and gather, avoiding deprecation warnings in Python 3.10.
Python 3.6 installs continue to use the old versions; this is only for 3.7+
sync_to_async and async_to_sync now have improved type hints that pass
through the underlying function type correctly.
All Websocket* types are now spelled WebSocket, to match our specs and the
official spelling. The old names will work until release 3.5.0, but will
raise deprecation warnings.
The typing for WebSocketScope and HTTPScope's extensions
key has been
fixed.
3.3.4 (2021-04-06)
The async_to_sync type error is now a warning due the high false negative
rate when trying to detect coroutine-returning callables in Python.
3.3.3 (2021-04-06)
The sync conversion functions now correctly detect functools.partial and other
wrappers around async functions on earlier Python releases.
3.3.2 (2021-04-05)
SyncToAsync now takes an optional "executor" argument if you want to supply
your own executor rather than using the built-in one.
async_to_sync and sync_to_async now check their arguments are functions of
the correct type.
Raising CancelledError inside a SyncToAsync function no longer stops a future
call from functioning.
ThreadSensitive now provides context hooks/override options so it can be
made to be sensitive in a unit smaller than threads (e.g. per request)
... (truncated)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1386/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
939051549,MDU6SXNzdWU5MzkwNTE1NDk=,1388,Serve using UNIX domain socket,80737,aslakr,closed,0,,,,,13,2021-07-07T16:13:37Z,2021-07-11T01:18:38Z,2021-07-10T23:38:32Z,CONTRIBUTOR,,Would it be possible to make datasette serve using UNIX domain socket similar to Uvicorn's ``--uds``?,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1388/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
646448486,MDExOlB1bGxSZXF1ZXN0NDQwNzM1ODE0,868,initial windows ci setup,702729,joshmgrant,open,0,,,,,3,2020-06-26T18:49:13Z,2021-07-10T23:41:43Z,,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/868,Picking up the work done on #557 with a new PR. Seeing if I can get this working.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/868/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
466996584,MDExOlB1bGxSZXF1ZXN0Mjk2NzM1MzIw,557,Get tests running on Windows using Travis CI,9599,simonw,closed,0,,,,,4,2019-07-11T16:36:57Z,2021-07-10T23:39:48Z,2021-07-10T23:39:48Z,OWNER,simonw/datasette/pulls/557,Refs #511,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/557/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
941300946,MDU6SXNzdWU5NDEzMDA5NDY=,1391,Stop using generated columns in fixtures.db,9599,simonw,closed,0,,,,,5,2021-07-10T18:26:11Z,2021-07-10T19:26:58Z,2021-07-10T19:26:00Z,OWNER,,"Refs #1376 - but I also keep running into this myself, where I try to run something against `fixtures.db` and get this confusing error:
sqlite3.DatabaseError: malformed database schema (generated_columns) - near ""AS"": syntax error
I'm going to stop using generated columns in `fixtures.db` and instead dynamically generate the generated column table for the duration of the relevant test.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1391/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
940077168,MDU6SXNzdWU5NDAwNzcxNjg=,1389,"""searchmode"": ""raw"" in table metadata",9599,simonw,closed,0,,,,,6,2021-07-08T17:32:10Z,2021-07-10T18:33:13Z,2021-07-10T18:33:13Z,OWNER,,"> http://localhost:8001/index/summary?_search=language%3Aeng&_sort=title&_searchmode=raw
>
> But I'm not able to manage it in the metadata file. Here is mine (note that the sort column is taken into account)
> Here it is:
>
> ```
> {
> ""databases"": {
> ""index"": {
> ""tables"": {
> ""summary"": {
> ""sort"": ""title"",
> ""searchmode"": ""raw""
> }
> }
> }
> }
> }
_Originally posted by @Krazybug in https://github.com/simonw/datasette/issues/759#issuecomment-624860451_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1389/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
940891698,MDU6SXNzdWU5NDA4OTE2OTg=,1390,Mention restarting systemd in documentation,9599,simonw,closed,0,,,,,2,2021-07-09T16:05:15Z,2021-07-09T16:32:57Z,2021-07-09T16:32:33Z,OWNER,,"https://docs.datasette.io/en/stable/deploying.html#running-datasette-using-systemd
Need to clarify that if you add a new database or change metadata you need to restart systemd.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1390/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
935930820,MDU6SXNzdWU5MzU5MzA4MjA=,1387,absolute_url() behind a proxy assembles incorrect http://127.0.0.1:8001/ URLs,9599,simonw,closed,0,,,,,8,2021-07-02T16:58:25Z,2021-07-02T17:58:23Z,2021-07-02T17:33:05Z,OWNER,,"Reported in the wild on https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-3d4a4b7/bib?_facet=bib_level_callnumber - the ""next page"" link links to https://127.0.0.1:8010/collection-analysis/current_collection-3d4a4b7/bib?_facet=bib_level_callnumber&_next=100
That installation uses `""base_url"": ""/collection-analysis/""`
Weirdly all of the other links on that page - to facet results, sort orders, row permalinks etc - work fine. It's JUST the `next_url` one that is broken.
Also broken in their JSON: https://ilsweb.cincinnatilibrary.org/collection-analysis/current_collection-3d4a4b7/bib.json?_size=1 returns
```json
""suggested_facets"": [],
""next"": ""1"",
""next_url"": ""https://127.0.0.1:8010/collection-analysis/current_collection-3d4a4b7/bib.json?_size=1&_next=1"",
""private"": false,
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1387/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
913865304,MDExOlB1bGxSZXF1ZXN0NjYzODM2OTY1,1368,DRAFT: A new plugin hook for dynamic metadata,2670795,brandonrobertz,closed,0,,,,,5,2021-06-07T18:56:00Z,2021-06-26T22:24:54Z,2021-06-26T22:24:54Z,CONTRIBUTOR,simonw/datasette/pulls/1368,"Note that this is a WORK IN PROGRESS!
This PR adds the following plugin hook:
get_metadata(
datasette=self, key=key, database=database, table=table,
fallback=fallback
)
This gets called when we're building our metdata for the rest of the system to use. Datasette merges whatever the plugins return with any local metadata (from metadata.yml/yaml/json) allowing for a live-editable dynamic Datasette. __A major design consideration is this: should Datasette perform the metadata merge? Or should Datasette allow plugins to perform any modifications themselves?__
As a security precation, local meta is *not* overwritable by plugin hooks. The workflow for transitioning to live-meta would be to load the plugin with the full metadata.yaml and save. Then remove the parts of the metadata that you want to be able to change from the file.
I have a WIP dynamic configuration plugin here, for reference: https://github.com/next-LI/datasette-live-config/",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1368/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
920884085,MDU6SXNzdWU5MjA4ODQwODU=,1377,Mechanism for plugins to exclude certain paths from CSRF checks,9599,simonw,closed,0,,,,,3,2021-06-15T00:48:20Z,2021-06-23T22:51:33Z,2021-06-23T22:51:33Z,OWNER,,I need this for a plugin I'm building that offers a POST API.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1377/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
913017577,MDU6SXNzdWU5MTMwMTc1Nzc=,1365,pathlib.Path breaks internal schema,25778,eyeseast,closed,0,,,,,1,2021-06-07T01:40:37Z,2021-06-21T15:57:39Z,2021-06-21T15:57:39Z,CONTRIBUTOR,,"Ran into an issue while trying to build a plugin to render GeoJSON. I'm using pytest's `tmp_path` fixture, which is a `pathlib.Path`, to get a temporary database path. I was getting a weird error involving writes, but I was doing reads. Turns out it's the internal database trying to insert a `Path` where it wants a string.
My test looked like this:
```python
@pytest.mark.asyncio
async def test_render_feature_collection(tmp_path):
database = tmp_path / ""test.db""
datasette = Datasette([database])
# this will break with a path
await datasette.refresh_schemas()
# build a url
url = datasette.urls.table(database.stem, TABLE_NAME, format=""geojson"")
response = await datasette.client.get(url)
fc = response.json()
assert 200 == response.status_code
```
I only ran into this while running tests, because passing in database paths from the CLI uses strings, but it's a weird error and probably something other people have run into.
The fix is easy enough: Convert the path to a string and everything works. So this:
```python
@pytest.mark.asyncio
async def test_render_feature_collection(tmp_path):
database = tmp_path / ""test.db""
datasette = Datasette([str(database)])
# this is fine now
await datasette.refresh_schemas()
```
This could (probably, haven't tested) be fixed [here](https://github.com/simonw/datasette/blob/03ec71193b9545536898a4bc7493274fec48bdd7/datasette/app.py#L357) by calling `str(db.path)` or by doing that conversion earlier.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1365/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
914130834,MDExOlB1bGxSZXF1ZXN0NjY0MDcyMDQ2,1370,Ensure db.path is a string before trying to insert into internal database,25778,eyeseast,closed,0,,,,,2,2021-06-08T01:16:48Z,2021-06-21T15:57:39Z,2021-06-21T15:57:39Z,CONTRIBUTOR,simonw/datasette/pulls/1370,"Fixes #1365
This is the simplest possible fix, with a test that will fail without it. There are a bunch of places where `db.path` is getting converted to and from a `Path` type, so this fix errs on the side of calling `str(db.path)` right before it's inserted.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1370/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
925491857,MDU6SXNzdWU5MjU0OTE4NTc=,1383,Improve test coverage for `inspect.py`,9599,simonw,open,0,,,,,0,2021-06-20T00:22:43Z,2021-06-20T00:22:49Z,,OWNER,,https://codecov.io/gh/simonw/datasette/src/main/datasette/inspect.py shows only 36% coverage for that module at the moment.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1383/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
925406964,MDU6SXNzdWU5MjU0MDY5NjQ=,1382,Datasette with Glitch - is it possible to use CSV with ISO-8859-1 encoding?,23701514,reichaves,closed,0,,,,,1,2021-06-19T14:37:20Z,2021-06-20T00:21:02Z,2021-06-20T00:20:06Z,NONE,,"Hi
Please, I used Remix on Glitch to create a project on Glitch and uploaded a CSV
But it's a CSV with ISO-8859-1 encoding (https://en.wikipedia.org/wiki/ISO/IEC_8859-1)
Is it possible for me to change the encoding to correctly visualize the data?
Example: https://emphasized-carpal-pillow.glitch.me/data/Emendas
Best",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1382/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
923910375,MDExOlB1bGxSZXF1ZXN0NjcyNjIwMTgw,1378,"Update pytest-xdist requirement from <2.3,>=2.2.1 to >=2.2.1,<2.4",49699333,dependabot[bot],closed,0,,,,,1,2021-06-17T13:11:56Z,2021-06-20T00:17:07Z,2021-06-20T00:17:06Z,CONTRIBUTOR,simonw/datasette/pulls/1378,"Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version.
Changelog
Sourced from pytest-xdist's changelog .
pytest-xdist 2.3.0 (2021-06-16)
Deprecations and Removals
[#654](https://github.com/pytest-dev/pytest-xdist/issues/654) <https://github.com/pytest-dev/pytest-xdist/issues/654>
_: Python 3.5 is no longer supported.
Features
[#646](https://github.com/pytest-dev/pytest-xdist/issues/646) <https://github.com/pytest-dev/pytest-xdist/issues/646>
_: Add --numprocesses=logical
flag, which automatically uses the number of logical CPUs available, instead of physical CPUs with auto
.
This is very useful for test suites which are not CPU-bound.
[#650](https://github.com/pytest-dev/pytest-xdist/issues/650) <https://github.com/pytest-dev/pytest-xdist/issues/650>
_: Added new pytest_handlecrashitem
hook to allow handling and rescheduling crashed items.
Bug Fixes
[#421](https://github.com/pytest-dev/pytest-xdist/issues/421) <https://github.com/pytest-dev/pytest-xdist/issues/421>
_: Copy the parent process sys.path into local workers, to work around execnet's python -c adding the current directory to sys.path.
[#638](https://github.com/pytest-dev/pytest-xdist/issues/638) <https://github.com/pytest-dev/pytest-xdist/issues/638>
_: Fix issue caused by changing the branch name of the pytest repository.
Trivial Changes
[#592](https://github.com/pytest-dev/pytest-xdist/issues/592) <https://github.com/pytest-dev/pytest-xdist/issues/592>
_: Replace master with controller where ever possible.
[#643](https://github.com/pytest-dev/pytest-xdist/issues/643) <https://github.com/pytest-dev/pytest-xdist/issues/643>
_: Use 'main' to refer to pytest default branch in tox env names.
pytest-xdist 2.2.1 (2021-02-09)
Bug Fixes
[#623](https://github.com/pytest-dev/pytest-xdist/issues/623) <https://github.com/pytest-dev/pytest-xdist/issues/623>
_: Gracefully handle the pending deprecation of Node.fspath by using config.rootpath for topdir.
pytest-xdist 2.2.0 (2020-12-14)
Features
... (truncated)
Commits
fe57b39
fixup: add release title underline for 2.3.0
26e7d95
prepare release 2.3.0
b02a6db
Merge pull request #667 from graingert/fix-sys-path
b072267
add newsfile
881cc48
Merge pull request #672 from pytest-dev/pre-commit-ci-update-config
958679e
[pre-commit.ci] pre-commit autoupdate
7f07d50
Merge pull request #646 from kroeschl/numprocesses-logical
fb518de
Merge pull request #669 from pytest-dev/pre-commit-ci-update-config
0b14d92
[pre-commit.ci] pre-commit autoupdate
02f971d
swap docstring
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1378/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
268176505,MDU6SXNzdWUyNjgxNzY1MDU=,34,Support CSV export with a .csv extension,9599,simonw,closed,0,,,,,1,2017-10-24T20:34:43Z,2021-06-17T18:14:48Z,2018-05-28T20:45:34Z,OWNER,,"Maybe do this using streaming with multiple pagination SQL queries so we can support arbritrarily large exports.
How would this work against a view which doesn’t have an obvious efficient pagination mechanism? Maybe limit views to up to 1000 exported records?
Relates to #5 ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/34/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
323681589,MDU6SXNzdWUzMjM2ODE1ODk=,266,Export to CSV,9599,simonw,closed,0,,,,,27,2018-05-16T15:50:24Z,2021-06-17T18:14:24Z,2018-06-18T06:05:25Z,OWNER,,Datasette needs to be able to export data to CSV.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
333000163,MDU6SXNzdWUzMzMwMDAxNjM=,312,"HTML, CSV and JSON views should support ?_col=&_col=",9599,simonw,closed,0,,,,,1,2018-06-16T16:53:35Z,2021-06-17T18:14:24Z,2018-06-16T17:00:12Z,OWNER,,To support whitelisting columns to display.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/312/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
335141434,MDU6SXNzdWUzMzUxNDE0MzQ=,326,CSV should respect --cors and return cors headers,9599,simonw,closed,0,,,,,1,2018-06-24T00:44:07Z,2021-06-17T18:14:24Z,2018-06-24T00:59:45Z,OWNER,,Otherwise tools like Vega can't load data via CSV.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/326/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
395236066,MDU6SXNzdWUzOTUyMzYwNjY=,393,"CSV export in ""Advanced export"" pane doesn't respect query",1727065,ltrgoddard,closed,0,,,,,6,2019-01-02T12:39:41Z,2021-06-17T18:14:24Z,2019-01-03T02:44:10Z,NONE,,"It looks like there's an inconsistency when exporting to CSV via the the web interface. Say I'm looking at [songs released in 1989](https://fivethirtyeight.datasettes.com/fivethirtyeight-c300360/classic-rock%2Fclassic-rock-song-list?Release+Year__exact=1989) in the `classic-rock/classic-rock-song-list` table from the Five Thirty Eight data. The JSON and CSV export links at the top of the page both give me filtered data using `Release+Year__exact=1989` in the URL. In the `Advanced export` tab, though, the CSV option gives me the whole data set, while the JSON options preserve the query.
It may be that this is intended behaviour related to the streaming CSV stuff [discussed here](https://github.com/simonw/datasette/issues/266), but if that's the case then I think it should be a little clearer.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/393/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
725184645,MDU6SXNzdWU3MjUxODQ2NDU=,1034,Better way of representing binary data in .csv output,9599,simonw,closed,0,,,6026070,0.51,19,2020-10-20T04:28:58Z,2021-06-17T18:13:21Z,2020-10-29T22:47:46Z,OWNER,,"I just noticed this: https://latest.datasette.io/fixtures/binary_data.csv
```csv
rowid,data
1,b'\x15\x1c\x02\xc7\xad\x05\xfe'
2,b'\x15\x1c\x03\xc7\xad\x05\xfe'
```
There's no good way to represent binary data in a CSV file, but this seems like one of the more-bad options.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1034/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
503190241,MDU6SXNzdWU1MDMxOTAyNDE=,584,Codec error in some CSV exports,9599,simonw,closed,0,,,,,2,2019-10-07T01:15:34Z,2021-06-17T18:13:20Z,2019-10-18T05:23:16Z,OWNER,,"Got this exploring my Swarm checkins:
![448DBFC4-71F8-4846-83C0-BEA511B2157A](https://user-images.githubusercontent.com/9599/66279259-3af53480-e865-11e9-9651-04fd2d895392.jpeg)
`/swarm/stickers.csv?stickerType=messageOnly&_size=max`",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/584/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
508100844,MDU6SXNzdWU1MDgxMDA4NDQ=,598,Character encoding bug with CSV export,46313,JoeGermuska,closed,0,,,,,1,2019-10-16T21:09:30Z,2021-06-17T18:13:20Z,2019-10-18T22:52:21Z,NONE,,"I was just poking around, and at [this URL](https://sql-murder-mystery.datasette.io/sql-murder-mystery/crime_scene_report.csv?_stream=on&type=arson&_size=max), I encountered this error:
```
'latin-1' codec can't encode character '\u2019' in position 27: ordinal not in range(256)
```
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/598/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
516748849,MDU6SXNzdWU1MTY3NDg4NDk=,612,CSV export is broken for tables with null foreign keys,9599,simonw,closed,0,,,,,2,2019-11-02T22:52:47Z,2021-06-17T18:13:20Z,2019-11-02T23:12:53Z,OWNER,,"Following on from #406 - this CSV export appears to be broken:
https://14da705.datasette.io/fixtures/foreign_key_references.csv?_labels=on&_size=max
```csv
pk,foreign_key_with_label,foreign_key_with_label_label,foreign_key_with_no_label,foreign_key_with_no_label_label
1,1,hello,1,1
2,,
```
That second row should have 5 values, but it only has 4.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/612/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
775666296,MDU6SXNzdWU3NzU2NjYyOTY=,1160,"""datasette insert"" command and plugin hook",9599,simonw,open,0,,,,,23,2020-12-29T02:37:03Z,2021-06-17T18:12:32Z,,OWNER,,"Tools for loading data into Datasette currently mostly exist as separate utilities - `yaml-to-sqlite` and `csvs-to-sqlite` and suchlike.
Bringing these into Datasette could have some interesting properties:
- A `datasette insert` command could be extended with plugins to handle more formats
- Any format that can be inserted on the command-line could also be inserted using a web UI or web API - which would benefit from new format plugin hooks
- If Datasette ever grows beyond SQLite (see #670) a built-in import mechanism could work for those other databases as well - without me needing to write `yaml-to-postgresql` and suchlike",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1160/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
776128269,MDU6SXNzdWU3NzYxMjgyNjk=,1162,"First working version of ""datasette insert data.db file.csv""",9599,simonw,open,0,,,,,0,2020-12-29T23:20:11Z,2021-06-17T18:12:32Z,,OWNER,,Refs #1160,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1162/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
776128565,MDU6SXNzdWU3NzYxMjg1NjU=,1163,"""datasette insert data.db url-to-csv""",9599,simonw,open,0,,,,,1,2020-12-29T23:21:21Z,2021-06-17T18:12:32Z,,OWNER,,"Refs #1160 - get filesystem imports working first for #1162, then add import-from-URL.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1163/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
906385991,MDU6SXNzdWU5MDYzODU5OTE=,1349,CSV ?_stream=on redundantly calculates facets for every page,9599,simonw,closed,0,,,,,9,2021-05-29T06:11:23Z,2021-06-17T18:12:32Z,2021-06-01T15:52:53Z,OWNER,,"I'm trying to figure out why a full CSV export from https://covid-19.datasettes.com/covid/ny_times_us_counties runs unbearably slowly.
It's because the streaming endpoint works by scrolling through every page, and it turns out every page calculates facets and suggested facets!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1349/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
906993731,MDU6SXNzdWU5MDY5OTM3MzE=,1351,Get `?_trace=1` working with CSV and streaming CSVs,9599,simonw,closed,0,,,,,1,2021-05-31T03:02:15Z,2021-06-17T18:12:32Z,2021-06-01T15:50:09Z,OWNER,,"> I think it's worth getting `?_trace=1` to work with streaming CSV - this would have helped me spot this issue a long time ago.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1349#issuecomment-851133125_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1351/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
736365306,MDU6SXNzdWU3MzYzNjUzMDY=,1083,Advanced CSV export for arbitrary queries,9599,simonw,open,0,,,,,2,2020-11-04T19:23:05Z,2021-06-17T18:12:31Z,,OWNER,,"There's no link to download the CSV file - the table page has that as an advanced export option, but this is missing from the query page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1083/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
743359646,MDU6SXNzdWU3NDMzNTk2NDY=,1096,TSV should be a default export option,9599,simonw,open,0,,,,,1,2020-11-15T22:24:02Z,2021-06-17T18:12:31Z,,OWNER,,Refs #1095,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1096/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
759695780,MDU6SXNzdWU3NTk2OTU3ODA=,1133,Option to omit header row in CSV export,9599,simonw,closed,0,,,,,2,2020-12-08T18:54:46Z,2021-06-17T18:12:31Z,2020-12-10T23:28:51Z,OWNER,,`?_header=off` - for symmetry with existing option `?_nl=on`.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1133/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
763361458,MDU6SXNzdWU3NjMzNjE0NTg=,1142,"""Stream all rows"" is not at all obvious",9599,simonw,open,0,,,,,9,2020-12-12T06:24:57Z,2021-06-17T18:12:31Z,,OWNER,,"Got a question about how to download all rows - the current option isn't at all clear.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1142/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
732685643,MDU6SXNzdWU3MzI2ODU2NDM=,1063,.csv should link to .blob downloads,9599,simonw,closed,0,,,6026070,0.51,3,2020-10-29T21:45:58Z,2021-06-17T18:12:30Z,2020-10-29T22:47:45Z,OWNER,,"- [x] Update `.csv` output to link to these things (and get that `xfail` test to pass)
- ~~Add a `.csv?_blob_base64=1` argument that causes them to be output in base64 in the CSV~~
> Moving the CSV work to a separate ticket.
_Originally posted by @simonw in https://github.com/simonw/datasette/pull/1061#issuecomment-719042601_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1063/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
924203783,MDU6SXNzdWU5MjQyMDM3ODM=,1379,Idea: ?_end=1 option for streaming CSV responses,9599,simonw,open,0,,,,,0,2021-06-17T18:11:21Z,2021-06-17T18:11:30Z,,OWNER,,"As discussed in this thread: https://twitter.com/simonw/status/1405554676993433605 - one of the disadvantages of Datasette's streaming CSV feature is that it's hard to tell if you got the whole file or if the connection ended early - or if an error occurred.
Idea: offer an optional `?_end=1` parameter which, if enabled, adds a single row to the end of the CSV file that looks like this:
`END,,,,,,,,,`
For however many columns the CSV file usually has.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1379/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
756876238,MDExOlB1bGxSZXF1ZXN0NTMyMzQ4OTE5,1130,Fix footer not sticking to bottom in short pages,3243482,abdusco,open,0,,,,,4,2020-12-04T07:29:01Z,2021-06-15T13:27:48Z,,CONTRIBUTOR,simonw/datasette/pulls/1130,Fixes https://github.com/simonw/datasette/issues/1129,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1130/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
919508498,MDU6SXNzdWU5MTk1MDg0OTg=,1375,JSON export dumps JSON fields as TEXT,4068,frafra,closed,0,,,,,2,2021-06-12T09:45:08Z,2021-06-14T09:41:59Z,2021-06-13T15:37:58Z,NONE,,"Hi!
When a user tries to export data as JSON, I would expect to see the value of JSON columns represented as JSON instead of being rendered as a string. What do you think?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1375/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
919822817,MDU6SXNzdWU5MTk4MjI4MTc=,1376,Official Datasette Docker image should use SQLite >= 3.31.0 (for generated columns),1726460,jcgregorio,open,0,,,,,3,2021-06-13T15:25:51Z,2021-06-13T15:39:37Z,,NONE,,"Trying to run datasette via the Docker container doesn't seem to work:
```
$ docker run -p 8001:8001 -v `pwd`:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/fixtures.db
Traceback (most recent call last):
File ""/usr/local/bin/datasette"", line 8, in
sys.exit(cli())
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 829, in __call__
return self.main(*args, **kwargs)
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 782, in main
rv = self.invoke(ctx)
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/usr/local/lib/python3.9/site-packages/click/core.py"", line 610, in invoke
return callback(*args, **kwargs)
File ""/usr/local/lib/python3.9/site-packages/datasette/cli.py"", line 544, in serve
asyncio.get_event_loop().run_until_complete(check_databases(ds))
File ""/usr/local/lib/python3.9/asyncio/base_events.py"", line 642, in run_until_complete
return future.result()
File ""/usr/local/lib/python3.9/site-packages/datasette/cli.py"", line 584, in check_databases
await database.execute_fn(check_connection)
File ""/usr/local/lib/python3.9/site-packages/datasette/database.py"", line 155, in execute_fn
return await asyncio.get_event_loop().run_in_executor(
File ""/usr/local/lib/python3.9/concurrent/futures/thread.py"", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File ""/usr/local/lib/python3.9/site-packages/datasette/database.py"", line 153, in in_thread
return fn(conn)
File ""/usr/local/lib/python3.9/site-packages/datasette/utils/__init__.py"", line 892, in check_connection
for r in conn.execute(
sqlite3.DatabaseError: malformed database schema (generated_columns) - near ""AS"": syntax error
```
I have confirmed that the downloaded `fixtures.db` database is fine:
```
[skia-public] jcgregorio@jcgregorio840 ~/Downloads
$ sqlite3 fixtures.db
SQLite version 3.34.1 2021-01-20 14:10:07
Enter "".help"" for usage hints.
sqlite> pragma integrity_check;
ok
sqlite>
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1376/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
916183914,MDExOlB1bGxSZXF1ZXN0NjY1ODkyMzEz,1373,"Update trustme requirement from <0.8,>=0.7 to >=0.7,<0.9",49699333,dependabot[bot],closed,0,,,,,1,2021-06-09T13:09:44Z,2021-06-13T15:38:47Z,2021-06-13T15:38:47Z,CONTRIBUTOR,simonw/datasette/pulls/1373,"Updates the requirements on [trustme](https://github.com/python-trio/trustme) to permit the latest version.
Commits
f9e13e0
Release 0.8.0
4ae4435
Merge pull request #304 from python-trio/dependabot/add-v2-config-file
8767902
Merge pull request #327 from graingert/test-on-py310
6abfddd
Merge branch 'master' of github.com:python-trio/trustme into test-on-py310
51d3bdf
Merge pull request #328 from tiran/correct_ku_eku
034fb3a
retry codecov more
53e121d
try codecov harder
c1e7923
require codecov in ci
e3ac2d6
Update tests/test_trustme.py
496dca6
close the wrapped sockets to prevent Unraisable ResourceWarnings
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1373/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
918730335,MDExOlB1bGxSZXF1ZXN0NjY4MTI5NDQx,1374,Bump black from 21.5b2 to 21.6b0,49699333,dependabot[bot],closed,0,,,,,1,2021-06-11T13:07:39Z,2021-06-13T15:33:23Z,2021-06-13T15:33:22Z,CONTRIBUTOR,simonw/datasette/pulls/1374,"Bumps [black](https://github.com/psf/black) from 21.5b2 to 21.6b0.
Release notes
Sourced from black's releases .
21.6b0
Black
Fix failure caused by fmt: skip
and indentation (#2281 )
Account for += assignment when deciding whether to split string (#2312 )
Correct max string length calculation when there are string operators (#2292 )
Fixed option usage when using the --code
flag (#2259 )
Do not call uvloop.install()
when Black is used as a library (#2303 )
Added --required-version
option to require a specific version to be running (#2300 )
Fix incorrect custom breakpoint indices when string group contains fake f-strings
(#2311 )
Fix regression where R
prefixes would be lowercased for docstrings (#2285 )
Fix handling of named escapes (\N{...}
) when --experimental-string-processing
is
used (#2319 )
Changelog
Sourced from black's changelog .
21.6b0
Black
Fix failure caused by fmt: skip
and indentation (#2281 )
Account for += assignment when deciding whether to split string (#2312 )
Correct max string length calculation when there are string operators (#2292 )
Fixed option usage when using the --code
flag (#2259 )
Do not call uvloop.install()
when Black is used as a library (#2303 )
Added --required-version
option to require a specific version to be running (#2300 )
Fix incorrect custom breakpoint indices when string group contains fake f-strings
(#2311 )
Fix regression where R
prefixes would be lowercased for docstrings (#2285 )
Fix handling of named escapes (\N{...}
) when --experimental-string-processing
is
used (#2319 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.5b2&new-version=21.6b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1374/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
849220154,MDU6SXNzdWU4NDkyMjAxNTQ=,1286,Better default display of arrays of items,192568,mroswell,open,0,,,,,5,2021-04-02T13:31:40Z,2021-06-12T12:36:15Z,,CONTRIBUTOR,,"Would be great to have template filters that convert array fields to bullets and/or delimited lists upon table display:
```
|to_bullets
|to_comma_delimited
|to_semicolon_delimited
```
or maybe:
```
|join_array(""bullet"")
|join_array(""bullet"",""square"")
|join_array("";"")
|join_array("","")
```
Keeping in mind that bullets show up in html as \ while other delimiting characters appear after the value.
Of course, the fields themselves would remain as facetable arrays. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1286/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
642388564,MDU6SXNzdWU2NDIzODg1NjQ=,858,publish heroku does not work on Windows 10,870912,simonlau,open,0,,,,,7,2020-06-20T14:40:28Z,2021-06-10T17:44:09Z,,NONE,,"When executing ""datasette publish heroku schools.db"" on Windows 10, I get the following error
```shell
File ""c:\users\dell\.virtualenvs\sec-schools-jn-cwk8z\lib\site-packages\datasette\publish\heroku.py"", line 54, in heroku
line.split()[0] for line in check_output([""heroku"", ""plugins""]).splitlines()
File ""c:\python38\lib\subprocess.py"", line 411, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File ""c:\python38\lib\subprocess.py"", line 489, in run
with Popen(*popenargs, **kwargs) as process:
File ""c:\python38\lib\subprocess.py"", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File ""c:\python38\lib\subprocess.py"", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
```
Changing https://github.com/simonw/datasette/blob/55a6ffb93c57680e71a070416baae1129a0243b8/datasette/publish/heroku.py#L54
to
```python
line.split()[0] for line in check_output([""heroku"", ""plugins""], shell=True).splitlines()
```
as well as the other `check_output()` and `call()` within the same file leads me to another recursive error about temp files",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/858/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
915455228,MDU6SXNzdWU5MTU0NTUyMjg=,1371,Menu plugin hooks should include the request,9599,simonw,closed,0,,,,,1,2021-06-08T20:23:35Z,2021-06-10T04:46:01Z,2021-06-10T04:46:01Z,OWNER,,"https://docs.datasette.io/en/stable/plugin_hooks.html#menu-links-datasette-actor
- `menu_links(datasette, actor)`
- `table_actions(datasette, actor, database, table)`
- `database_actions(datasette, actor, database)`
All three of these should optionally also accept the `request` object. This would allow them to take into account additional cookies, `Authorization` headers or the current request URL (including the domain/subdomain) - or even access `request.scope` for extra context that might have been passed down from ASGI middleware.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1371/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
915488244,MDU6SXNzdWU5MTU0ODgyNDQ=,1372,"Add section to ""writing plugins"" about security, e.g. avoiding XSS",9599,simonw,open,0,,,,,0,2021-06-08T20:49:33Z,2021-06-08T20:49:46Z,,OWNER,,https://docs.datasette.io/en/stable/writing_plugins.html should have tips on writing secure plugins.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1372/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
913900374,MDU6SXNzdWU5MTM5MDAzNzQ=,1369,Don't show foreign key IDs twice if no label,9599,simonw,open,0,,,,,1,2021-06-07T19:47:02Z,2021-06-07T19:47:24Z,,OWNER,,"![B5B54D94-A768-4544-A88D-CDCAB417CD3C](https://user-images.githubusercontent.com/9599/121078979-6e9d0600-c78e-11eb-8b70-20e6d29b48b1.jpeg)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1369/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
913823889,MDU6SXNzdWU5MTM4MjM4ODk=,1367,Navigation menu display bug,9599,simonw,closed,0,,,,,1,2021-06-07T18:18:08Z,2021-06-07T18:24:19Z,2021-06-07T18:24:19Z,OWNER,,"With Datasette 0.57 the navigation menu looks like this:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1367/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
913809802,MDU6SXNzdWU5MTM4MDk4MDI=,1366,Get rid of this `restore_working_directory` hack entirely,9599,simonw,open,0,,,,,2,2021-06-07T18:01:21Z,2021-06-07T18:03:03Z,,OWNER,,"> That seems to have fixed it. I'd love to get rid of this `restore_working_directory` hack entirely.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1361#issuecomment-855308811_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1366/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
912959264,MDU6SXNzdWU5MTI5NTkyNjQ=,1364,Don't truncate columns on the list of databases,9599,simonw,closed,0,,,,,0,2021-06-06T22:01:56Z,2021-06-06T22:07:50Z,2021-06-06T22:07:50Z,OWNER,,"https://covid-19.datasettes.com/covid currently truncates at 9 database columns:
Django SQL Dashboard showed me that this is a bad idea - having the full list of columns is actually really useful documentation for crafting custom SQL queries.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1364/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
325958506,MDU6SXNzdWUzMjU5NTg1MDY=,283,Support cross-database joins,9599,simonw,closed,0,,,,,26,2018-05-24T04:18:39Z,2021-06-06T09:40:18Z,2021-02-18T22:16:46Z,OWNER,,"SQLite has the ability to attach multiple databases to a single connection and then run joins across multiple databases.
Since Datasette supports more than one database, this would make a pretty neat feature.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/283/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
912485040,MDU6SXNzdWU5MTI0ODUwNDA=,1361,Intermittent CI failure: restore_working_directory FileNotFoundError,9599,simonw,closed,0,,,,,4,2021-06-05T22:48:13Z,2021-06-05T23:16:24Z,2021-06-05T23:16:24Z,OWNER,,"e.g. in https://github.com/simonw/datasette/runs/2754772233 - this is an intermittent error:
```
__________ ERROR at setup of test_hook_register_routes_render_message __________
[gw0] linux -- Python 3.8.10 /opt/hostedtoolcache/Python/3.8.10/x64/bin/python
tmpdir = local('/tmp/pytest-of-runner/pytest-0/popen-gw0/test_hook_register_routes_rend0')
request = >
@pytest.fixture
def restore_working_directory(tmpdir, request):
> previous_cwd = os.getcwd()
E FileNotFoundError: [Errno 2] No such file or directory
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1361/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
912464443,MDU6SXNzdWU5MTI0NjQ0NDM=,1360,"Security flaw, to be fixed in 0.56.1 and 0.57",9599,simonw,closed,0,,,,,2,2021-06-05T21:53:51Z,2021-06-05T22:23:23Z,2021-06-05T22:22:06Z,OWNER,,"See security advisory here for details: https://github.com/simonw/datasette/security/advisories/GHSA-xw7c-jx9m-xh5g - the `?_trace=1` debugging option was not correctly escaping its JSON output, resulting in a [reflected cross-site scripting](https://owasp.org/www-community/attacks/xss/#reflected-xss-attacks) vulnerability.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1360/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
912418094,MDU6SXNzdWU5MTI0MTgwOTQ=,1358,Release Datasette 0.57,9599,simonw,closed,0,,,,,3,2021-06-05T19:56:13Z,2021-06-05T22:20:07Z,2021-06-05T22:20:07Z,OWNER,,"Need release notes. Changes are here: https://github.com/simonw/datasette/compare/0.56...368aa5f1b16ca35f82d90ff747023b9a2bfa27c1
Partial release notes already exist for the two alphas, https://github.com/simonw/datasette/releases/tag/0.57a0 and https://github.com/simonw/datasette/releases/tag/0.57a1",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1358/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
912419349,MDU6SXNzdWU5MTI0MTkzNDk=,1359,`?_trace=1` should only be available with a new `trace_debug` setting,9599,simonw,closed,0,,,,,0,2021-06-05T19:59:27Z,2021-06-05T20:18:46Z,2021-06-05T20:18:46Z,OWNER,,Just like template debug mode is controlled by this off-by-default setting: https://github.com/simonw/datasette/blob/368aa5f1b16ca35f82d90ff747023b9a2bfa27c1/datasette/app.py#L160-L164,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1359/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
849582643,MDExOlB1bGxSZXF1ZXN0NjA4MzM0MDk2,1291,Update docs: explain allow_download setting,5413548,louispotok,closed,0,,,,,2,2021-04-03T05:28:33Z,2021-06-05T19:48:51Z,2021-06-05T19:48:51Z,CONTRIBUTOR,simonw/datasette/pulls/1291,"This fixes one possible source of confusion seen in #502 and clarifies
when database downloads will be shown and allowed.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1291/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
813899472,MDU6SXNzdWU4MTM4OTk0NzI=,1238,Custom pages don't work with base_url setting,79913,tsibley,closed,0,,,,,9,2021-02-22T21:58:58Z,2021-06-05T18:59:55Z,2021-06-05T18:59:55Z,NONE,,"It seems that custom pages aren't routing properly when the `base_url` setting is used.
To reproduce, with Datasette 0.55.
Create a `templates/pages/custom.html` with some text.
```
mkdir -p templates/pages/
echo ""Hello, world!"" > templates/pages/custom.html
```
Start Datasette.
```
datasette --template-dir templates/
```
Visit http://localhost:8001/custom and see ""Hello, world!"".
Start Datasette with a `base_url`.
```
datasette --template-dir templates/ --setting base_url /prefix/
```
Visit http://localhost:8001/prefix/custom and see a ""Database not found: custom"" 404.
Note that like all routes, http://localhost:8001/custom still works when run with `base_url`.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1238/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
912394511,MDExOlB1bGxSZXF1ZXN0NjYyNTU3MjQw,1357,Make custom pages compatible with base_url setting,9599,simonw,closed,0,,,,,1,2021-06-05T18:54:39Z,2021-06-05T18:59:54Z,2021-06-05T18:59:54Z,OWNER,simonw/datasette/pulls/1357,Refs #1238.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1357/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
656959584,MDU6SXNzdWU2NTY5NTk1ODQ=,893,pip3 install datasette not serving static on linuxbrew.,44167,zodman,closed,0,,,,,1,2020-07-14T23:33:38Z,2021-06-02T04:29:56Z,2021-06-02T04:29:56Z,NONE,,"*This error wasn't thrown*
```
Traceback (most recent call last):
File ""/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 289, in inner_static
full_path.relative_to(root_path)
File ""/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/pathlib.py"", line 904, in relative_to
raise ValueError(""{!r} does not start with {!r}""
ValueError: '/home/linuxbrew/.linuxbrew/lib/python3.8/site-packages/datasette/static/app.css' does not start with '/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages/datasette/static'
```
Linuxbrew install python@3.8 with symbolic links when You call the full_path.relative_to(root_path) throw ValueError. This happened when you install from pip3
when you install with python3 setup.py develop , works good.
Well at the end the static wasn't serving.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/893/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
756818250,MDU6SXNzdWU3NTY4MTgyNTA=,1127,Make the custom SQL query text box larger or resizable,596279,zaneselvans,closed,0,,,,,1,2020-12-04T05:37:11Z,2021-06-02T04:29:06Z,2021-06-02T04:28:55Z,NONE,,"The text entry field for custom SQL queries is too small to display a moderately complex query, especially when it's been formatted. Would it be easy to make the textbox resizable by the user rather than having a fixed height?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1127/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
864979486,MDExOlB1bGxSZXF1ZXN0NjIxMTE3OTc4,1306,Avoid error sorting by relationships if related tables are not allowed,416374,gfrmin,closed,0,,,,,4,2021-04-22T13:53:17Z,2021-06-02T04:27:00Z,2021-06-02T04:25:28Z,CONTRIBUTOR,simonw/datasette/pulls/1306,Refs #1305,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1306/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
864969683,MDU6SXNzdWU4NjQ5Njk2ODM=,1305,Index view crashes when any database table is not accessible to actor,416374,gfrmin,closed,0,,,,,0,2021-04-22T13:44:22Z,2021-06-02T04:26:29Z,2021-06-02T04:26:29Z,CONTRIBUTOR,,"Because of https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L63, the ```tables``` dict built does not include invisible tables; however, if https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L80 is reached (because table_counts was not successfully initialized, e.g. due to a very large database) then as db.get_all_foreign_keys() returns ALL tables, a KeyError will be raised.
This error can be recreated with the fixtures.db if any table is hidden, e.g. by adding something like ```""foreign_key_references"": {
""allow"": {}
}``` to fixtures-metadata.json and deleting ```or not table_counts``` from https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L77.
I'm not sure how to fix this error; perhaps by testing if the table is in the aforementions ```tables``` dict.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1305/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
904537568,MDExOlB1bGxSZXF1ZXN0NjU1Njg0NDc3,1346,Re-display user's query with an error message if an error occurs,9599,simonw,closed,0,,,,,3,2021-05-28T02:04:20Z,2021-06-02T03:46:21Z,2021-06-02T03:46:21Z,OWNER,simonw/datasette/pulls/1346,Refs #619,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1346/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
828811618,MDU6SXNzdWU4Mjg4MTE2MTg=,1257,Table names containing single quotes break things,9599,simonw,closed,0,,,,,2,2021-03-11T06:29:38Z,2021-06-02T03:28:29Z,2021-06-02T03:28:29Z,OWNER,,"e.g. I found a table called `Yesterday's ELRs by County`
It threw an error inside the `detect_fts()` function attempting to run this SQL query:
```sql
select name from sqlite_master
where rootpage = 0
and (
sql like '%VIRTUAL TABLE%USING FTS%content=""Yesterday's ELRs by County""%'
or sql like '%VIRTUAL TABLE%USING FTS%content=[Yesterday's ELRs by County]%'
or (
tbl_name = ""Yesterday's ELRs by County""
and sql like '%VIRTUAL TABLE%USING FTS%'
)
)
```
Here's the code at fault: https://github.com/simonw/datasette/blob/640ac7071b73111ba4423812cd683756e0e1936b/datasette/utils/__init__.py#L534-L548",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1257/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
800669347,MDU6SXNzdWU4MDA2NjkzNDc=,1216,"/-/databases should reflect connection order, not alphabetical order",9599,simonw,closed,0,,,,,1,2021-02-03T20:20:23Z,2021-06-02T03:10:19Z,2021-06-02T03:10:19Z,OWNER,,"The order in which databases are attached to Datasette matters - it affects the homepage, and it's beginning to influence how certain plugins work (see https://github.com/simonw/datasette-tiles/issues/8).
Two years ago in cccea85be6aaaeadb31f3b588ec7f732628815f5 I made `/-/databases` return things in alphabetical order, to fix a test failure in Python 3.5.
Python 3.5 is no longer supported, so this is no longer necessary - and this behaviour should now be treated as a bug.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1216/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
908276134,MDExOlB1bGxSZXF1ZXN0NjU4OTkxNDA0,1352,Bump black from 21.5b1 to 21.5b2,49699333,dependabot[bot],closed,0,,,,,1,2021-06-01T13:08:52Z,2021-06-02T02:56:45Z,2021-06-02T02:56:44Z,CONTRIBUTOR,simonw/datasette/pulls/1352,"Bumps [black](https://github.com/psf/black) from 21.5b1 to 21.5b2.
Release notes
Sourced from black's releases .
21.5b2
Black
A space is no longer inserted into empty docstrings (#2249 )
Fix handling of .gitignore files containing non-ASCII characters on Windows (#2229 )
Respect .gitignore
files in all levels, not only root/.gitignore
file (apply
.gitignore
rules like git
does) (#2225 )
Restored compatibility with Click 8.0 on Python 3.6 when LANG=C used (#2227 )
Add extra uvloop install + import support if in python env (#2258 )
Fix --experimental-string-processing crash when matching parens are not found (#2283 )
Make sure to split lines that start with a string operator (#2286 )
Fix regular expression that black uses to identify f-expressions (#2287 )
Blackd
Add a lower bound for the aiohttp-cors
dependency. Only 0.4.0 or higher is
supported. (#2231 )
Packaging
Release self-contained x86_64 MacOS binaries as part of the GitHub release pipeline
(#2198 )
Always build binaries with the latest available Python (#2260 )
Documentation
Add discussion of magic comments to FAQ page (#2272 )
--experimental-string-processing
will be enabled by default in the future (#2273 )
Fix typos discovered by codespell (#2228 )
Fix Vim plugin installation instructions. (#2235 )
Add new Frequently Asked Questions page (#2247 )
Fix encoding + symlink issues preventing proper build on Windows (#2262 )
Changelog
Sourced from black's changelog .
21.5b2
Black
A space is no longer inserted into empty docstrings (#2249 )
Fix handling of .gitignore files containing non-ASCII characters on Windows (#2229 )
Respect .gitignore
files in all levels, not only root/.gitignore
file (apply
.gitignore
rules like git
does) (#2225 )
Restored compatibility with Click 8.0 on Python 3.6 when LANG=C used (#2227 )
Add extra uvloop install + import support if in python env (#2258 )
Fix --experimental-string-processing crash when matching parens are not found (#2283 )
Make sure to split lines that start with a string operator (#2286 )
Fix regular expression that black uses to identify f-expressions (#2287 )
Blackd
Add a lower bound for the aiohttp-cors
dependency. Only 0.4.0 or higher is
supported. (#2231 )
Integrations
The official Black action now supports choosing what version to use, and supports the
major 3 OSes. (#1940 )
Packaging
Release self-contained x86_64 MacOS binaries as part of the GitHub release pipeline
(#2198 )
Always build binaries with the latest available Python (#2260 )
Documentation
Add discussion of magic comments to FAQ page (#2272 )
--experimental-string-processing
will be enabled by default in the future (#2273 )
Fix typos discovered by codespell (#2228 )
Fix Vim plugin installation instructions. (#2235 )
Add new Frequently Asked Questions page (#2247 )
Fix encoding + symlink issues preventing proper build on Windows (#2262 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.5b1&new-version=21.5b2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1352/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
323671577,MDU6SXNzdWUzMjM2NzE1Nzc=,263,Facets should not execute for ?shape=array|object,9599,simonw,closed,0,,,,,3,2018-05-16T15:26:13Z,2021-06-02T02:54:34Z,2021-06-02T02:54:34Z,OWNER,,Split off from #255 - there's no point executing the facet SQL for the `?_shape=array` and `?_shape=object` API responses.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/263/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
906977719,MDU6SXNzdWU5MDY5Nzc3MTk=,1350,?_nofacets=1 query string argument for disabling facets and suggested facets,9599,simonw,closed,0,,,,,2,2021-05-31T02:22:29Z,2021-06-01T16:19:38Z,2021-05-31T02:39:18Z,OWNER,,"This is needed as an internal option for #1349. `datasette-graphql` can benefit from this too - maybe can even use it so that if you pass `?_shape=array` it gets automatically added, fixing #263.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1350/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
908446997,MDU6SXNzdWU5MDg0NDY5OTc=,1353,?_nocount=1 for opting out of table counts,9599,simonw,closed,0,,,,,2,2021-06-01T15:53:27Z,2021-06-01T16:18:54Z,2021-06-01T16:17:04Z,OWNER,,"Running a trace against a CSV streaming export with the new `_trace=1` feature from #1351 shows that the following code is executing a `select count(*) from table` for every page of results returned: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/table.py#L700-L705
This is inefficient - a new `?_nocount=1` option would let us disable this count in the same way as #1349: https://github.com/simonw/datasette/blob/d1d06ace49606da790a765689b4fbffa4c6deecb/datasette/views/base.py#L264-L276
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1353/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
908465747,MDU6SXNzdWU5MDg0NjU3NDc=,1354,Update help in tests for latest Click,9599,simonw,closed,0,,,,,1,2021-06-01T16:14:31Z,2021-06-01T16:17:04Z,2021-06-01T16:17:04Z,OWNER,,"Now that Uvicorn 0.14 is out with an unpinned Click dependency - https://github.com/encode/uvicorn/pull/1033 - our test suite runs against Click 8.0 - which subtly changes the output of `--help` causing test failures: https://github.com/simonw/datasette/runs/2720383031?check_suite_focus=true
```
def test_help_includes(name, filename):
expected = (docs_path / filename).read_text()
runner = CliRunner()
result = runner.invoke(cli, name.split() + [""--help""], terminal_width=88)
actual = f""$ datasette {name} --help\n\n{result.output}""
# actual has ""Usage: cli package [OPTIONS] FILES""
# because it doesn't know that cli will be aliased to datasette
expected = expected.replace(""Usage: datasette"", ""Usage: cli"")
> assert expected == actual
E AssertionError: assert '$ datasette ...e and exit.\n' == '$ datasette ...e and exit.\n'
E Skipping 848 identical leading characters in diff, use -v to show
E nt_id xxx
E +
E --version-note TEXT Additional note to show on /-/versions
E --secret TEXT Secret used for signing secure values, such as signed
E cookies
E +
E --title TEXT Title for metadata
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1354/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
904071938,MDU6SXNzdWU5MDQwNzE5Mzg=,1345,?_nocol= does not interact well with default facets,9599,simonw,closed,0,,,,,7,2021-05-27T18:39:55Z,2021-05-31T02:40:44Z,2021-05-31T02:31:21Z,OWNER,,"Clicking ""Hide this column"" on `fips` on https://covid-19.datasettes.com/covid/ny_times_us_counties shows this error:
https://covid-19.datasettes.com/covid/ny_times_us_counties?_nocol=fips
> ## Invalid SQL
> no such column: fips
The reason is that https://covid-19.datasettes.com/-/metadata sets up the following:
```json
""ny_times_us_counties"": {
""sort_desc"": ""date"",
""facets"": [
""state"",
""county"",
""fips""
],
```
It's setting `fips` as a default facet, which breaks if you attempt to remove the column using `?_nocol`.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1345/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
904598267,MDExOlB1bGxSZXF1ZXN0NjU1NzQxNDI4,1348,DRAFT: add test and scan for docker images,10801138,blairdrummond,open,0,,,,,2,2021-05-28T03:02:12Z,2021-05-28T03:06:16Z,,CONTRIBUTOR,simonw/datasette/pulls/1348,"**NOTE: I don't think this PR is ready, since the arm/v6 and arm/v7 images are failing pytest due to missing dependencies (gcc and friends). But it's pretty close.**
Closes https://github.com/simonw/datasette/issues/1344 . Using a build-matrix for the platforms and [this test](https://github.com/simonw/datasette/issues/1344#issuecomment-849820019), we test all the platforms in parallel. I also threw in container scanning.
### Switch `pip install` to use either tags or commit shas
Notably! This also [changes the Dockerfile](https://github.com/blairdrummond/datasette/blob/7fe5315d68e04fce64b5bebf4e2d7feec44f8546/Dockerfile#L20) so that it accepts tags or commit-shas.
```
# It's backwards compatible with tags, but also lets you use shas
root@712071df17af:/# pip install git+git://github.com/simonw/datasette.git@0.56
Collecting git+git://github.com/simonw/datasette.git@0.56
Cloning git://github.com/simonw/datasette.git (to revision 0.56) to /tmp/pip-req-build-u6dhm945
Running command git clone -q git://github.com/simonw/datasette.git /tmp/pip-req-build-u6dhm945
Running command git checkout -q af5a7f1c09f6a902bb2a25e8edf39c7034d2e5de
Collecting Jinja2<2.12.0,>=2.10.3
Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
```
This lets you build the containers in CI every push for testing, which maybe resolves [this problem](https://github.com/simonw/datasette/issues/1272#issuecomment-808648974)?
# Workflow run example
You can see the results in my workflow [here](https://github.com/blairdrummond/datasette/pull/2/checks?check_run_id=2690570717). The commit history is different because I squashed this branch, also in the testing branch I had to change `github.com/simonw` to `github.com/blairdrummond` for the CI to pick up my git_sha.
## Why did the builds fail?
**NOTE:** The results of all the tests fail, but for different reasons! A few fail to install Rust, the amd64 passes the tests (phew!) but has critical CVEs which fail the container scan, the Arm/v6 and Arm/v7 seem to fail to install the test dependencies due to missing programs like `gcc`. (`gcc` is not sufficient though, as [this run](https://github.com/blairdrummond/datasette/pull/3/checks?check_run_id=2690672982) indicates) ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1348/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
904582277,MDExOlB1bGxSZXF1ZXN0NjU1NzI2Mzg3,1347,Test docker platform blair only,10801138,blairdrummond,closed,0,,,,,0,2021-05-28T02:47:09Z,2021-05-28T02:47:28Z,2021-05-28T02:47:28Z,CONTRIBUTOR,simonw/datasette/pulls/1347,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1347/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
881219362,MDExOlB1bGxSZXF1ZXN0NjM0ODIxMDY1,1319,Add Docker multi-arch support with Buildx,10801138,blairdrummond,closed,0,,,,,5,2021-05-08T19:35:03Z,2021-05-27T16:49:24Z,2021-05-27T16:49:24Z,CONTRIBUTOR,simonw/datasette/pulls/1319,"This adds Docker support to extra CPU architectures (like arm) using [Docker's Buildx action](https://github.com/marketplace/actions/docker-setup-buildx)
You can see [what that looks like on Dockerhub](https://hub.docker.com/r/blairdrummond/datasette/tags?page=1&ordering=last_updated)
And how it lets Datasette run on a Raspberry Pi (top is my dockerhub, bottom is upstream)
![Screenshot from 2021-05-08 15-32-25](https://user-images.githubusercontent.com/10801138/117551210-a17a9f80-b012-11eb-966b-10e1590dd4a9.png)
The workflow log [here](https://github.com/blairdrummond/datasette/runs/2535743398?check_suite_focus=true) (I subbed `blairdrummond` for datasetteproject in my branch) ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1319/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
903978133,MDU6SXNzdWU5MDM5NzgxMzM=,1343,Figure out how to publish alpha/beta releases to Docker Hub,9599,simonw,closed,0,,,,,4,2021-05-27T16:42:17Z,2021-05-27T16:46:37Z,2021-05-27T16:45:41Z,OWNER,,"> It looks like all I need to do to ship an alpha version to Docker Hub is NOT point the `latest` tag at it after it goes live: https://github.com/simonw/datasette/blob/1a8972f9c012cd22b088c6b70661a9c3d3847853/.github/workflows/publish.yml#L75-L77
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1319#issuecomment-849780481_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1343/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
898904402,MDU6SXNzdWU4OTg5MDQ0MDI=,1337,"""More"" link for facets that shows _facet_size=max results",9599,simonw,closed,0,,,,,7,2021-05-23T00:08:51Z,2021-05-27T16:14:14Z,2021-05-27T16:01:03Z,OWNER,,"_Original title: ""More"" link for facets that shows the full set of results_
The simplest way to do this will be to have it link to a generated SQL query.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1332#issuecomment-846479062_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1337/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
893890496,MDU6SXNzdWU4OTM4OTA0OTY=,1332,?_facet_size=X to increase number of facets results on the page,192568,mroswell,closed,0,,,,,5,2021-05-18T02:40:16Z,2021-05-27T16:13:07Z,2021-05-23T00:34:37Z,CONTRIBUTOR,,"Is there a way to add a parameter to the URL to modify default_facet_size?
LIkewise, a way to produce a link on the three dots to expand to all items (or match previous number of items, or add x more)?
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1332/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
903902495,MDU6SXNzdWU5MDM5MDI0OTU=,1342,Improve `path_with_replaced_args()` and friends and document them,9599,simonw,open,0,,,,,3,2021-05-27T15:18:28Z,2021-05-27T15:23:02Z,,OWNER,,"> In order to cleanly implement this I need to expose the `path_with_replaced_args` utility function to Datasette's template engine. This is the first time this will become an exposed (and hence should-by-documented) API and I don't like its shape much.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1337#issuecomment-849721280_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1342/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
903200328,MDU6SXNzdWU5MDMyMDAzMjg=,1341,"""Show all columns"" cog menu item should show if ?_col= is used",9599,simonw,closed,0,,,,,1,2021-05-27T04:28:17Z,2021-05-27T04:31:16Z,2021-05-27T04:31:16Z,OWNER,,"On https://latest.datasette.io/fixtures/sortable?_col=sortable the ""Show all columns"" item (from #615) is not shown (it should be):
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1341/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
517451234,MDU6SXNzdWU1MTc0NTEyMzQ=,615,?_col= and ?_nocol= support for toggling columns on table view,9599,simonw,closed,0,,,,,16,2019-11-04T22:55:41Z,2021-05-27T04:26:10Z,2021-05-27T04:17:44Z,OWNER,,Split off from #292 (I guess this is a re-opening of #312).,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/615/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
326800219,MDU6SXNzdWUzMjY4MDAyMTk=,292,Mechanism for customizing the SQL used to select specific columns in the table view,9599,simonw,closed,0,,,,,15,2018-05-27T09:05:52Z,2021-05-27T04:25:01Z,2021-05-27T04:25:01Z,OWNER,,"Some columns don't make a lot of sense in their default representation - binary blobs such as SpatiaLite geometries for example, or lengthy columns that really should be truncated somehow.
We may also find that there are tables where we don't want to show all of the columns - so a mechanism to select a subset of columns would be nice.
I think there are two features here:
* the ability to request a subset of columns on the table view
* the ability to override the SQL for a specific column and/or add extra columns - `AsGeoJSON(Geometry)` for example
Both features should be available via both querystring arguments and in `metadata.json`
The querystring argument for custom SQL should only work if `allow_sql` config is turned on.
Refs #276",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/292/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
899851083,MDExOlB1bGxSZXF1ZXN0NjUxNDkyODg4,1339,?_col=/?_nocol= to show/hide columns on the table page,9599,simonw,closed,0,,,,,1,2021-05-24T17:15:20Z,2021-05-27T04:17:44Z,2021-05-27T04:17:43Z,OWNER,simonw/datasette/pulls/1339,"See #615. Still to do:
- [x] Allow combination of `?_col=` and `?_nocol=` (`_nocol` wins)
- [x] Deduplicate same column if passed in `?_col=` multiple times
- [x] Validate that user did not try to remove a primary key
- [x] Add tests
- [x] Ensure this works correctly for SQL views
- [x] Add documentation
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1339/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
901009787,MDU6SXNzdWU5MDEwMDk3ODc=,1340,Research: Cell action menu (like column action but for individual cells),9599,simonw,open,0,,,,,1,2021-05-25T15:49:16Z,2021-05-26T18:59:58Z,,OWNER,,"Had an idea today that it might be useful to select an individual cell and say things like ""show me all other rows with the same value"" - maybe even a set of other menu options against cells as well.
Mocked up a show-on-hover ellipses demo using the CSS inspector:
![idea](https://user-images.githubusercontent.com/9599/119528316-f0744480-bd35-11eb-8eb4-1deea6d60cce.gif)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1340/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
892457208,MDU6SXNzdWU4OTI0NTcyMDg=,1327,Support Unicode characters in metadata.json,20846286,GmGniap,closed,0,,,,,2,2021-05-15T14:33:58Z,2021-05-24T19:10:21Z,2021-05-24T19:10:21Z,NONE,,"Hello , when I used Burmese (Unicode) characters in metadata.json like below -
![image](https://user-images.githubusercontent.com/20846286/118364978-cba70100-b5c0-11eb-967c-7dc3b62478f2.png)
It gave wrong results when I run datasette -
![image](https://user-images.githubusercontent.com/20846286/118365025-fc873600-b5c0-11eb-97ce-19541b8cc6d8.png)
It would be great & helpful for us if metadata.json can support in Unicode supported Asian Languages.
Thanks & Regards. ",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1327/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
884952179,MDU6SXNzdWU4ODQ5NTIxNzk=,1320,Can't use apt-get in Dockerfile when using datasetteproj/datasette as base,2670795,brandonrobertz,closed,0,,,,,4,2021-05-10T19:37:27Z,2021-05-24T18:15:56Z,2021-05-24T18:07:08Z,CONTRIBUTOR,,"The datasette base Docker image is super convenient, but there's one problem: if any of the plugins you install require additional system dependencies (e.g., xz, git, curl) then any attempt to use apt in said Dockerfile results in an explosion:
```
$ docker-compose build
Building server
[+] Building 9.9s (7/9)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 666B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/datasetteproject/datasette:latest 0.6s
=> [base 1/4] FROM docker.io/datasetteproject/datasette@sha256:2250d0fbe57b1d615a8d6df0c9d43deb9533532e00bac68854773d8ff8dcf00a 0.0s
=> [internal] load build context 1.8s
=> => transferring context: 2.44MB 1.8s
=> CACHED [base 2/4] WORKDIR /datasette 0.0s
=> ERROR [base 3/4] RUN apt-get update && apt-get install --no-install-recommends -y git ssh curl xz-utils 9.2s
------
> [base 3/4] RUN apt-get update && apt-get install --no-install-recommends -y git ssh curl xz-utils:
#6 0.446 Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
#6 0.449 Get:2 http://deb.debian.org/debian buster InRelease [121 kB]
#6 0.459 Get:3 http://httpredir.debian.org/debian sid InRelease [157 kB]
#6 0.784 Get:4 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
#6 0.790 Get:5 http://httpredir.debian.org/debian sid/main amd64 Packages [8626 kB]
#6 1.003 Get:6 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
#6 1.180 Get:7 http://security.debian.org/debian-security buster/updates/main amd64 Packages [286 kB]
#6 7.095 Get:8 http://deb.debian.org/debian buster-updates/main amd64 Packages [10.9 kB]
#6 8.058 Fetched 17.2 MB in 8s (2243 kB/s)
#6 8.058 Reading package lists...
#6 9.166 E: flAbsPath on /var/lib/dpkg/status failed - realpath (2: No such file or directory)
#6 9.166 E: Could not open file - open (2: No such file or directory)
#6 9.166 E: Problem opening
#6 9.166 E: The package lists or status file could not be parsed or opened.
```
The problem seems to be from completely wiping out `/var/lib/dpkg` in the upstream Dockerfile:
https://github.com/simonw/datasette/blob/1b697539f5b53cec3fe13c0f4ada13ba655c88c7/Dockerfile#L18
I've tested without removing the directory and apt works as expected.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1320/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
899169307,MDU6SXNzdWU4OTkxNjkzMDc=,1338,Fix jinja2 warnings,9599,simonw,closed,0,,,,,0,2021-05-24T01:38:23Z,2021-05-24T01:41:55Z,2021-05-24T01:41:55Z,OWNER,,"Lots of these in the test suite now, after the Jinja upgrade in #1331:
```
tests/test_plugins.py::test_hook_render_cell_link_from_json
datasette/tests/plugins/my_plugin_2.py:45: DeprecationWarning: 'jinja2.escape' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.escape' instead.
label=jinja2.escape(data[""label""] or """") or "" "",
tests/test_plugins.py::test_hook_render_cell_link_from_json
datasette/tests/plugins/my_plugin_2.py:41: DeprecationWarning: 'jinja2.Markup' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.Markup' instead.
return jinja2.Markup(
```",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1338/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
891969037,MDU6SXNzdWU4OTE5NjkwMzc=,1326,How to limit fields returned from the JSON API?,5268174,bram2000,closed,0,,,,,1,2021-05-14T14:27:41Z,2021-05-23T02:55:06Z,2021-05-23T02:55:00Z,NONE,,"Hi,
I have quite wide tables, and in many cases only want a subset of the data (to save on network bandwidth). I need to use the JSON API as handling pagination is so much easier, but I can't see a way to select specific columns.
Is there a way to do this, or is it a feature request?
Thanks!",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1326/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
893537744,MDU6SXNzdWU4OTM1Mzc3NDQ=,1331,Add support for Jinja2 version 3.0,475613,MarkusH,closed,0,,,,,10,2021-05-17T17:14:36Z,2021-05-23T00:57:39Z,2021-05-23T00:57:39Z,NONE,,"A week ago, [The Pallets Project](https://github.com/pallets) released [new major versions of several of its projects](https://palletsprojects.com/blog/flask-2-0-released/). Among those updates is one for Jinja2, which bumps it to version 3.0.0.
I'd like for datasette to support Jinaj2 version 3.0.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1331/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
887241681,MDExOlB1bGxSZXF1ZXN0NjQwNDg0OTY2,1321,Bump black from 21.4b2 to 21.5b1,49699333,dependabot[bot],closed,0,,,,,1,2021-05-11T13:12:28Z,2021-05-22T23:55:39Z,2021-05-22T23:55:39Z,CONTRIBUTOR,simonw/datasette/pulls/1321,"Bumps [black](https://github.com/psf/black) from 21.4b2 to 21.5b1.
Release notes
Sourced from black's releases .
21.5b1
Black
Refactor src/black/__init__.py
into many files (#2206 )
Documentation
Replaced all remaining references to the master
branch with the main
branch. Some additional changes in the source code were also made. (#2210 )
Sigificantly reorganized the documentation to make much more sense. Check them out by heading over to the stable docs on RTD . (#2174 )
21.5b0
Black
Set --pyi
mode if --stdin-filename
ends in .pyi
(#2169 )
Stop detecting target version as Python 3.9+ with pre-PEP-614 decorators that are
being called but with no arguments (#2182 )
Black-Primer
Add --no-diff
to black-primer to suppress formatting changes (#2187 )
Changelog
Sourced from black's changelog .
21.5b1
Black
Refactor src/black/__init__.py
into many files (#2206 )
Documentation
Replaced all remaining references to the
master
branch with the
main
branch. Some additional changes in
the source code were also made. (#2210 )
Sigificantly reorganized the documentation to make much more sense. Check them out by
heading over to the stable docs on RTD .
(#2174 )
21.5b0
Black
Set --pyi
mode if --stdin-filename
ends in .pyi
(#2169 )
Stop detecting target version as Python 3.9+ with pre-PEP-614 decorators that are
being called but with no arguments (#2182 )
Black-Primer
Add --no-diff
to black-primer to suppress formatting changes (#2187 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.4b2&new-version=21.5b1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1321/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
890073888,MDExOlB1bGxSZXF1ZXN0NjQzMTQ5Mjcz,1323,"Update click requirement from ~=7.1.1 to >=7.1.1,<8.1.0",49699333,dependabot[bot],closed,0,,,,,1,2021-05-12T13:08:56Z,2021-05-22T23:54:48Z,2021-05-22T23:54:48Z,CONTRIBUTOR,simonw/datasette/pulls/1323,"Updates the requirements on [click](https://github.com/pallets/click) to permit the latest version.
Release notes
Sourced from click's releases .
8.0.0
New major versions of all the core Pallets libraries, including Click 8.0, have been released! :tada:
This represents a significant amount of work, and there are quite a few changes. Be sure to carefully read the changelog, and use tools such as pip-compile and Dependabot to pin your dependencies and control your updates.
Changelog
Sourced from click's changelog .
Version 8.0.0
Released 2021-05-11
Drop support for Python 2 and 3.5.
Colorama is always installed on Windows in order to provide style
and color support. :pr:1784
Adds a repr to Command, showing the command name for friendlier
debugging. :issue:1267
, :pr:1295
Add support for distinguishing the source of a command line
parameter. :issue:1264
, :pr:1329
Add an optional parameter to ProgressBar.update
to set the
current_item
. :issue:1226
, :pr:1332
version_option
uses importlib.metadata
(or the
importlib_metadata
backport) instead of pkg_resources
.
:issue:1582
If validation fails for a prompt with hide_input=True
, the value
is not shown in the error message. :issue:1460
An IntRange
or FloatRange
option shows the accepted range in
its help text. :issue:1525
, :pr:1303
IntRange
and FloatRange
bounds can be open (<
) instead
of closed (<=
) by setting min_open
and max_open
. Error
messages have changed to reflect this. :issue:1100
An option defined with duplicate flag names ("--foo/--foo"
)
raises a ValueError
. :issue:1465
echo()
will not fail when using pytest's capsys
fixture on
Windows. :issue:1590
Resolving commands returns the canonical command name instead of the
matched name. This makes behavior such as help text and
Context.invoked_subcommand
consistent when using patterns like
AliasedGroup
. :issue:1422
The BOOL
type accepts the values "on" and "off". :issue:1629
A Group
with invoke_without_command=True
will always invoke
its result callback. :issue:1178
nargs == -1
and nargs > 1
is parsed and validated for
values from environment variables and defaults. :issue:729
Detect the program name when executing a module or package with
python -m name
. :issue:1603
Include required parent arguments in help synopsis of subcommands.
:issue:1475
Help for boolean flags with show_default=True
shows the flag
name instead of True
or False
. :issue:1538
Non-string objects passed to style()
and secho()
will be
converted to string. :pr:1146
edit(require_save=True)
will detect saves for editors that exit
very fast on filesystems with 1 second resolution. :pr:1050
New class attributes make it easier to use custom core objects
throughout an entire application. :pr:938
... (truncated)
Commits
9da1669
Merge pull request #1877 from pallets/release-8.0.0
dfa6369
release version 8.0.0
b862cb1
update requirements
f51584c
Merge pull request #1876 from pallets/pre-commit-ci-schedule
804c71c
update pre-commit monthly
ac655f8
Merge pull request #1872 from janLuke/fix/formatter_write_text
dcd991d
HelpFormatter.write_text uses full width
5215fc1
Merge pull request #1870 from AdrienPensart/allow_colors_in_metavar
e3e1691
repr is erasing ANSI escapes codes
482e6e6
Merge pull request #1875 from pallets/pre-commit-ci-update-config
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1323/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
890073989,MDExOlB1bGxSZXF1ZXN0NjQzMTQ5MzY0,1325,"Update itsdangerous requirement from ~=1.1 to >=1.1,<3.0",49699333,dependabot[bot],closed,0,,,,,2,2021-05-12T13:09:03Z,2021-05-22T23:54:25Z,2021-05-22T23:54:25Z,CONTRIBUTOR,simonw/datasette/pulls/1325,"Updates the requirements on [itsdangerous](https://github.com/pallets/itsdangerous) to permit the latest version.
Release notes
Sourced from itsdangerous's releases .
2.0.0
New major versions of all the core Pallets libraries, including ItsDangerous 2.0, have been released! :tada:
This represents a significant amount of work, and there are quite a few changes. Be sure to carefully read the changelog, and use tools such as pip-compile and Dependabot to pin your dependencies and control your updates.
Changelog
Sourced from itsdangerous's changelog .
Version 2.0.0
Released 2021-05-11
Drop support for Python 2 and 3.5.
JWS support (JSONWebSignatureSerializer
,
TimedJSONWebSignatureSerializer
) is deprecated. Use a dedicated
JWS/JWT library such as authlib instead. :issue:129
Importing itsdangerous.json
is deprecated. Import Python's
json
module instead. :pr:152
Simplejson is no longer used if it is installed. To use a different
library, pass it as Serializer(serializer=...)
. :issue:146
datetime
values are timezone-aware with timezone.utc
. Code
using TimestampSigner.unsign(return_timestamp=True)
or
BadTimeSignature.date_signed
may need to change. :issue:150
If a signature has an age less than 0, it will raise
SignatureExpired
rather than appearing valid. This can happen if
the timestamp offset is changed. :issue:126
BadTimeSignature.date_signed
is always a datetime
object
rather than an int
in some cases. :issue:124
Added support for key rotation. A list of keys can be passed as
secret_key
, oldest to newest. The newest key is used for
signing, all keys are tried for unsigning. :pr:141
Removed the default SHA-512 fallback signer from
default_fallback_signers
. :issue:155
Add type information for static typing tools. :pr:186
Version 1.1.0
Released 2018-10-26
Change default signing algorithm back to SHA-1. :pr:113
Added a default SHA-512 fallback for users who used the yanked 1.0.0
release which defaulted to SHA-512. :pr:114
Add support for fallback algorithms during deserialization to
support changing the default in the future without breaking existing
signatures. :pr:113
Changed capitalization of packages back to lowercase as the change
in capitalization broke some tooling. :pr:113
Version 1.0.0
Released 2018-10-18
YANKED
... (truncated)
Commits
d101100
Merge pull request #235 from pallets/release-2.0.0
ca0f59a
release version 2.0.0
d1ed89f
update requirements
d1722ea
Merge pull request #234 from pallets/pre-commit-ci-schedule
d1eb7aa
update pre-commit monthly
acbc456
Merge pull request #233 from pallets/pre-commit-ci-update-config
04e485a
[pre-commit.ci] pre-commit autoupdate
c0e6b48
Merge pull request #232 from pallets/pre-commit-ci-update-config
6a9df83
[pre-commit.ci] pre-commit autoupdate
477f42c
Merge pull request #231 from pallets/dependabot/pip/pre-commit-2.12.1
Additional commits viewable in compare view
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1325/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
893314402,MDExOlB1bGxSZXF1ZXN0NjQ1ODQ5MDI3,1330,"Update aiofiles requirement from <0.7,>=0.4 to >=0.4,<0.8",49699333,dependabot[bot],closed,0,,,,,1,2021-05-17T13:07:31Z,2021-05-22T23:53:57Z,2021-05-22T23:53:56Z,CONTRIBUTOR,simonw/datasette/pulls/1330,"Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version.
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1330/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
895315478,MDExOlB1bGxSZXF1ZXN0NjQ3NTUyMTQx,1335,Fix small typo,3243482,abdusco,closed,0,,,,,1,2021-05-19T11:17:04Z,2021-05-22T23:53:34Z,2021-05-22T23:53:34Z,CONTRIBUTOR,simonw/datasette/pulls/1335,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1335/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
642296989,MDU6SXNzdWU2NDIyOTY5ODk=,856,Consider pagination of canned queries,9599,simonw,open,0,,,,,3,2020-06-20T03:15:59Z,2021-05-21T14:22:41Z,,OWNER,,The new `canned_queries()` plugin hook from #852 combined with plugins like https://github.com/simonw/datasette-saved-queries could mean that some installations end up with hundreds or even thousands of canned queries. I should consider pagination or some other way of ensuring that this doesn't cause performance problems for Datasette.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/856/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
812228314,MDU6SXNzdWU4MTIyMjgzMTQ=,1236,Ability to increase size of the SQL editor window,9599,simonw,closed,0,,,,,9,2021-02-19T18:09:27Z,2021-05-18T03:28:25Z,2021-02-22T21:05:21Z,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1236/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
890073940,MDExOlB1bGxSZXF1ZXN0NjQzMTQ5MzIw,1324,"Update jinja2 requirement from <2.12.0,>=2.10.3 to >=2.10.3,<3.1.0",49699333,dependabot[bot],closed,0,,,,,2,2021-05-12T13:08:59Z,2021-05-17T17:19:41Z,2021-05-17T17:19:40Z,CONTRIBUTOR,simonw/datasette/pulls/1324,"Updates the requirements on [jinja2](https://github.com/pallets/jinja) to permit the latest version.
Release notes
Sourced from jinja2's releases .
3.0.0
New major versions of all the core Pallets libraries, including Jinja 3.0, have been released! :tada:
This represents a significant amount of work, and there are quite a few changes. Be sure to carefully read the changelog, and use tools such as pip-compile and Dependabot to pin your dependencies and control your updates.
Changelog
Sourced from jinja2's changelog .
Version 3.0.0
Released 2021-05-11
Drop support for Python 2.7 and 3.5.
Bump MarkupSafe dependency to >=1.1.
Bump Babel optional dependency to >=2.1.
Remove code that was marked deprecated.
Add type hinting. :pr:1412
Use :pep:451
API to load templates with
:class:~loaders.PackageLoader
. :issue:1168
Fix a bug that caused imported macros to not have access to the
current template's globals. :issue:688
Add ability to ignore trim_blocks
using +%}
. :issue:1036
Fix a bug that caused custom async-only filters to fail with
constant input. :issue:1279
Fix UndefinedError incorrectly being thrown on an undefined variable
instead of Undefined
being returned on
NativeEnvironment
on Python 3.10. :issue:1335
Blocks can be marked as required
. They must be overridden at
some point, but not necessarily by the direct child. :issue:1147
Deprecate the autoescape
and with
extensions, they are
built-in to the compiler. :issue:1203
The urlize
filter recognizes mailto:
links and takes
extra_schemes
(or env.policies["urlize.extra_schemes"]
) to
recognize other schemes. It tries to balance parentheses within a
URL instead of ignoring trailing characters. The parsing in general
has been updated to be more efficient and match more cases. URLs
without a scheme are linked as https://
instead of http://
.
:issue:522, 827, 1172
, :pr:1195
Filters that get attributes, such as map
and groupby
, can
use a false or empty value as a default. :issue:1331
Fix a bug that prevented variables set in blocks or loops from
being accessed in custom context functions. :issue:768
Fix a bug that caused scoped blocks from accessing special loop
variables. :issue:1088
Update the template globals when calling
Environment.get_template(globals=...)
even if the template was
already loaded. :issue:295
Do not raise an error for undefined filters in unexecuted
if-statements and conditional expressions. :issue:842
Add is filter
and is test
tests to test if a name is a
registered filter or test. This allows checking if a filter is
available in a template before using it. Test functions can be
decorated with @pass_environment
, @pass_eval_context
,
or @pass_context
. :issue:842
, :pr:1248
Support pgettext
and npgettext
(message contexts) in i18n
extension. :issue:441
The |indent
filter's width
argument can be a string to
... (truncated)
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1324/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
876431852,MDExOlB1bGxSZXF1ZXN0NjMwNTc4NzM1,1318,Bump black from 21.4b2 to 21.5b0,49699333,dependabot[bot],closed,0,,,,,2,2021-05-05T13:07:51Z,2021-05-11T13:12:32Z,2021-05-11T13:12:31Z,CONTRIBUTOR,simonw/datasette/pulls/1318,"Bumps [black](https://github.com/psf/black) from 21.4b2 to 21.5b0.
Release notes
Sourced from black's releases .
21.5b0
Black
Set --pyi
mode if --stdin-filename
ends in .pyi
(#2169 )
Stop detecting target version as Python 3.9+ with pre-PEP-614 decorators that are
being called but with no arguments (#2182 )
Black-Primer
Add --no-diff
to black-primer to suppress formatting changes (#2187 )
Changelog
Sourced from black's changelog .
21.5b0
Black
Set --pyi
mode if --stdin-filename
ends in .pyi
(#2169 )
Stop detecting target version as Python 3.9+ with pre-PEP-614 decorators that are
being called but with no arguments (#2182 )
Black-Primer
Add --no-diff
to black-primer to suppress formatting changes (#2187 )
Commits
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=21.4b2&new-version=21.5b0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1318/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
842862708,MDU6SXNzdWU4NDI4NjI3MDg=,1280,Ability to run CI against multiple SQLite versions,9599,simonw,open,0,,,,,2,2021-03-28T23:54:50Z,2021-05-10T19:07:46Z,,OWNER,,"Issue #1276 happened because I didn't run tests against a SQLite version prior to 3.16.0 (released 2017-01-02).
Glitch is a deployment target and runs SQLite 3.11.0 from 2016-02-15.
If CI ran against that version of SQLite this bug could have been avoided.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1280/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
870125126,MDU6SXNzdWU4NzAxMjUxMjY=,1310,I'm creating a plugin to export a spreadsheet file (.ods or .xlsx),3747136,ColinMaudry,closed,0,,,,,2,2021-04-28T16:20:11Z,2021-04-30T07:26:11Z,2021-04-30T06:58:46Z,NONE,,"Hi,
I have started developing a plugin to export records as a spreadsheet file. It could be ods or xlsx, whatever is easier.
I have spotted the following packages:
- ods files: https://pypi.org/project/odswriter/
- xlsx files: https://openpyxl.readthedocs.io/en/stable/index.html (quite powerful) or https://xlsxwriter.readthedocs.io/ (faster)
This is the code I have so far, I test it with the `--plugins-dir` option:
```python
from datasette import hookimpl
from datasette.utils.asgi import Response
import odswriter as ods
def render_spreadsheet(rows):
with ods.writer(open(""test.ods"",""wb"")) as odsfile:
for row in rows:
odsfile.writerow([""String"", ""ABCDEF123456"", ""123456""])
return Response(odsfile, content_type=""application/vnd.oasis.opendocument.spreadsheet"", status=200)
@hookimpl
def register_output_renderer():
return {""extension"": ""ods"", ""render"": render_spreadsheet}
```
I get the following error:
```
Traceback (most recent call last):
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1128, in route_path
await response.asgi_send(send)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 339, in asgi_send
body = body.encode(""utf-8"")
AttributeError: 'ODSWriter' object has no attribute 'encode'
ERROR: Exception in ASGI application
Traceback (most recent call last):
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1128, in route_path
await response.asgi_send(send)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 339, in asgi_send
body = body.encode(""utf-8"")
AttributeError: 'ODSWriter' object has no attribute 'encode'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File ""/home/colin/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py"", line 396, in run_asgi
result = await app(self.scope, self.receive, self.send)
File ""/home/colin/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py"", line 45, in __call__
return await self.app(scope, receive, send)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 161, in __call__
await self.app(scope, receive, send)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/tracer.py"", line 75, in __call__
await self.app(scope, receive, send)
File ""/home/colin/.local/lib/python3.8/site-packages/asgi_csrf.py"", line 107, in app_wrapped_with_csrf
await app(scope, receive, wrapped_send)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1086, in __call__
return await self.route_path(scope, receive, send, path)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1133, in route_path
return await self.handle_500(request, send, exception)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/app.py"", line 1267, in handle_500
await asgi_send_html(
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 217, in asgi_send_html
await asgi_send(
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 237, in asgi_send
await asgi_start(send, status, headers, content_type)
File ""/home/colin/.local/lib/python3.8/site-packages/datasette/utils/asgi.py"", line 246, in asgi_start
await send(
File ""/home/colin/.local/lib/python3.8/site-packages/asgi_csrf.py"", line 103, in wrapped_send
await send(event)
File ""/home/colin/.local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py"", line 482, in send
raise RuntimeError(msg % message_type)
RuntimeError: Expected ASGI message 'http.response.body', but got 'http.response.start'.
```
I tried with `AsgiFileDownload` like in [DatabaseDownload](https://github.com/simonw/datasette/blob/main/datasette/views/database.py#L150) to deal with the binary nature of the ods file, but the renderer expects a Response:
> should be dict or Response
However, the `Response` class only supports the following methods, not binary:
- html
- text
- json
- redirect
How would you suggest me to proceed to have my ods file downloaded?
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1310/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
871304967,MDU6SXNzdWU4NzEzMDQ5Njc=,1315,"settings.json should be picked up by ""datasette publish cloudrun""",9599,simonw,open,0,,,,,0,2021-04-29T18:16:41Z,2021-04-29T18:16:41Z,,OWNER,,,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1315/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
871046111,MDExOlB1bGxSZXF1ZXN0NjI2MTMwMTM1,1313,Bump black from 20.8b1 to 21.4b2,27856297,dependabot-preview[bot],closed,0,,,,,2,2021-04-29T13:58:06Z,2021-04-29T15:47:50Z,2021-04-29T15:47:49Z,CONTRIBUTOR,simonw/datasette/pulls/1313,"Bumps [black](https://github.com/psf/black) from 20.8b1 to 21.4b2.
Release notes
Sourced from black's releases .
21.4b2
Black
Packaging
Install primer.json
(used by black-primer
by default) with black. (#2154 )
21.4b1
Black
Fix crash on docstrings ending with "\ ". (#2142 )
Fix crash when atypical whitespace is cleaned out of dostrings (#2120 )
Reflect the --skip-magic-trailing-comma
and --experimental-string-processing
flags
in the name of the cache file. Without this fix, changes in these flags would not take
effect if the cache had already been populated. (#2131 )
Don't remove necessary parentheses from assignment expression containing assert /
return statements. (#2143 )
Packaging
Bump pathspec to >= 0.8.1 to solve invalid .gitignore exclusion handling
21.4b0
Black
Fixed a rare but annoying formatting instability created by the combination of
optional trailing commas inserted by Black
and optional parentheses looking at
pre-existing "magic" trailing commas. This fixes issue #1629 and all of its many many
duplicates. (#2126 )
Black
now processes one-line docstrings by stripping leading and trailing spaces,
and adding a padding space when needed to break up """". (#1740 )
Black
now cleans up leading non-breaking spaces in comments (#2092 )
Black
now respects --skip-string-normalization
when normalizing multiline
docstring quotes (#1637 )
Black
no longer removes all empty lines between non-function code and decorators
when formatting typing stubs. Now Black
enforces a single empty line. (#1646 )
... (truncated)
Changelog
Sourced from black's changelog .
21.4b2
Black
Packaging
Install primer.json
(used by black-primer
by default) with black. (#2154 )
21.4b1
Black
Fix crash on docstrings ending with "\ ". (#2142 )
Fix crash when atypical whitespace is cleaned out of dostrings (#2120 )
Reflect the --skip-magic-trailing-comma
and --experimental-string-processing
flags
in the name of the cache file. Without this fix, changes in these flags would not take
effect if the cache had already been populated. (#2131 )
Don't remove necessary parentheses from assignment expression containing assert /
return statements. (#2143 )
Packaging
Bump pathspec to >= 0.8.1 to solve invalid .gitignore exclusion handling
21.4b0
Black
Fixed a rare but annoying formatting instability created by the combination of
optional trailing commas inserted by Black
and optional parentheses looking at
pre-existing "magic" trailing commas. This fixes issue #1629 and all of its many many
duplicates. (#2126 )
Black
now processes one-line docstrings by stripping leading and trailing spaces,
and adding a padding space when needed to break up """". (#1740 )
Black
now cleans up leading non-breaking spaces in comments (#2092 )
Black
now respects --skip-string-normalization
when normalizing multiline
docstring quotes (#1637 )
... (truncated)
Commits
[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=20.8b1&new-version=21.4b2)](https://dependabot.com/compatibility-score/?dependency-name=black&package-manager=pip&previous-version=20.8b1&new-version=21.4b2)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a ""Dependabot enabled"" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1313/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
871157602,MDExOlB1bGxSZXF1ZXN0NjI2MjIyNjc2,1314,Upgrade to GitHub-native Dependabot,27856297,dependabot-preview[bot],closed,0,,,,,1,2021-04-29T15:36:41Z,2021-04-29T15:47:22Z,2021-04-29T15:47:21Z,CONTRIBUTOR,simonw/datasette/pulls/1314,"_Dependabot Preview will be shut down on August 3rd, 2021. In order to keep getting Dependabot updates, please merge this PR and migrate to GitHub-native Dependabot before then._
Dependabot has been fully integrated into GitHub, so you no longer have to install and manage a separate app. This pull request migrates your configuration from Dependabot.com to a config file, using the [new syntax][new_syntax]. When merged, we'll swap out `dependabot-preview` (me) for a new `dependabot` app, and you'll be all set!
With this change, you'll now use the [Dependabot page in GitHub][dependabot_page], rather than the [Dependabot dashboard][dashboard], to monitor your version updates, and you'll configure Dependabot through the new config file rather than a UI.
If you've got any questions or feedback for us, please let us know by creating an issue in the [dependabot/dependabot-core][issues] repository.
[Learn more about migrating to GitHub-native Dependabot][learn]
Please note that regular `@dependabot` commands do not work on this pull request.
[dashboard]: https://app.dependabot.com/
[dependabot_page]: https://github.com/simonw/datasette/network/updates
[issues]: https://github.com/dependabot/dependabot-core/issues/new?assignees=%40dependabot%2Fpreview-migration-reviewers&labels=E%3A+preview-migration&template=migration-issue.md
[learn]: http://docs.github.com/code-security/supply-chain-security/upgrading-from-dependabotcom-to-github-native-dependabot
[new_syntax]: https://help.github.com/en/github/administering-a-repository/configuration-options-for-dependency-updates
[org_secrets_url]: https://github.com/settings/secrets/dependabot
[repo_secrets_url]: https://github.com/simonw/datasette/settings/secrets/dependabot
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1314/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
870227815,MDExOlB1bGxSZXF1ZXN0NjI1NDU3NTc5,1311,Bump black from 20.8b1 to 21.4b1,27856297,dependabot-preview[bot],closed,0,,,,,2,2021-04-28T18:25:58Z,2021-04-29T13:58:11Z,2021-04-29T13:58:09Z,CONTRIBUTOR,simonw/datasette/pulls/1311,"Bumps [black](https://github.com/psf/black) from 20.8b1 to 21.4b1.
Release notes
Sourced from black's releases .
21.4b1
Black
Fix crash on docstrings ending with "\ ". (#2142 )
Fix crash when atypical whitespace is cleaned out of dostrings (#2120 )
Reflect the --skip-magic-trailing-comma
and --experimental-string-processing
flags
in the name of the cache file. Without this fix, changes in these flags would not take
effect if the cache had already been populated. (#2131 )
Don't remove necessary parentheses from assignment expression containing assert /
return statements. (#2143 )
Packaging
Bump pathspec to >= 0.8.1 to solve invalid .gitignore exclusion handling
21.4b0
Black
Fixed a rare but annoying formatting instability created by the combination of
optional trailing commas inserted by Black
and optional parentheses looking at
pre-existing "magic" trailing commas. This fixes issue #1629 and all of its many many
duplicates. (#2126 )
Black
now processes one-line docstrings by stripping leading and trailing spaces,
and adding a padding space when needed to break up """". (#1740 )
Black
now cleans up leading non-breaking spaces in comments (#2092 )
Black
now respects --skip-string-normalization
when normalizing multiline
docstring quotes (#1637 )
Black
no longer removes all empty lines between non-function code and decorators
when formatting typing stubs. Now Black
enforces a single empty line. (#1646 )
Black
no longer adds an incorrect space after a parenthesized assignment expression
in if/while statements (#1655 )
Added --skip-magic-trailing-comma
/ -C
to avoid using trailing commas as a reason
to split lines (#1824 )
fixed a crash when PWD=/ on POSIX (#1631 )
fixed "I/O operation on closed file" when using --diff (#1664 )
Prevent coloured diff output being interleaved with multiple files (#1673 )
Added support for PEP 614 relaxed decorator syntax on python 3.9 (#1711 )
... (truncated)
Changelog
Sourced from black's changelog .
21.4b1
Black
Fix crash on docstrings ending with "\ ". (#2142 )
Fix crash when atypical whitespace is cleaned out of dostrings (#2120 )
Reflect the --skip-magic-trailing-comma
and --experimental-string-processing
flags
in the name of the cache file. Without this fix, changes in these flags would not take
effect if the cache had already been populated. (#2131 )
Don't remove necessary parentheses from assignment expression containing assert /
return statements. (#2143 )
Packaging
Bump pathspec to >= 0.8.1 to solve invalid .gitignore exclusion handling
21.4b0
Black
Fixed a rare but annoying formatting instability created by the combination of
optional trailing commas inserted by Black
and optional parentheses looking at
pre-existing "magic" trailing commas. This fixes issue #1629 and all of its many many
duplicates. (#2126 )
Black
now processes one-line docstrings by stripping leading and trailing spaces,
and adding a padding space when needed to break up """". (#1740 )
Black
now cleans up leading non-breaking spaces in comments (#2092 )
Black
now respects --skip-string-normalization
when normalizing multiline
docstring quotes (#1637 )
Black
no longer removes all empty lines between non-function code and decorators
when formatting typing stubs. Now Black
enforces a single empty line. (#1646 )
Black
no longer adds an incorrect space after a parenthesized assignment expression
in if/while statements (#1655 )
Added --skip-magic-trailing-comma
/ -C
to avoid using trailing commas as a reason
to split lines (#1824 )
fixed a crash when PWD=/ on POSIX (#1631 )
fixed "I/O operation on closed file" when using --diff (#1664 )
Prevent coloured diff output being interleaved with multiple files (#1673 )
... (truncated)
Commits
[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=20.8b1&new-version=21.4b1)](https://dependabot.com/compatibility-score/?dependency-name=black&package-manager=pip&previous-version=20.8b1&new-version=21.4b1)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a ""Dependabot enabled"" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1311/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
870946764,MDU6SXNzdWU4NzA5NDY3NjQ=,1312,how to query many-to-many relationship via json API?,5268174,bram2000,open,0,,,,,0,2021-04-29T12:09:49Z,2021-04-29T12:09:49Z,,NONE,,"Hi,
Firstly thanks for Datasette, it's great!
I'm trying to use the JSON API to query data from a Datasette instance. I have a simple 3 table many-to-many relationship, like so:
`category` - list of categories
`document` - list of documents
`document_category` - join table (a category contains many documents, and a document can be a member of multiple categories)
the `document_category` table foreign keys to the other two using their respective row_ids.
Now I want to return ""all documents within category X"" but I cannot see a way to do this without executing two queries; the first to lookup the row_id of category X, and the second to join `document` with `document_category` where category ID is .
I could easily write this in SQL, but this makes programmatic handling of pagination much more difficult (we'd have to dynamically modify the SQL to select the row_id and include the correct where and limit clauses).
Is there a way to achieve this using the JSON API?
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1312/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
869237023,MDExOlB1bGxSZXF1ZXN0NjI0NjM1NDQw,1309,Bump black from 20.8b1 to 21.4b0,27856297,dependabot-preview[bot],closed,0,,,,,2,2021-04-27T20:28:11Z,2021-04-28T18:26:06Z,2021-04-28T18:26:04Z,CONTRIBUTOR,simonw/datasette/pulls/1309,"Bumps [black](https://github.com/psf/black) from 20.8b1 to 21.4b0.
Release notes
Sourced from black's releases .
21.4b0
Black
Fixed a rare but annoying formatting instability created by the combination of
optional trailing commas inserted by Black
and optional parentheses looking at
pre-existing "magic" trailing commas. This fixes issue #1629 and all of its many many
duplicates. (#2126 )
Black
now processes one-line docstrings by stripping leading and trailing spaces,
and adding a padding space when needed to break up """". (#1740 )
Black
now cleans up leading non-breaking spaces in comments (#2092 )
Black
now respects --skip-string-normalization
when normalizing multiline
docstring quotes (#1637 )
Black
no longer removes all empty lines between non-function code and decorators
when formatting typing stubs. Now Black
enforces a single empty line. (#1646 )
Black
no longer adds an incorrect space after a parenthesized assignment expression
in if/while statements (#1655 )
Added --skip-magic-trailing-comma
/ -C
to avoid using trailing commas as a reason
to split lines (#1824 )
fixed a crash when PWD=/ on POSIX (#1631 )
fixed "I/O operation on closed file" when using --diff (#1664 )
Prevent coloured diff output being interleaved with multiple files (#1673 )
Added support for PEP 614 relaxed decorator syntax on python 3.9 (#1711 )
Added parsing support for unparenthesized tuples and yield expressions in annotated
assignments (#1835 )
use lowercase hex strings (#1692 )
added --extend-exclude
argument (PR #2005 )
speed up caching by avoiding pathlib (#1950 )
--diff
correctly indicates when a file doesn't end in a newline (#1662 )
Added --stdin-filename
argument to allow stdin to respect --force-exclude
rules
(#1780 )
Lines ending with fmt: skip
will now be not formatted (#1800 )
PR #2053 : Black no longer relies on typed-ast for Python 3.8 and higher
... (truncated)
Changelog
Sourced from black's changelog .
21.4b0
Black
Fixed a rare but annoying formatting instability created by the combination of
optional trailing commas inserted by Black
and optional parentheses looking at
pre-existing "magic" trailing commas. This fixes issue #1629 and all of its many many
duplicates. (#2126 )
Black
now processes one-line docstrings by stripping leading and trailing spaces,
and adding a padding space when needed to break up """". (#1740 )
Black
now cleans up leading non-breaking spaces in comments (#2092 )
Black
now respects --skip-string-normalization
when normalizing multiline
docstring quotes (#1637 )
Black
no longer removes all empty lines between non-function code and decorators
when formatting typing stubs. Now Black
enforces a single empty line. (#1646 )
Black
no longer adds an incorrect space after a parenthesized assignment expression
in if/while statements (#1655 )
Added --skip-magic-trailing-comma
/ -C
to avoid using trailing commas as a reason
to split lines (#1824 )
fixed a crash when PWD=/ on POSIX (#1631 )
fixed "I/O operation on closed file" when using --diff (#1664 )
Prevent coloured diff output being interleaved with multiple files (#1673 )
Added support for PEP 614 relaxed decorator syntax on python 3.9 (#1711 )
Added parsing support for unparenthesized tuples and yield expressions in annotated
assignments (#1835 )
added --extend-exclude
argument (PR #2005 )
speed up caching by avoiding pathlib (#1950 )
--diff
correctly indicates when a file doesn't end in a newline (#1662 )
Added --stdin-filename
argument to allow stdin to respect --force-exclude
rules
(#1780 )
Lines ending with fmt: skip
will now be not formatted (#1800 )
PR #2053 : Black no longer relies on typed-ast for Python 3.8 and higher
... (truncated)
Commits
[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=black&package-manager=pip&previous-version=20.8b1&new-version=21.4b0)](https://dependabot.com/compatibility-score/?dependency-name=black&package-manager=pip&previous-version=20.8b1&new-version=21.4b0)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a ""Dependabot enabled"" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1309/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
281110295,MDU6SXNzdWUyODExMTAyOTU=,173,I18n and L10n support,50138,janimo,open,0,,,,,2,2017-12-11T17:49:58Z,2021-04-26T12:10:01Z,,NONE,,It would be less geeky and more user friendly if the display strings in the filter menu and possibly other parts could be localized.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/173/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
866668415,MDU6SXNzdWU4NjY2Njg0MTU=,1308,"Columns named ""link"" display in bold",9599,simonw,closed,0,,,,,3,2021-04-24T05:58:11Z,2021-04-24T06:07:49Z,2021-04-24T06:07:49Z,OWNER,,Reported in office hours today.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1308/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
860734722,MDU6SXNzdWU4NjA3MzQ3MjI=,1302,Fix disappearing facets,192568,mroswell,open,0,,,,,0,2021-04-18T18:42:33Z,2021-04-20T07:40:15Z,,CONTRIBUTOR,,"1. Clone https://github.com/mroswell/list-N
2. Run `datasette disinfectants.db -o`
3. Select the `Safer_or_Toxic` facet.
4. Select `Toxic`.
5. Close out the `Safer_or_Toxic` facet.
6. Examine `Suggested facets` list. `Safer_or_Toxic` is GONE.
7. Try some other facets. When you select an element, and then close the list, in some cases, the facet properly returns to the `Suggested facet` list... Arrays and dates properly return to the list, but fields with strings don't return to the list.
Since my site is devoted to whether disinfectants are Safer or Toxic, having the suggested facet disappear from the suggested facet list is very confusing* to end-users. This, along with a few other issues, unfortunately proved beyond my own programming ability to address. So I hired a Senior-level developer to address a number of issues, including this disappearing act.
8. Open a new terminal. Run `datasette disinfectants.db -m metadata.json --static static:static/ --template-dir templates/ --plugins-dir plugins/ -p 8001 -o`
9. Repeat steps 3-6, but this time, the Safer_or_Toxic facet returns to the list (and the related URL parameters are removed).
I'm not sure how to do a pull request for this, because the plugin contains other functionality that goes beyond this bug. I wanted the facets sorted in a certain order (both in the suggested facet list, and the detail lists) (... the detail lists were hopping around all over the place before...) I wanted the duplicate facets removed (leaving only the one where you can facet by individual item in an array.) I wanted the arrays to be presented in a prettier fashion (I did that in the template... That could be moved over to the plugin at some point)
I'm thinking it'll be very helpful if applicable parts of my project's plugin (sort_suggested_facets_plugin.py) will be able to be incorporated back into datasette, but I leave that to you to consider.
(* The disappearing facet bug was especially confusing because I'm removing the filters and sql from the table page, at the request of the organization. The filters and sql detail created a lot of confusion for end users who try to find disinfectants used by Hospitals, for instance, as an '=' won't find them, since they are part of the Use_site array.) My disappearing-facet confusion was documented in my own issue: https://github.com/mroswell/list-N/issues/57 (addressed by the plugin). Other facet-related issues here: https://github.com/mroswell/list-N/issues/54 (addressed by the plugin); https://github.com/mroswell/list-N/issues/15 (addressed by template); https://github.com/mroswell/list-N/issues/53 (not yet addressed).
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1302/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
861331159,MDExOlB1bGxSZXF1ZXN0NjE4MDExOTc3,1303,"Update pytest-asyncio requirement from <0.15,>=0.10 to >=0.10,<0.16",27856297,dependabot-preview[bot],closed,0,,,,,1,2021-04-19T13:49:12Z,2021-04-19T18:18:17Z,2021-04-19T18:18:17Z,CONTRIBUTOR,simonw/datasette/pulls/1303,"Updates the requirements on [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) to permit the latest version.
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a ""Dependabot enabled"" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1303/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
791237799,MDU6SXNzdWU3OTEyMzc3OTk=,1196,Access Denied Error in Windows,2826376,QAInsights,open,0,,,,,2,2021-01-21T15:40:40Z,2021-04-14T19:28:38Z,,NONE,,"I am trying to publish a db to vercel. But while issuing the below command throwing `Access Denied` error which is leading to `RecursionError: maximum recursion depth exceeded while calling a Python object`.
I am using PyCharm and Python 3.9. I have reinstalled both and launched PyCharm as Admin in Windows 10. But still the issue persists.
Issued command `datasette publish vercel jmeter.db --project jmeter --install datasette-vega`
PS: localhost is working fine.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1196/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
856895291,MDU6SXNzdWU4NTY4OTUyOTE=,1299,Design better empty states,9599,simonw,open,0,,,,,0,2021-04-13T12:06:12Z,2021-04-13T12:06:12Z,,OWNER,,Inspiration here: https://emptystat.es/,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1299/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
855451460,MDU6SXNzdWU4NTU0NTE0NjA=,1297,"Documentation: json1, and introspection endpoints",192568,mroswell,open,0,,,,,0,2021-04-12T00:38:00Z,2021-04-12T01:29:33Z,,CONTRIBUTOR,,"https://docs.datasette.io/en/stable/facets.html notes that:
> If your SQLite installation provides the json1 extension (you can check using /-/versions) Datasette will automatically detect columns that contain JSON arrays...
When I check -/versions I see two sections relevant to json1:
```
""extensions"": {
""json1"": null
},
""compile_options"": [
...
""ENABLE_JSON1"",
```
The ENABLE_JSON1 makes me think json1 is likely available. But the `""json1"": null` made me think it wasn't available (because of the `null`). It would help if the documentation provided clarity about how to know if json1 is installed. It would also be helpful if the `/-/versions` information signalled somehow that that is to be appended to the hostname or domain name (or whatever you want to call it, or simply show it, using `example.com/-/versions` instead of `/-/versions`. Likewise on that last point, for https://docs.datasette.io/en/stable/introspection.html#introspection , at least at some point on that page detailing where those introspection endpoints go. (Sometimes documentation can be so abbreviated that it's hard for new users to figure out what's going on.)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1297/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
853672224,MDU6SXNzdWU4NTM2NzIyMjQ=,1294,"""You can check out any time you like. But you can never leave!""",192568,mroswell,open,0,,,,,0,2021-04-08T17:02:15Z,2021-04-08T18:35:50Z,,CONTRIBUTOR,,"(Feel free to rename this one.)
- The column gear lets you ""Show not-blank rows."" Then it places a parameter in the URL, which a web developer would notice, but a lot of users won't notice, or know to delete it. Would be good to toggle ""Show not-blank rows"" with ""Show all rows."" (Also would be quite helpful to have a ""Show blank rows | Show all rows"" option)
- The column gear lets you ""Sort ascending"" and ""Sort descending"" but then you're stuck with some sort of sorted version thereafter, unless you know to sort the ID column, or to remove the full _sort parameter and its value in the URL. Would be good to offer a ""Remove sort"" option in the gear.
- These requests are in the same camp as: https://github.com/simonw/datasette-vega/issues/36
- I suspect there are other url parameter instances where similar analysis would be helpful, but the three above are the use cases I've run across.
UPDATE:
- It would be helpful to have a ""Previous page"" available for all but the first table page.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1294/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
520667773,MDU6SXNzdWU1MjA2Njc3NzM=,620,Mechanism for indicating foreign key relationships in the table and query page URLs,9599,simonw,open,0,,,,,6,2019-11-10T22:26:27Z,2021-04-05T03:57:22Z,,OWNER,,"Datasette currently only inflates foreign keys (into names hyperlinks) if it detects them as foreign key constraints in the underlying database.
It would be useful if you could specify additional ""foreign keys"" using both `metadata.json` and the querystring - similar time how you can pass `?_fts_table=x` https://datasette.readthedocs.io/en/stable/full_text_search.html#configuring-full-text-search-for-a-table-or-view",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/620/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 1}",,
672421411,MDU6SXNzdWU2NzI0MjE0MTE=,916,"Support reverse pagination (previous page, has-previous-items)",9599,simonw,open,0,,,,,7,2020-08-04T00:32:06Z,2021-04-03T23:43:11Z,,OWNER,,"I need this for `datasette-graphql` for full compatibility with the way Relay likes to paginate - using cursors for paginating backwards as well as for paginating forwards.
> This may be the kick I need to get Datasette pagination to work in reverse too.
_Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/2#issuecomment-668305853_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/916/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
849396758,MDU6SXNzdWU4NDkzOTY3NTg=,1287,Upgrade to Python 3.9.4,9599,simonw,open,0,,,,,5,2021-04-02T18:43:15Z,2021-04-03T22:38:39Z,,OWNER,,Has some security fixes https://pythoninsider.blogspot.com/2021/04/python-393-and-389-are-now-available.html,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1287/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
453131917,MDU6SXNzdWU0NTMxMzE5MTc=,502,Exporting sqlite database(s)?,7936571,chrismp,closed,0,,,,,3,2019-06-06T16:39:53Z,2021-04-03T05:16:54Z,2019-06-11T18:50:42Z,NONE,,"I'm working on datasette from one computer. But if I want to work on it from another computer and want to copy the SQLite database(s) already on the Heroku datasette instance, how to I copy the database(s) to the second computer so that I can then update it and push to online via datasette's command line code that pushes code to Heroku?",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/502/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
849568079,MDExOlB1bGxSZXF1ZXN0NjA4MzIzMDI4,1290,Use pytest-xdist to speed up tests,9599,simonw,closed,0,,,,,1,2021-04-03T03:34:36Z,2021-04-03T03:42:29Z,2021-04-03T03:42:28Z,OWNER,simonw/datasette/pulls/1290,"Closes #1289, refs #1212.",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1290/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
849543502,MDU6SXNzdWU4NDk1NDM1MDI=,1289,Speed up tests with pytest-xdist,9599,simonw,closed,0,,,,,3,2021-04-03T00:47:39Z,2021-04-03T03:42:28Z,2021-04-03T03:42:28Z,OWNER,,"I think I can get this working for almost every test, then use the pattern in https://github.com/pytest-dev/pytest-xdist/issues/385#issuecomment-444545641 to opt specific tests out of being run in parallel.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1289/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
849512840,MDU6SXNzdWU4NDk1MTI4NDA=,1288,Facets: show counts for null,1111743,jungle-boogie,open,0,,,,,0,2021-04-02T22:33:44Z,2021-04-02T22:33:44Z,,NONE,,"Hi,
Thank you for Datasette and being a fan of SQLite!
Not all rows in a record will always contain data.
So when using a facet on a column where some records have data and others don't, you don't get an accurate count of the results.
Please consider also counting and showing null records with facets.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1288/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
817544251,MDU6SXNzdWU4MTc1NDQyNTE=,1245,"Sticky table column headers would be useful, especially on the query page",9599,simonw,open,0,,,,,1,2021-02-26T17:42:51Z,2021-04-02T20:53:35Z,,OWNER,,Suggestion from office hours.,107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1245/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
826700095,MDU6SXNzdWU4MjY3MDAwOTU=,1255,Facets timing out but work when filtering,1219001,robroc,open,0,,,,,2,2021-03-09T22:01:39Z,2021-04-02T20:50:08Z,,NONE,,"System info:
Windows 10
Datasette 0.55 installed via pip
Python 3.8.5 in a conda environment
I'm getting the message `These facets timed out` on any faceting operation. However, when I apply a filter, the facets appear in the filtered view. The error returns when the filter is removed. My data only has 38,450 rows.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1255/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
847700726,MDU6SXNzdWU4NDc3MDA3MjY=,1285,Feature Request or Plugin Request: Numeric Range Facets,192568,mroswell,open,0,,,,,0,2021-04-01T01:50:20Z,2021-04-01T02:28:19Z,,CONTRIBUTOR,,"It would be great to offer facets for numeric data ranges.
The ranges could pull from typical GIS methods of creating choropleth maps.
https://gisgeography.com/choropleth-maps-data-classification/
Of the following, for mapping, I've always preferred a Jenks Natural Breaks, or a cross between Jenks and Pretty breaks.
- Equal Intervals
- Quantile (equal count)
- Standard Deviation
- Natural Breaks (Jenks) Classification
- Pretty Breaks
- Some sort of Aggregate Jenks Classification (this isn't standard, but it would be nice to be able to set classification ranges that work across tables.)
Here are some links for Natural Breaks, in case this method is unfamiliar.
- https://en.wikipedia.org/wiki/Jenks_natural_breaks_optimization
- http://wiki.gis.com/wiki/index.php/Jenks_Natural_Breaks_Classification
- https://medium.com/analytics-vidhya/jenks-natural-breaks-best-range-finder-algorithm-8d1907192051
Per that last link, there is a Jenks Python module... They also describe it as data-intensive for larger datasets. Maybe this is a good plugin idea.
An example of equal Intervals would be
0 – < 10
10 – < 20
20 – < 30
30 – < 40
It's kind of confusing to have that less-than sign in there. it could also be displayed as:
0 – 10
10 – 20
20 – 30
30 – 40
But then it's not completely clear which category 10 is in, for instance.
(Best to right-justify.. and use an ""en dash"" between numbers.)
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1285/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
841456306,MDU6SXNzdWU4NDE0NTYzMDY=,1276,"Invalid SQL: ""no such table: pragma_database_list"" on database page",1314318,justinallen,closed,0,,,,,7,2021-03-26T00:03:53Z,2021-03-31T16:27:27Z,2021-03-28T23:52:31Z,NONE,,"Don't think this has been covered here yet. I'm a little stumped with this one and can't tell if it's a bug or I have something misconfigured.
Oddly, when running locally the usual list of tables populates (i.e. at /charts a list of tables in charts.db). But when on the web server it throws an Invalid SQL error and ""no such table: pragma_database_list"" below.
All the url endpoints seem to work fine aside from this - individual tables (/charts/chart_one), as well as stored queries (/charts/query_one).
Not sure if this has anything to do with upgrading to Datasette 0.55, or something to do with our setup, which uses a metadata build script similar to [the one for the 538 server](https://github.com/simonw/fivethirtyeight-datasette/blob/main/make_metadata.py), or something else.
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1276/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
843884745,MDU6SXNzdWU4NDM4ODQ3NDU=,1283,advanced #export causes unexpected scrolling,192568,mroswell,open,0,,,,,0,2021-03-29T22:46:57Z,2021-03-29T22:46:57Z,,CONTRIBUTOR,,"1. Visit a datasette table page
2. Click on the ""(advanced)"" link. This adds a fragment identifier ""#export"" to the URL, and scrolls down to the ""Advanced export"" div with the ""export"" id.
3. Manually scroll back up, and click on a suggested facet. The fragment identifier is still present, and the app scrolls back down to the ""Advanced export"" div. I think this is unwanted behavior.
The user remedy seems to be to manually remove the ""#export"" from the URL.
This behavior happens in my project, and in:
https://covid-19.datasettes.com/covid/economist_excess_deaths (for instance)
but not in this table:
https://global-power-plants.datasettes.com/global-power-plants/global-power-plants",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1283/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
843739658,MDExOlB1bGxSZXF1ZXN0NjAzMDgyMjgw,1282,Fix little typo,192568,mroswell,closed,0,,,,,2,2021-03-29T19:45:28Z,2021-03-29T19:57:34Z,2021-03-29T19:57:34Z,CONTRIBUTOR,simonw/datasette/pulls/1282,,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1282/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
576722115,MDU6SXNzdWU1NzY3MjIxMTU=,696,Single failing unit test when run inside the Docker image,9599,simonw,closed,0,,,3268330,Datasette 1.0,2,2020-03-06T06:16:36Z,2021-03-29T17:04:19Z,2021-03-07T07:41:18Z,OWNER,,"```
docker run -it -v `pwd`:/mnt datasetteproject/datasette:latest /bin/bash
root@0e1928cfdf79:/# cd /mnt
root@0e1928cfdf79:/mnt# pip install -e .[test]
root@0e1928cfdf79:/mnt# pytest
```
I get one failure!
It was for `test_searchable[/fixtures/searchable.json?_search=te*+AND+do*&_searchmode=raw-expected_rows3]`
```
def test_searchable(app_client, path, expected_rows):
response = app_client.get(path)
> assert expected_rows == response.json[""rows""]
E AssertionError: assert [[1, 'barry c...sel', 'puma']] == []
E Left contains 2 more items, first extra item: [1, 'barry cat', 'terry dog', 'panther']
E Full diff:
E + []
E - [[1, 'barry cat', 'terry dog', 'panther'],
E - [2, 'terry dog', 'sara weasel', 'puma']]
```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/695#issuecomment-595614469_",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/696/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
724369025,MDExOlB1bGxSZXF1ZXN0NTA1NzY5NDYy,1031,Fallback to databases in inspect-data.json when no -i options are passed,299380,frankier,closed,0,,,,,6,2020-10-19T07:51:06Z,2021-03-29T01:46:45Z,2021-03-29T00:23:41Z,FIRST_TIME_CONTRIBUTOR,simonw/datasette/pulls/1031,Currenlty `Datasette.__init__` checks immutables against None to decide whether to fallback to inspect-data.json. This patch modifies the serve command to pass None when no -i options are passed so this fallback works correctly.,107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1031/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
842881221,MDU6SXNzdWU4NDI4ODEyMjE=,1281,Latest Datasette tags missing from Docker Hub,9599,simonw,closed,0,,,,,7,2021-03-29T00:58:30Z,2021-03-29T01:41:48Z,2021-03-29T01:41:48Z,OWNER,,"Spotted this while testing https://github.com/simonw/datasette/issues/1249#issuecomment-808998719_
https://hub.docker.com/r/datasetteproject/datasette/tags?page=1&ordering=last_updated isn't showing the tags for any version more recent than 0.54.1 - we are up to 0.56 now.
But the `:latest` tag is for the new 0.56 release.",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1281/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
831163537,MDExOlB1bGxSZXF1ZXN0NTkyNTQ4MTAz,1260,Fix: code quality issues,25361949,withshubh,closed,0,,,,,2,2021-03-14T13:56:10Z,2021-03-29T00:22:41Z,2021-03-29T00:22:41Z,NONE,simonw/datasette/pulls/1260,"### Description
Hi :wave: I work at [DeepSource](https://deepsource.io), I ran DeepSource analysis on the forked copy of this repo and found some interesting [code quality issues](https://deepsource.io/gh/withshubh/datasette/issues/?category=recommended) in the codebase, opening this PR so you can assess if our platform is right and helpful for you.
### Summary of changes
- Replaced ternary syntax with if expression
- Removed redundant `None` default
- Used `is` to compare type of objects
- Iterated dictionary directly
- Removed unnecessary lambda expression
- Refactored unnecessary `else` / `elif` when `if` block has a `return` statement
- Refactored unnecessary `else` / `elif` when `if` block has a `raise` statement
- Added .deepsource.toml to continuously analyze and detect code quality issues",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1260/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
807433181,MDU6SXNzdWU4MDc0MzMxODE=,1224,can't start immutable databases from configuration dir mode,295329,camallen,closed,0,,,,,0,2021-02-12T17:50:13Z,2021-03-29T00:17:31Z,2021-03-29T00:17:31Z,CONTRIBUTOR,,"Say I have a `/databases/` directory with multiple sqlite db files in that dir (`1.db` & `2.db`) and an `inspect-data.json` file.
If I start datasette via `datasette -h 0.0.0.0 /databases/` then the resulting databases are set to `is_mutable: true` as inspected via http://127.0.0.1:8001/-/databases.json
I don't want to have to list out the databases by name, e.g. `datasette -i /databases/1.db -i /databases/2.db` as i want the system to autodetect the sqlite dbs i have in the configuration directory
According to the docs outlined in https://docs.datasette.io/en/latest/settings.html?highlight=immutable#configuration-directory-mode this should be possible
> `inspect-data.json` the result of running datasette inspect - any database files listed here will be treated as immutable, so they should not be changed while Datasette is running
I believe that if the `inspect-json.json` file present, then in theory the databases will be automatically set to immutable via this code https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L211-L216
However it appears the Click Multiple Options will return a tuple via https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/cli.py#L311-L317
The resulting tuple is passed to the Datasette app via `kwargs` and overrides the behaviour to set the databases to immutable via this arg https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L182
If you think this is a bug and needs fixing, I am willing to make a PR to check for the empty `immutable` tuple before calling the Datasette class initializer as I think leaving that class interface alone is the best path here.
Thoughts?
Also - i'm loving Datasette, it truly is a wonderful tool, thank you :)",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1224/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
763207948,MDU6SXNzdWU3NjMyMDc5NDg=,1141,Default styling for bullet point lists,9599,simonw,closed,0,,,,,0,2020-12-12T02:49:33Z,2021-03-29T00:14:05Z,2021-03-29T00:14:05Z,OWNER,,"I just noticed that https://datasette.io/content/recent_releases (which uses `datasette-render-markdown`) is missing its bullet points:
",107914493,datasette,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1141/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
825217564,MDExOlB1bGxSZXF1ZXN0NTg3MzMyNDcz,1252,Add back styling to lists within table cells (fixes #1141),7476523,bobwhitelock,closed,0,,,,,2,2021-03-09T03:00:57Z,2021-03-29T00:14:04Z,2021-03-29T00:14:04Z,CONTRIBUTOR,simonw/datasette/pulls/1252,"This overrides the Datasette reset - see https://github.com/simonw/datasette/blob/d0fd833b8cdd97e1b91d0f97a69b494895d82bee/datasette/static/app.css#L35-L38 - to add back the default styling of list items displayed within Datasette table cells.
Following this change, the same content as in the original issue looks like this:
![2021-03-09_02:57:32](https://user-images.githubusercontent.com/7476523/110411982-63e5ae80-8083-11eb-9b5c-e5dc825073e2.png)
",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1252/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
842556944,MDExOlB1bGxSZXF1ZXN0NjAyMTA3OTM1,1279,Minor Docs Update. Added `--app` to fly install command.,1019791,koaning,closed,0,,,,,2,2021-03-27T16:58:08Z,2021-03-29T00:11:55Z,2021-03-29T00:11:55Z,CONTRIBUTOR,simonw/datasette/pulls/1279,"Without this flag, there's an error locally.
```
> datasette publish fly bigmac.db
Usage: datasette publish fly [OPTIONS] [FILES]...
Try 'datasette publish fly --help' for help.
Error: Missing option '-a' / '--app'.
```
I also got an error message which later turned out to be because I hadn't added my credit card information yet to `fly`. I wasn't sure if I should add that mention to the docs here, or to submit a bug-report over at https://github.com/simonw/datasette-publish-fly. ",107914493,datasette,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1279/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
741862364,MDU6SXNzdWU3NDE4NjIzNjQ=,1090,Custom widgets for canned query forms,9599,simonw,open,0,,,,,3,2020-11-12T19:21:07Z,2021-03-27T16:25:25Z,,OWNER,,"This is an idea that was cut from the first version of writable canned queries:
> I really want the option to use a `