html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/datasette/issues/2143#issuecomment-1690787394,https://api.github.com/repos/simonw/datasette/issues/2143,1690787394,IC_kwDOBm6k_c5kx15C,9599,2023-08-23T23:52:02Z,2023-08-23T23:52:02Z,OWNER,"> This also makes it simple to separate out secrets.
>
> `datasette --config settings.yaml --config secrets.yaml --config db-docs.yaml --config db-fixtures.yaml`
Having multiple configs that combine in that way is a really interesting direction.
> To chime in from a poweruser perspective: I'm worried that this is an overengineering trap. Yes, the current solution is somewhat messy. But there are datasette-wide settings, there are database-scope settings, there are table-scope settings etc, but then there are database-scope metadata and table-scope metadata. Trying to cleanly separate ""settings"" from ""configuration"" is, I believe, an uphill fight.
I'm very keen on separating out the ""metadata"" - where metadata is the slimmest possible set of things, effectively the data license and the source and the column and table descriptions - from everything else, mainly because I want metadata to be able to travel with the data.
One idea that's been discussed before is having an optional mechanism for storing metadata in the SQLite database file itself - potentially in a `_datasette_metadata` table. That way you could distribute a DB file and anyone who opened it in Datasette would also see the correct metadata about it.
That's why I'm so keen on splitting out metadata from all of the other stuff - settings and plugin configuration and authentication rules.
So really it becomes ""true metadata"" v.s. ""all of the other junk that's accumulated in metadata and `settings.json`"".","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427,
https://github.com/simonw/datasette/issues/2102#issuecomment-1690705243,https://api.github.com/repos/simonw/datasette/issues/2102,1690705243,IC_kwDOBm6k_c5kxh1b,9599,2023-08-23T22:03:54Z,2023-08-23T22:03:54Z,OWNER,Idea: `datasette-permissions-debug` plugin which simply prints out a stacktrace for every permission check so you can see where in the code they are.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818,
https://github.com/simonw/datasette/issues/2102#issuecomment-1690703764,https://api.github.com/repos/simonw/datasette/issues/2102,1690703764,IC_kwDOBm6k_c5kxheU,9599,2023-08-23T22:02:14Z,2023-08-23T22:02:14Z,OWNER,"Built this new test:
```python
@pytest.mark.asyncio
async def test_view_table_token_can_access_table(perms_ds):
actor = {
""id"": ""restricted-token"",
""token"": ""dstok"",
# Restricted to just view-table on perms_ds_two/t1
""_r"": {""r"": {""perms_ds_two"": {""t1"": [""vt""]}}},
}
cookies = {""ds_actor"": perms_ds.client.actor_cookie(actor)}
response = await perms_ds.client.get(""/perms_ds_two/t1.json"", cookies=cookies)
assert response.status_code == 200
```
The test fails. Running it with `pytest --pdb` let me do this:
```
(Pdb) from pprint import pprint
(Pdb) pprint(perms_ds._permission_checks)
deque([{'action': 'view-table',
'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}},
'id': 'restricted-token',
'token': 'dstok'},
'resource': ('perms_ds_two', 't1'),
'result': None,
'used_default': True,
'when': '2023-08-23T21:59:45.117155'},
{'action': 'view-database',
'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}},
'id': 'restricted-token',
'token': 'dstok'},
'resource': 'perms_ds_two',
'result': False,
'used_default': False,
'when': '2023-08-23T21:59:45.117189'},
{'action': 'view-instance',
'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}},
'id': 'restricted-token',
'token': 'dstok'},
'resource': None,
'result': False,
'used_default': False,
'when': '2023-08-23T21:59:45.126751'},
{'action': 'debug-menu',
'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}},
'id': 'restricted-token',
'token': 'dstok'},
'resource': None,
'result': False,
'used_default': False,
'when': '2023-08-23T21:59:45.126777'}],
maxlen=200)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818,
https://github.com/simonw/datasette/issues/2102#issuecomment-1690693830,https://api.github.com/repos/simonw/datasette/issues/2102,1690693830,IC_kwDOBm6k_c5kxfDG,9599,2023-08-23T21:51:52Z,2023-08-23T21:52:58Z,OWNER,"This is the hook in question: https://github.com/simonw/datasette/blob/bdf59eb7db42559e538a637bacfe86d39e5d17ca/datasette/hookspecs.py#L108-L110
- `True` means they are allowed to access it. You only need a single`True` from a plugin to allow it.
- `False` means they are not, and just one `False` from a plugin will deny it (even if another one returned `True` I think)
- `None` means that the plugin has no opinion on this question.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818,
https://github.com/simonw/datasette/issues/950#issuecomment-680262437,https://api.github.com/repos/simonw/datasette/issues/950,680262437,MDEyOklzc3VlQ29tbWVudDY4MDI2MjQzNw==,9599,2020-08-25T20:49:24Z,2023-08-23T21:34:09Z,OWNER,"The alternative to this would be to use regular databases and control access to them using [Authentication and permissions](https://docs.datasette.io/en/stable/authentication.html). My concern there is that it's just too easy for someone to mess up their configuration, which would be really bad. I like the idea of a much stronger defense mechanism specifically designed for secrets that should not be exposed.
Outside of secrets, passwords and tokens this mechanism could also be useful for the use-case of using Datasette to power websites - as seen on https://www.niche-museums.com/ and https://www.rockybeaches.com/ - maybe those sites don't want to expose their data through their API but still want to use `datasette-template-sql` and the `graphql()` template tag in `datasette-graphql` to render data.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",685806511,
https://github.com/simonw/datasette/issues/2150#issuecomment-1690438270,https://api.github.com/repos/simonw/datasette/issues/2150,1690438270,IC_kwDOBm6k_c5kwgp-,9599,2023-08-23T18:27:47Z,2023-08-23T18:27:47Z,OWNER,"I added `outline: 3px dotted pink` to that `form label` block to help spot where else it's being used.
Oh interesting, looks like it's over-ridden here too:
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783,
https://github.com/simonw/datasette/issues/2150#issuecomment-1690435866,https://api.github.com/repos/simonw/datasette/issues/2150,1690435866,IC_kwDOBm6k_c5kwgEa,9599,2023-08-23T18:25:51Z,2023-08-23T18:25:51Z,OWNER,"Looks like that affects a few forms:
![]()
The search form on the table page over-rides it already:
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783,
https://github.com/simonw/datasette/issues/2150#issuecomment-1690432928,https://api.github.com/repos/simonw/datasette/issues/2150,1690432928,IC_kwDOBm6k_c5kwfWg,9599,2023-08-23T18:23:26Z,2023-08-23T18:23:26Z,OWNER,"That should be scoped just to the query filters form on the table page.
I'll fix it in `main` but I'm still going to ship a fix for those plugins separately so they work well before people upgrade to the next Datasette release with this change in it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783,
https://github.com/simonw/datasette/issues/2150#issuecomment-1690431509,https://api.github.com/repos/simonw/datasette/issues/2150,1690431509,IC_kwDOBm6k_c5kwfAV,9599,2023-08-23T18:22:47Z,2023-08-23T18:22:47Z,OWNER,https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/static/app.css#L485-L489,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783,
https://github.com/simonw/datasette/issues/2091#issuecomment-1690423878,https://api.github.com/repos/simonw/datasette/issues/2091,1690423878,IC_kwDOBm6k_c5kwdJG,9599,2023-08-23T18:18:18Z,2023-08-23T18:18:18Z,OWNER,"Dupe of:
- #2097 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1781022369,
https://github.com/simonw/datasette/issues/2123#issuecomment-1689207309,https://api.github.com/repos/simonw/datasette/issues/2123,1689207309,IC_kwDOBm6k_c5kr0IN,9599,2023-08-23T03:07:27Z,2023-08-23T03:07:27Z,OWNER,"> I'm happy to debug and land a patch if it's welcome.
Yes please! What an odd bug.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1825007061,
https://github.com/simonw/datasette/issues/2147#issuecomment-1689206768,https://api.github.com/repos/simonw/datasette/issues/2147,1689206768,IC_kwDOBm6k_c5krz_w,9599,2023-08-23T03:06:32Z,2023-08-23T03:06:32Z,OWNER,"I'm less convinced by the ""rewrite the query in some way"" optional idea. What kind of use-cases can you imagine for that?
My hunch is that it's much more likely to cause weird breakages than it is to allow for useful plugin extensions, but I'm willing to be convinced otherwise.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1858228057,
https://github.com/simonw/datasette/issues/2147#issuecomment-1689206170,https://api.github.com/repos/simonw/datasette/issues/2147,1689206170,IC_kwDOBm6k_c5krz2a,9599,2023-08-23T03:05:32Z,2023-08-23T03:05:32Z,OWNER,"Interestingly enough there's actually a mechanism that looks like that a bit already: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/views/database.py#L496-L508
That `validate_sql_select()` function is defined here: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/utils/__init__.py#L256-L265
Against these constants:
https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/utils/__init__.py#L223-L253
Which isn't a million miles away from your suggestion to have a hook that can say if the query should be executed or not.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1858228057,
https://github.com/simonw/datasette/pull/2148#issuecomment-1689198368,https://api.github.com/repos/simonw/datasette/issues/2148,1689198368,IC_kwDOBm6k_c5krx8g,9599,2023-08-23T02:57:53Z,2023-08-23T02:57:53Z,OWNER,@dependabot rebase,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334,
https://github.com/simonw/datasette/pull/2148#issuecomment-1689177556,https://api.github.com/repos/simonw/datasette/issues/2148,1689177556,IC_kwDOBm6k_c5krs3U,9599,2023-08-23T02:44:29Z,2023-08-23T02:44:29Z,OWNER,"Simplest possible solution is to only run the `pip install .[docs]` bit under Python 3.9+, ditto for the docs tests. I think I'll try that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334,
https://github.com/simonw/datasette/pull/2148#issuecomment-1689175062,https://api.github.com/repos/simonw/datasette/issues/2148,1689175062,IC_kwDOBm6k_c5krsQW,9599,2023-08-23T02:40:46Z,2023-08-23T02:40:46Z,OWNER,"Here's why the tests are failing:
https://github.com/simonw/datasette/blob/2ce7872e3ba8d07248c194ef554bbdc1df510f32/.github/workflows/test.yml#L30-L46
It looks like those tests don't actually need Sphinx installed - they install `pip install -e '.[test,docs]'` to get the other docs dependencies:
https://github.com/simonw/datasette/blob/2ce7872e3ba8d07248c194ef554bbdc1df510f32/setup.py#L70-L80","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334,
https://github.com/simonw/datasette/pull/2148#issuecomment-1689173748,https://api.github.com/repos/simonw/datasette/issues/2148,1689173748,IC_kwDOBm6k_c5krr70,9599,2023-08-23T02:38:31Z,2023-08-23T02:38:31Z,OWNER,"Sphinx dropped support for Python 3.8 in their version 7.2.0:
- https://www.sphinx-doc.org/en/master/changes.html#release-7-2-0-released-aug-17-2023
- https://www.sphinx-doc.org/en/master/internals/release-process.html#python-version-support-policy
- https://github.com/sphinx-doc/sphinx/pull/11511
- ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334,
https://github.com/simonw/datasette/issues/516#issuecomment-1689159200,https://api.github.com/repos/simonw/datasette/issues/516,1689159200,IC_kwDOBm6k_c5kroYg,9599,2023-08-23T02:15:36Z,2023-08-23T02:15:36Z,OWNER,This could play havoc with unmerged PRs. I should merge any big ones (like the JavaScript plugins work) first.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459509126,
https://github.com/simonw/datasette/issues/516#issuecomment-1689158712,https://api.github.com/repos/simonw/datasette/issues/516,1689158712,IC_kwDOBm6k_c5kroQ4,9599,2023-08-23T02:14:45Z,2023-08-23T02:14:45Z,OWNER,"Thinking about this again today. Posted about it on Discord: https://discord.com/channels/823971286308356157/823971286941302908/1143729300349132933
I won't enforce it in a `pytest` test, I'll enforce it with a CI lint check that's also in the `Justfile` here instead:
https://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/Justfile#L19-L23","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459509126,
https://github.com/simonw/datasette/issues/516#issuecomment-1689154837,https://api.github.com/repos/simonw/datasette/issues/516,1689154837,IC_kwDOBm6k_c5krnUV,9599,2023-08-23T02:08:33Z,2023-08-23T02:08:50Z,OWNER,Browse this commit to see the result: https://github.com/simonw/datasette/tree/59a5d336bd4336bc53103922ada4bf726f4336c9,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459509126,
https://github.com/simonw/datasette/issues/516#issuecomment-1689153446,https://api.github.com/repos/simonw/datasette/issues/516,1689153446,IC_kwDOBm6k_c5krm-m,9599,2023-08-23T02:06:35Z,2023-08-23T02:06:35Z,OWNER,"I just tried this again today by dropping this into `.isort.cfg`:
```ini
[settings]
multi_line_output=3
include_trailing_comma=True
force_grid_wrap=0
use_parentheses=True
line_length=88
known_first_party=datasette
```
And running this in the root of the project:
```
isort .
```
It produced a huge diff, but when I ran the tests:
```bash
pytest -n auto
```
The tests all passed. I'll push a PR.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459509126,
https://github.com/simonw/datasette/pull/524#issuecomment-1689130061,https://api.github.com/repos/simonw/datasette/issues/524,1689130061,IC_kwDOBm6k_c5krhRN,9599,2023-08-23T01:31:08Z,2023-08-23T01:31:08Z,OWNER,This branch is WAY out of date.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459689615,
https://github.com/simonw/datasette/issues/493#issuecomment-1689128911,https://api.github.com/repos/simonw/datasette/issues/493,1689128911,IC_kwDOBm6k_c5krg_P,9599,2023-08-23T01:29:20Z,2023-08-23T01:29:20Z,OWNER,"It's going to be called `datasette.json` and the concept of metadata will be split out separately. See:
- #2149 ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",449886319,
https://github.com/simonw/datasette/issues/275#issuecomment-1689128553,https://api.github.com/repos/simonw/datasette/issues/275,1689128553,IC_kwDOBm6k_c5krg5p,9599,2023-08-23T01:28:37Z,2023-08-23T01:28:37Z,OWNER,"This is obsoleted by the work happening here:
- #2093 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",324720095,
https://github.com/simonw/datasette/pull/2148#issuecomment-1689127479,https://api.github.com/repos/simonw/datasette/issues/2148,1689127479,IC_kwDOBm6k_c5krgo3,9599,2023-08-23T01:26:53Z,2023-08-23T01:26:53Z,OWNER,@dependabot recreate,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334,
https://github.com/simonw/datasette/pull/2149#issuecomment-1689125244,https://api.github.com/repos/simonw/datasette/issues/2149,1689125244,IC_kwDOBm6k_c5krgF8,9599,2023-08-23T01:23:27Z,2023-08-23T01:23:27Z,OWNER,"This is a really great start - tests pass, implementation looks clean, the new stubbed documentation page makes sense. Let's land it in `main` and iterate on it further in future PRs.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1861812208,