html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app https://github.com/simonw/sqlite-utils/pull/591#issuecomment-1708695907,https://api.github.com/repos/simonw/sqlite-utils/issues/591,1708695907,IC_kwDOCGYnMM5l2KFj,9599,2023-09-06T16:15:59Z,2023-09-06T16:19:14Z,OWNER,"The test failure was while installing `numpy`, relating to importing `distutils` - maybe relevant: - https://github.com/pypa/setuptools/issues/3661 ``` 25h Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' ERROR: Exception: Traceback (most recent call last): ... File ""/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_internal/utils/misc.py"", line 697, in get_requires_for_build_wheel return super().get_requires_for_build_wheel(config_settings=cs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py"", line 166, in get_requires_for_build_wheel return self._call_hook('get_requires_for_build_wheel', { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ""/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py"", line 321, in _call_hook raise BackendUnavailable(data.get('traceback', '')) pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last): File ""/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py"", line 77, in _build_backend obj = import_module(mod_path) ^^^^^^^^^^^^^^^^^^^^^^^ File ""/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/importlib/__init__.py"", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File """", line 1381, in _gcd_import File """", line 1354, in _find_and_load File """", line 1304, in _find_and_load_unlocked File """", line 488, in _call_with_frames_removed File """", line 1381, in _gcd_import File """", line 1354, in _find_and_load File """", line 1325, in _find_and_load_unlocked File """", line 929, in _load_unlocked File """", line 994, in exec_module File """", line 488, in _call_with_frames_removed File ""/tmp/pip-build-env-x9nyg3kd/overlay/lib/python3.12/site-packages/setuptools/__init__.py"", line 10, in import distutils.core ModuleNotFoundError: No module named 'distutils' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1884335789, https://github.com/simonw/datasette/pull/2166#issuecomment-1701045404,https://api.github.com/repos/simonw/datasette/issues/2166,1701045404,IC_kwDOBm6k_c5lY-Sc,22429695,2023-08-31T13:31:15Z,2023-09-06T13:25:17Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2166?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`05707aa`)](https://app.codecov.io/gh/simonw/datasette/commit/05707aa16b5c6c39fbe48b3176b85a8ffe493938?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.68% compared to head [(`7c9df6e`)](https://app.codecov.io/gh/simonw/datasette/pull/2166?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.68%.
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2166 +/- ## ======================================= Coverage 92.68% 92.68% ======================================= Files 40 40 Lines 6012 6012 ======================================= Hits 5572 5572 Misses 440 440 ```
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2166?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1875519316, https://github.com/simonw/datasette/pull/2173#issuecomment-1707570044,https://api.github.com/repos/simonw/datasette/issues/2173,1707570044,IC_kwDOBm6k_c5lx3N8,22429695,2023-09-06T02:39:59Z,2023-09-06T02:49:01Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2173?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`fd083e3`)](https://app.codecov.io/gh/simonw/datasette/commit/fd083e37ec53e7e625111168d324a572344a3b19?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.68% compared to head [(`a481ebd`)](https://app.codecov.io/gh/simonw/datasette/pull/2173?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.68%. > Report is 1 commits behind head on main.
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2173 +/- ## ======================================= Coverage 92.68% 92.68% ======================================= Files 40 40 Lines 6012 6012 ======================================= Hits 5572 5572 Misses 440 440 ```
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2173?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1883055640, https://github.com/simonw/datasette/pull/2173#issuecomment-1707570378,https://api.github.com/repos/simonw/datasette/issues/2173,1707570378,IC_kwDOBm6k_c5lx3TK,9599,2023-09-06T02:40:31Z,2023-09-06T02:40:31Z,OWNER,"Surprising error: ``` blacken-docs -l 60 docs/*.rst shell: /usr/bin/bash -e {0} env: pythonLocation: /opt/hostedtoolcache/Python/3.11.4/x64 PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.4/x64/lib/pkgconfig Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64 Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64 Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64 LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.4/x64/lib docs/writing_plugins.rst:365: code block parse error Cannot parse: 8:0: ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1883055640, https://github.com/simonw/datasette/pull/2173#issuecomment-1707565495,https://api.github.com/repos/simonw/datasette/issues/2173,1707565495,IC_kwDOBm6k_c5lx2G3,9599,2023-09-06T02:33:56Z,2023-09-06T02:33:56Z,OWNER,Running tests to see if Pyodide works - that was the reason I switched to my `click-default-group-wheel` package originally.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1883055640, https://github.com/simonw/datasette/issues/328#issuecomment-1706701195,https://api.github.com/repos/simonw/datasette/issues/328,1706701195,IC_kwDOBm6k_c5lujGL,7983005,2023-09-05T14:10:39Z,2023-09-05T14:10:39Z,NONE,"Hey @simonw I hit the same issue as mentionned by @chmaynard on a fresh install, ""/mnt/fixtures.db"" doesn't seem to exist in the docker image","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",336464733, https://github.com/simonw/sqlite-utils/issues/590#issuecomment-1704387161,https://api.github.com/repos/simonw/sqlite-utils/issues/590,1704387161,IC_kwDOCGYnMM5lluJZ,9599,2023-09-03T19:50:36Z,2023-09-03T19:50:36Z,OWNER,"Maybe just populate `db.memory: bool` and `db.memory_name: Optional[str]` for this, then document them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1879214365, https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704384393,https://api.github.com/repos/simonw/sqlite-utils/issues/589,1704384393,IC_kwDOCGYnMM5llteJ,9599,2023-09-03T19:36:34Z,2023-09-03T19:36:34Z,OWNER,"Here's a prototype: https://github.com/simonw/sqlite-utils/commit/62f673835c4a66f87cf6f949eaff43c8b014619b Still needs tests and documentation (and some more thought to make sure it's doing the right thing).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1879209560, https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704384111,https://api.github.com/repos/simonw/sqlite-utils/issues/589,1704384111,IC_kwDOCGYnMM5lltZv,9599,2023-09-03T19:35:03Z,2023-09-03T19:35:03Z,OWNER,"Normally in Python/`sqlite3` you de-register a function by passing `None` to it. You can't do that with `db.register_function()` at the moment because a `fn` of `None` does something else: https://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L461-L464","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1879209560, https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704383901,https://api.github.com/repos/simonw/sqlite-utils/issues/589,1704383901,IC_kwDOCGYnMM5lltWd,9599,2023-09-03T19:34:05Z,2023-09-03T19:34:05Z,OWNER,"For that particular case I realized I'd quite like to have a mechanism for applying functions for a block of code and then de-registering them at the end - a context manager. I played with this idea a bit: ```python with db.register_functions(md5, md5_random): db.query(...) ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1879209560, https://github.com/simonw/datasette/issues/1765#issuecomment-1701894468,https://api.github.com/repos/simonw/datasette/issues/1765,1701894468,IC_kwDOBm6k_c5lcNlE,9599,2023-08-31T23:08:24Z,2023-08-31T23:08:24Z,OWNER,https://docs.datasette.io/en/latest/writing_plugins.html#plugins-that-define-new-plugin-hooks,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1292370469, https://github.com/simonw/datasette/issues/2168#issuecomment-1701831013,https://api.github.com/repos/simonw/datasette/issues/2168,1701831013,IC_kwDOBm6k_c5lb-Fl,9599,2023-08-31T21:51:12Z,2023-08-31T21:52:15Z,OWNER,"Need to make sure the design of this takes streaming responses into account. Those could be pretty tricky here. I nice thing about `asgi_wrapper()` is that it handles those already.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1876353656, https://github.com/simonw/datasette/issues/2168#issuecomment-1701830241,https://api.github.com/repos/simonw/datasette/issues/2168,1701830241,IC_kwDOBm6k_c5lb95h,9599,2023-08-31T21:50:18Z,2023-08-31T21:50:18Z,OWNER,"The hook could be called `register_middleware()` and could work like `register_routes()` and `register_commands()`: ```python @hookspec def register_middleware(datasette): """"""Register middleware: returns a list of async def middleware functions"""""" ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1876353656, https://github.com/simonw/datasette/issues/2168#issuecomment-1701828197,https://api.github.com/repos/simonw/datasette/issues/2168,1701828197,IC_kwDOBm6k_c5lb9Zl,9599,2023-08-31T21:48:00Z,2023-08-31T21:48:57Z,OWNER,"A pattern like this could be interesting: ```python async def my_middleware(datasette, request, get_response): # Mess with request here if neccessary response = await get_response(request) # mess with response return response ``` The Django pattern is more complicated but does have that mechanism for running one-time configuration prior to defining the `middleware()` function, which is neat.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1876353656, https://github.com/simonw/datasette/issues/2168#issuecomment-1701826521,https://api.github.com/repos/simonw/datasette/issues/2168,1701826521,IC_kwDOBm6k_c5lb8_Z,9599,2023-08-31T21:46:13Z,2023-08-31T21:46:13Z,OWNER,"This could even be a pair of hooks - `process_request()` and `process_response()`. Or could take a leaf from Django, which redesigned middleware to use this pattern instead: ```python def simple_middleware(get_response): # One-time configuration and initialization. def middleware(request): # Code to be executed for each request before # the view (and later middleware) are called. response = get_response(request) # Code to be executed for each request/response after # the view is called. return response return middleware ``` Or even borrow an idea from `pytest` where fixtures can `yield` in the middle, like this: ```python @pytest.fixture def sending_user(mail_admin): user = mail_admin.create_user() yield user mail_admin.delete_user(user) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1876353656, https://github.com/simonw/datasette/issues/2168#issuecomment-1701823609,https://api.github.com/repos/simonw/datasette/issues/2168,1701823609,IC_kwDOBm6k_c5lb8R5,9599,2023-08-31T21:43:06Z,2023-08-31T21:44:13Z,OWNER,"Not sure what to call this. Maybe `app_wrapper()`? Or perhaps it's simpler than that, something like this: ```python @hookspec def process_response(datasette, request, response): """"""Last chance to modify the response before it is returned to the client"""""" ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1876353656, https://github.com/simonw/datasette/issues/2157#issuecomment-1700291967,https://api.github.com/repos/simonw/datasette/issues/2157,1700291967,IC_kwDOBm6k_c5lWGV_,15178711,2023-08-31T02:45:56Z,2023-08-31T02:45:56Z,CONTRIBUTOR,"@simonw what do you think about adding a `DATASETTE_INTERNAL_DB_PATH` env variable, where when defined, is the default location of the internal DB? This means when the `--internal` flag is NOT provided, Datasette would check to see if `DATASETTE_INTERNAL_DB_PATH` exists, and if so, uses that as the internal database (and would fallback to an ephemeral memory database) My rationale: some plugins may require, or strongly encourage, a persistent internal database (`datasette-comments`, `datasette-bookmarks`, `datasette-link-shortener`, etc.). However, for users that have a global installation of Datasette (say from `brew install` or a global `pip install`), it would be annoying having to specify `--internal` every time. So instead, they can just add `export DATASETTE_INTERNAL_DB_PATH=""/path/to/internal.db""` to their bashrc/zshrc/whereever to not have to worry about `--internal`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865869205, https://github.com/simonw/datasette/issues/1092#issuecomment-1699926384,https://api.github.com/repos/simonw/datasette/issues/1092,1699926384,IC_kwDOBm6k_c5lUtFw,9599,2023-08-30T22:17:42Z,2023-08-30T22:17:42Z,OWNER,"This is implemented now: - https://docs.datasette.io/en/stable/internals.html#await-ensure-permissions-actor-permissions","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",742041667, https://github.com/simonw/datasette/issues/1190#issuecomment-1699925224,https://api.github.com/repos/simonw/datasette/issues/1190,1699925224,IC_kwDOBm6k_c5lUszo,9599,2023-08-30T22:16:38Z,2023-08-30T22:16:38Z,OWNER,"This is going to happen in this tool instead: - https://github.com/simonw/dclient","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",787098146, https://github.com/simonw/datasette/pull/2165#issuecomment-1699910555,https://api.github.com/repos/simonw/datasette/issues/2165,1699910555,IC_kwDOBm6k_c5lUpOb,9599,2023-08-30T22:05:14Z,2023-08-30T22:05:14Z,OWNER,Documentation preview: https://github.com/simonw/datasette/blob/6321c9c055a640ed6ea98e231dc5813dcde1f773/docs/plugins.rst#controlling-which-plugins-are-loaded,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1874327336, https://github.com/simonw/datasette/pull/2165#issuecomment-1699812599,https://api.github.com/repos/simonw/datasette/issues/2165,1699812599,IC_kwDOBm6k_c5lURT3,22429695,2023-08-30T20:43:19Z,2023-08-30T22:05:05Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2165?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`26.66%`** and project coverage change: **`-0.17%`** :warning: > Comparison is base [(`30b28c8`)](https://app.codecov.io/gh/simonw/datasette/commit/30b28c8367a9c6870386ea10a202705b40862457?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.84% compared to head [(`5eddf34`)](https://app.codecov.io/gh/simonw/datasette/pull/2165?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.68%. > :exclamation: Current head 5eddf34 differs from pull request most recent head 6321c9c. Consider uploading reports for the commit 6321c9c to get more accurate results
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2165 +/- ## ========================================== - Coverage 92.84% 92.68% -0.17% ========================================== Files 40 40 Lines 5998 6012 +14 ========================================== + Hits 5569 5572 +3 - Misses 429 440 +11 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2165?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/plugins.py](https://app.codecov.io/gh/simonw/datasette/pull/2165?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3BsdWdpbnMucHk=) | `64.58% <26.66%> (-17.77%)` | :arrow_down: |
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2165?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1874327336, https://github.com/simonw/datasette/pull/2165#issuecomment-1699884314,https://api.github.com/repos/simonw/datasette/issues/2165,1699884314,IC_kwDOBm6k_c5lUi0a,9599,2023-08-30T21:38:29Z,2023-08-30T21:39:51Z,OWNER,"Here's the reason for that name disparity: https://github.com/simonw/datasette/blob/30b28c8367a9c6870386ea10a202705b40862457/datasette/plugins.py#L54-L65 Note how the `distinfo.project_name` name is used when available. That seems to work for regularly installed plugins but not for plugins loaded via `DATASETTE_LOAD_PLUGINS`. And that's looking things up in `plugin_to_distinfo` which is populated here: https://github.com/simonw/datasette/blob/30b28c8367a9c6870386ea10a202705b40862457/datasette/plugins.py#L37","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1874327336, https://github.com/simonw/datasette/pull/2165#issuecomment-1699811810,https://api.github.com/repos/simonw/datasette/issues/2165,1699811810,IC_kwDOBm6k_c5lURHi,9599,2023-08-30T20:42:41Z,2023-08-30T20:42:41Z,OWNER,"The `load_setuptools_entrypoints()` function in Pluggy [does this](https://github.com/pytest-dev/pluggy/blob/0b41c9766508a46ae666cf281684df3164b3e2a9/src/pluggy/_manager.py#L376): ```python for ep in dist.entry_points: if ( ep.group != group or (name is not None and ep.name != name) # already registered or self.get_plugin(ep.name) or self.is_blocked(ep.name) ): continue plugin = ep.load() self.register(plugin, name=ep.name) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1874327336, https://github.com/simonw/datasette/pull/2165#issuecomment-1699809688,https://api.github.com/repos/simonw/datasette/issues/2165,1699809688,IC_kwDOBm6k_c5lUQmY,9599,2023-08-30T20:41:06Z,2023-08-30T20:41:06Z,OWNER,"Slight weirdness: I noticed that the output from the `datasette plugins` command looks like this for plugins loaded with the new environment variable: ```json { ""name"": ""datasette_pretty_json"", ""static"": false, ""templates"": false, ""version"": null, ""hooks"": [ ""render_cell"" ] }, ``` That should ideally be `datasette-pretty-json`, not `datasette_pretty_json`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1874327336, https://github.com/simonw/datasette/pull/2165#issuecomment-1699802028,https://api.github.com/repos/simonw/datasette/issues/2165,1699802028,IC_kwDOBm6k_c5lUOus,9599,2023-08-30T20:35:02Z,2023-08-30T20:35:02Z,OWNER,"Testing this is going to be a bit of a pain. I think I'll add a whole separate test block to CI which installs a couple of plugins and then exercises this feature using `datasette plugins`. I'll use `datasette-init` and `datasette-json-html` just because they are small and simple.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1874327336, https://github.com/simonw/datasette/issues/2164#issuecomment-1699728102,https://api.github.com/repos/simonw/datasette/issues/2164,1699728102,IC_kwDOBm6k_c5lT8rm,9599,2023-08-30T19:35:32Z,2023-08-30T19:35:47Z,OWNER,"Figured this out so far with the help of GPT-4: https://chat.openai.com/share/0e785865-621b-4fb3-ba05-7449e57c8496 Now this works: ```bash DATASETTE_LOAD_PLUGINS=datasette-write-ui datasette plugins ``` ```json [ { ""name"": ""datasette_write_ui"", ""static"": true, ""templates"": true, ""version"": null, ""hooks"": [ ""extra_template_vars"", ""register_routes"" ] } ] ``` Or multiple plugins: ```bash DATASETTE_LOAD_PLUGINS=datasette-write-ui,datasette-pretty-json datasette plugins ``` Outputs: ```json [ { ""name"": ""datasette_pretty_json"", ""static"": false, ""templates"": false, ""version"": null, ""hooks"": [ ""render_cell"" ] }, { ""name"": ""datasette_write_ui"", ""static"": true, ""templates"": true, ""version"": null, ""hooks"": [ ""extra_template_vars"", ""register_routes"" ] } ] ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1874255116, https://github.com/simonw/datasette/issues/2163#issuecomment-1697818917,https://api.github.com/repos/simonw/datasette/issues/2163,1697818917,IC_kwDOBm6k_c5lMqkl,9599,2023-08-29T16:56:20Z,2023-08-29T16:56:20Z,OWNER,https://github.com/simonw/datasette/blob/50da908213a0fc405ecd7a40090dfea7a2e7395c/datasette/utils/internal_db.py#L8-L62,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1872043170, https://github.com/simonw/datasette/pull/2154#issuecomment-1691798722,https://api.github.com/repos/simonw/datasette/issues/2154,1691798722,IC_kwDOBm6k_c5k1szC,22429695,2023-08-24T14:31:42Z,2023-08-29T16:15:12Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`100.00%`** and project coverage change: **`+0.03%`** :tada: > Comparison is base [(`2e28258`)](https://app.codecov.io/gh/simonw/datasette/commit/2e2825869fc2655b5fcadc743f6f9dec7a49bc65?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.82% compared to head [(`3e49fd3`)](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.85%. > Report is 3 commits behind head on main.
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2154 +/- ## ========================================== + Coverage 92.82% 92.85% +0.03% ========================================== Files 40 40 Lines 5948 6008 +60 ========================================== + Hits 5521 5579 +58 - Misses 427 429 +2 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/views/special.py](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3NwZWNpYWwucHk=) | `94.06% <ø> (-0.85%)` | :arrow_down: | | [datasette/app.py](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.37% <100.00%> (+0.07%)` | :arrow_up: | | [datasette/default\_permissions.py](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2RlZmF1bHRfcGVybWlzc2lvbnMucHk=) | `97.48% <100.00%> (+0.57%)` | :arrow_up: | | [datasette/permissions.py](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3Blcm1pc3Npb25zLnB5) | `100.00% <100.00%> (ø)` | | ... and [3 files with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2154/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2154?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/pull/2154#issuecomment-1697725150,https://api.github.com/repos/simonw/datasette/issues/2154,1697725150,IC_kwDOBm6k_c5lMTre,9599,2023-08-29T15:54:12Z,2023-08-29T15:54:12Z,OWNER,"In that last commit I also upgraded `Permission` from a named tuple to a dataclass, and added a `implies_can_view=True` private (not documented) option to it. Last step is to refactor the code to use that new property.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/pull/2154#issuecomment-1697612168,https://api.github.com/repos/simonw/datasette/issues/2154,1697612168,IC_kwDOBm6k_c5lL4GI,9599,2023-08-29T14:57:54Z,2023-08-29T14:57:54Z,OWNER,"The code to refactor is this: https://github.com/simonw/datasette/blob/d64a9896f743f87b673d58859a0ec16685594e79/datasette/default_permissions.py#L181-L280 I'm going to turn that into a more general `restrictions_allow_action` function.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/pull/2162#issuecomment-1696710911,https://api.github.com/repos/simonw/datasette/issues/2162,1696710911,IC_kwDOBm6k_c5lIcD_,9599,2023-08-29T03:23:44Z,2023-08-29T03:23:44Z,OWNER,I'm going to merge this so we can see how it feels.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1870672704, https://github.com/simonw/datasette/pull/2162#issuecomment-1696709110,https://api.github.com/repos/simonw/datasette/issues/2162,1696709110,IC_kwDOBm6k_c5lIbn2,9599,2023-08-29T03:20:40Z,2023-08-29T03:22:47Z,OWNER,"> However, one important notes about those new `core_` tables: If a `--internal` DB is passed in, that means those `core_` tables will persist across multiple Datasette instances. This wasn't the case before, since `_internal` was always an in-memory database created from scratch. I'm completely happy for the `core_*` tables (or `datasette_*` or some other name) to live in the persisted-to-disk `internal.db` database, even though they're effectively meant to be an in-memory cache. I don't think it causes any harm, and it could even be quite useful to have them visible on disk - other applications could read the `internal.db` database while Datasette itself is running, should they have some weird reason to want to do that! Having those tables stick around in `internal.db` after Datasette shuts down could be useful for other debugging activities as well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1870672704, https://github.com/simonw/datasette/pull/2162#issuecomment-1696594855,https://api.github.com/repos/simonw/datasette/issues/2162,1696594855,IC_kwDOBm6k_c5lH_un,22429695,2023-08-29T00:19:29Z,2023-08-29T03:22:21Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`100.00%`** and project coverage change: **`-0.03%`** :warning: > Comparison is base [(`2e28258`)](https://app.codecov.io/gh/simonw/datasette/commit/2e2825869fc2655b5fcadc743f6f9dec7a49bc65?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.82% compared to head [(`73489ca`)](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.79%. > Report is 1 commits behind head on main.
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2162 +/- ## ========================================== - Coverage 92.82% 92.79% -0.03% ========================================== Files 40 40 Lines 5948 5953 +5 ========================================== + Hits 5521 5524 +3 - Misses 427 429 +2 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/default\_permissions.py](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2RlZmF1bHRfcGVybWlzc2lvbnMucHk=) | `96.87% <ø> (-0.04%)` | :arrow_down: | | [datasette/app.py](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.32% <100.00%> (+0.03%)` | :arrow_up: | | [datasette/cli.py](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `82.53% <100.00%> (-0.06%)` | :arrow_down: | | [datasette/database.py](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2RhdGFiYXNlLnB5) | `94.72% <100.00%> (+0.04%)` | :arrow_up: | | [datasette/utils/internal\_db.py](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL2ludGVybmFsX2RiLnB5) | `100.00% <100.00%> (ø)` | | | [datasette/views/database.py](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL2RhdGFiYXNlLnB5) | `95.45% <100.00%> (ø)` | | | [datasette/views/special.py](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3NwZWNpYWwucHk=) | `94.06% <100.00%> (-0.85%)` | :arrow_down: |
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2162?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1870672704, https://github.com/simonw/datasette/pull/2162#issuecomment-1696707458,https://api.github.com/repos/simonw/datasette/issues/2162,1696707458,IC_kwDOBm6k_c5lIbOC,9599,2023-08-29T03:17:54Z,2023-08-29T03:17:54Z,OWNER,Documentation preview: https://datasette--2162.org.readthedocs.build/en/2162/internals.html#datasette-s-internal-database,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1870672704, https://github.com/simonw/datasette/pull/2154#issuecomment-1696644066,https://api.github.com/repos/simonw/datasette/issues/2154,1696644066,IC_kwDOBm6k_c5lILvi,9599,2023-08-29T01:37:47Z,2023-08-29T03:00:25Z,OWNER,"Code for this might be cleaner with a `Restrictions()` class that takes a `""_r""` dictionary to the constructor and can then answer questions like `.any_resource_has_permission(""view-table"")` - where it can resolve aliases etc as well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/pull/2154#issuecomment-1696642671,https://api.github.com/repos/simonw/datasette/issues/2154,1696642671,IC_kwDOBm6k_c5lILZv,9599,2023-08-29T01:35:37Z,2023-08-29T01:35:37Z,OWNER,"Reminder that I also need to confirm that `insert-row` works if you have it at the instance level, the database level or the resource level in `_r`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/pull/2154#issuecomment-1696618784,https://api.github.com/repos/simonw/datasette/issues/2154,1696618784,IC_kwDOBm6k_c5lIFkg,9599,2023-08-29T00:58:34Z,2023-08-29T00:58:34Z,OWNER,Should this have `implies_can_view=True` too? Probably: https://github.com/simonw/datasette/blob/d64a9896f743f87b673d58859a0ec16685594e79/datasette/default_permissions.py#L20-L22,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/pull/2160#issuecomment-1695754277,https://api.github.com/repos/simonw/datasette/issues/2160,1695754277,IC_kwDOBm6k_c5lEygl,22429695,2023-08-28T14:00:17Z,2023-08-29T00:32:52Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2160?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`2e28258`)](https://app.codecov.io/gh/simonw/datasette/commit/2e2825869fc2655b5fcadc743f6f9dec7a49bc65?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.82% compared to head [(`c81b024`)](https://app.codecov.io/gh/simonw/datasette/pull/2160?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.82%. > :exclamation: Current head c81b024 differs from pull request most recent head 3529c9c. Consider uploading reports for the commit 3529c9c to get more accurate results
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2160 +/- ## ======================================= Coverage 92.82% 92.82% ======================================= Files 40 40 Lines 5948 5948 ======================================= Hits 5521 5521 Misses 427 427 ```
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2160?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1869807874, https://github.com/simonw/datasette/pull/2160#issuecomment-1696595326,https://api.github.com/repos/simonw/datasette/issues/2160,1696595326,IC_kwDOBm6k_c5lH_1-,9599,2023-08-29T00:20:08Z,2023-08-29T00:20:08Z,OWNER,Cog failed!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1869807874, https://github.com/simonw/datasette/pull/2160#issuecomment-1696592763,https://api.github.com/repos/simonw/datasette/issues/2160,1696592763,IC_kwDOBm6k_c5lH_N7,9599,2023-08-29T00:16:38Z,2023-08-29T00:16:38Z,OWNER,"Since this bumps Sphinx I'm manually reviewing the ReadTheDocs preview a bit, looks good to me: https://datasette--2160.org.readthedocs.build/en/2160/","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1869807874, https://github.com/simonw/datasette/pull/2148#issuecomment-1696591957,https://api.github.com/repos/simonw/datasette/issues/2148,1696591957,IC_kwDOBm6k_c5lH_BV,49699333,2023-08-29T00:15:29Z,2023-08-29T00:15:29Z,CONTRIBUTOR,This pull request was built based on a group rule. Closing it will not ignore any of these versions in future pull requests.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334, https://github.com/simonw/datasette/pull/2148#issuecomment-1696591943,https://api.github.com/repos/simonw/datasette/issues/2148,1696591943,IC_kwDOBm6k_c5lH_BH,9599,2023-08-29T00:15:27Z,2023-08-29T00:15:27Z,OWNER,"Now solving this here: - #2160 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334, https://github.com/simonw/datasette/pull/2160#issuecomment-1696586767,https://api.github.com/repos/simonw/datasette/issues/2160,1696586767,IC_kwDOBm6k_c5lH9wP,9599,2023-08-29T00:08:26Z,2023-08-29T00:08:35Z,OWNER,"https://github.com/simonw/datasette/issues/2148#issuecomment-1689177556 > Simplest possible solution is to only run the `pip install .[docs]` bit under Python 3.9+, ditto for the docs tests. I think I'll try that. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1869807874, https://github.com/simonw/datasette/pull/2160#issuecomment-1696586213,https://api.github.com/repos/simonw/datasette/issues/2160,1696586213,IC_kwDOBm6k_c5lH9nl,9599,2023-08-29T00:07:46Z,2023-08-29T00:07:46Z,OWNER,"I figured out why this was failing in: - https://github.com/simonw/datasette/pull/2148 It's because Sphinx dropped support for Python 3.8. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1869807874, https://github.com/simonw/datasette/issues/2102#issuecomment-1696378239,https://api.github.com/repos/simonw/datasette/issues/2102,1696378239,IC_kwDOBm6k_c5lHK1_,9599,2023-08-28T20:38:01Z,2023-08-28T20:38:01Z,OWNER,"I want to test ""for this set of restrictions, does a GET/POST to this path return 200 or 403""?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1696361304,https://api.github.com/repos/simonw/datasette/issues/2102,1696361304,IC_kwDOBm6k_c5lHGtY,9599,2023-08-28T20:23:47Z,2023-08-28T20:24:35Z,OWNER,"Here's an existing relevant test: https://github.com/simonw/datasette/blob/2e2825869fc2655b5fcadc743f6f9dec7a49bc65/tests/test_permissions.py#L616-L666 It's not quite right for this new set of tests though, since they need to be exercising actual endpoints (`/.json` etc) in order to check that this works correctly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2153#issuecomment-1696355634,https://api.github.com/repos/simonw/datasette/issues/2153,1696355634,IC_kwDOBm6k_c5lHFUy,9599,2023-08-28T20:19:15Z,2023-08-28T20:19:15Z,OWNER,"Documentation: - https://docs.datasette.io/en/latest/cli-reference.html#datasette-get ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865232341, https://github.com/simonw/datasette/pull/2161#issuecomment-1696267473,https://api.github.com/repos/simonw/datasette/issues/2161,1696267473,IC_kwDOBm6k_c5lGvzR,22429695,2023-08-28T19:38:28Z,2023-08-28T19:39:33Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2161?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage: **`75.00%`** and project coverage change: **`+0.05%`** :tada: > Comparison is base [(`527cec6`)](https://app.codecov.io/gh/simonw/datasette/commit/527cec66b0403e689c8fb71fc8b381a1d7a46516?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.76% compared to head [(`a5cbf80`)](https://app.codecov.io/gh/simonw/datasette/pull/2161?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.81%.
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2161 +/- ## ========================================== + Coverage 92.76% 92.81% +0.05% ========================================== Files 40 40 Lines 5943 5944 +1 ========================================== + Hits 5513 5517 +4 + Misses 430 427 -3 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/cli.py](https://app.codecov.io/gh/simonw/datasette/pull/2161?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `82.37% <75.00%> (+1.02%)` | :arrow_up: |
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2161?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1870345352, https://github.com/simonw/datasette/issues/2156#issuecomment-1696260446,https://api.github.com/repos/simonw/datasette/issues/2156,1696260446,IC_kwDOBm6k_c5lGuFe,9599,2023-08-28T19:33:05Z,2023-08-28T19:33:05Z,OWNER,"Now a PR: - #2161 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865649347, https://github.com/simonw/datasette/pull/2152#issuecomment-1695736691,https://api.github.com/repos/simonw/datasette/issues/2152,1695736691,IC_kwDOBm6k_c5lEuNz,49699333,2023-08-28T13:49:35Z,2023-08-28T13:49:35Z,CONTRIBUTOR,Superseded by #2160.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865174661, https://github.com/simonw/sqlite-utils/issues/588#issuecomment-1694823972,https://api.github.com/repos/simonw/sqlite-utils/issues/588,1694823972,IC_kwDOCGYnMM5lBPYk,9599,2023-08-28T00:41:54Z,2023-08-28T00:41:54Z,OWNER,Tips on typing `**kwargs`: https://adamj.eu/tech/2021/05/11/python-type-hints-args-and-kwargs/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1868713944, https://github.com/simonw/datasette/pull/2152#issuecomment-1691710474,https://api.github.com/repos/simonw/datasette/issues/2152,1691710474,IC_kwDOBm6k_c5k1XQK,22429695,2023-08-24T13:45:21Z,2023-08-25T13:18:34Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2152?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`527cec6`)](https://app.codecov.io/gh/simonw/datasette/commit/527cec66b0403e689c8fb71fc8b381a1d7a46516?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.76% compared to head [(`2e45686`)](https://app.codecov.io/gh/simonw/datasette/pull/2152?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.76%. > :exclamation: Current head 2e45686 differs from pull request most recent head 5dfa305. Consider uploading reports for the commit 5dfa305 to get more accurate results
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2152 +/- ## ======================================= Coverage 92.76% 92.76% ======================================= Files 40 40 Lines 5943 5919 -24 ======================================= - Hits 5513 5491 -22 + Misses 430 428 -2 ``` [see 3 files with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2152/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2152?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865174661, https://github.com/simonw/datasette/issues/950#issuecomment-1692494455,https://api.github.com/repos/simonw/datasette/issues/950,1692494455,IC_kwDOBm6k_c5k4Wp3,9599,2023-08-24T22:26:08Z,2023-08-24T22:26:08Z,OWNER,"Closing this issue in favour of this one: - #2157 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",685806511, https://github.com/simonw/datasette/issues/2157#issuecomment-1692465763,https://api.github.com/repos/simonw/datasette/issues/2157,1692465763,IC_kwDOBm6k_c5k4Ppj,9599,2023-08-24T21:54:29Z,2023-08-24T21:54:29Z,OWNER,"But yes, I'm a big +1 on this whole plan.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865869205, https://github.com/simonw/datasette/issues/2157#issuecomment-1692465334,https://api.github.com/repos/simonw/datasette/issues/2157,1692465334,IC_kwDOBm6k_c5k4Pi2,9599,2023-08-24T21:54:09Z,2023-08-24T21:54:09Z,OWNER,"We discussed this in-person this morning and these notes reflect what we talked about perfectly. I've had so many bugs with plugins that I've written myself that have forgotten to special-case the `_internal` database when looping through `datasette.databases.keys()` - removing it from there entirely would help a lot. Just one tiny disagreement: for `datasette-comments` I think having it store things in `_internal` could be an option, but in most cases I expect users to chose NOT to do that - because being able to join against those tables for more advanced queries is going to be super useful. Show me all rows in `foia_requests` with at least one associated comment in `datasette_comments.comments` kind of tihng.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865869205, https://github.com/simonw/datasette/issues/1241#issuecomment-1692322342,https://api.github.com/repos/simonw/datasette/issues/1241,1692322342,IC_kwDOBm6k_c5k3som,52261150,2023-08-24T19:56:15Z,2023-08-24T20:09:52Z,NONE,"Something to think about, but I hate how long the url is when sharing a custom SQL query. Would it be possible to hash the query and state of a page instead so the url is more manageable? The mapping from hash to query would have to be stored in order to recover/lookup the page after sharing. It's not uncommon to have things like this currently: ```https://global-power-plants.datasettes.com/global-power-plants?sql=select+rowid%2C+country%2C+country_long%2C+name%2C+gppd_idnr%2C+capacity_mw%2C+latitude%2C+longitude%2C+primary_fuel%2C+other_fuel1%2C+other_fuel2%2C+other_fuel3%2C+commissioning_year%2C+owner%2C+source%2C+url%2C+geolocation_source%2C+wepp_id%2C+year_of_capacity_data%2C+generation_gwh_2013%2C+generation_gwh_2014%2C+generation_gwh_2015%2C+generation_gwh_2016%2C+generation_gwh_2017%2C+generation_gwh_2018%2C+generation_gwh_2019%2C+generation_data_source%2C+estimated_generation_gwh_2013%2C+estimated_generation_gwh_2014%2C+estimated_generation_gwh_2015%2C+estimated_generation_gwh_2016%2C+estimated_generation_gwh_2017%2C+estimated_generation_note_2013%2C+estimated_generation_note_2014%2C+estimated_generation_note_2015%2C+estimated_generation_note_2016%2C+estimated_generation_note_2017+from+%5Bglobal-power-plants%5D+order+by+rowid+limit+101``` I'm thinking a plugin like [https://datasette.io/plugins/datasette-query-files](https://datasette.io/plugins/datasette-query-files), but could be created and managed from the UI (with the right permissions).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",814595021, https://github.com/simonw/datasette/issues/2143#issuecomment-1692210044,https://api.github.com/repos/simonw/datasette/issues/2143,1692210044,IC_kwDOBm6k_c5k3RN8,9599,2023-08-24T18:28:27Z,2023-08-24T18:28:27Z,OWNER,"Just spotted this: https://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/datasette/app.py#L328-L332 https://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/datasette/app.py#L359-L360 Looks to me like that second bit of code doesn't yet handle `datasette.yml` This code does though: https://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/datasette/app.py#L333-L335 `parse_metadata()` is clearly a bad name for this function: https://github.com/simonw/datasette/blob/d97e82df3c8a3f2e97038d7080167be9bb74a68d/datasette/utils/__init__.py#L980-L990 That ` @documented` decorator indicates that it's part of the documented API used by plugin authors: https://docs.datasette.io/en/1.0a4/internals.html#parse-metadata-content So we should rename it to something better like `parse_json_or_yaml()` but keep `parse_metadata` as an undocumented alias for that to avoid any unnecessary plugin breaks.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/issues/2156#issuecomment-1692206200,https://api.github.com/repos/simonw/datasette/issues/2156,1692206200,IC_kwDOBm6k_c5k3QR4,9599,2023-08-24T18:25:23Z,2023-08-24T18:25:23Z,OWNER,"Ran out of time for this, I'll look at the next step next week.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865649347, https://github.com/simonw/datasette/issues/2156#issuecomment-1692201647,https://api.github.com/repos/simonw/datasette/issues/2156,1692201647,IC_kwDOBm6k_c5k3PKv,9599,2023-08-24T18:21:53Z,2023-08-24T18:21:53Z,OWNER,"Oops, that was meant to be a PR. It's just a utility function though so it's safe to land already. I'll do a PR for the actual integration of it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865649347, https://github.com/simonw/datasette/issues/2156#issuecomment-1692186522,https://api.github.com/repos/simonw/datasette/issues/2156,1692186522,IC_kwDOBm6k_c5k3Lea,9599,2023-08-24T18:10:04Z,2023-08-24T18:10:04Z,OWNER,I have an implementation in https://github.com/simonw/datasette/issues/2143#issuecomment-1690792514 too - I'm going to land that as a PR.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865649347, https://github.com/simonw/datasette/issues/2143#issuecomment-1692182910,https://api.github.com/repos/simonw/datasette/issues/2143,1692182910,IC_kwDOBm6k_c5k3Kl-,9599,2023-08-24T18:06:57Z,2023-08-24T18:08:17Z,OWNER,"The other thing that could work is something like this: ```bash export AUTH_TOKENS_DB=""tokens"" datasette \ -s settings.sql_time_limit_ms 1000 \ -s plugins.datasette-auth-tokens.manage_tokens true \ -e plugins.datasette-auth-tokens.manage_tokens_database AUTH_TOKENS_DB ``` So `-e` is an alternative version of `-s` which reads from the named environment variable instead of having the value provided directly as the second value in the pair. I quite like this, because it could replace the really ugly `$ENV` pattern we have in plugin configuration at the moment: https://docs.datasette.io/en/1.0a4/plugins.html#secret-configuration-values ```yaml plugins: datasette-auth-github: client_secret: $env: GITHUB_CLIENT_SECRET ```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/issues/2143#issuecomment-1692180683,https://api.github.com/repos/simonw/datasette/issues/2143,1692180683,IC_kwDOBm6k_c5k3KDL,9599,2023-08-24T18:05:17Z,2023-08-24T18:05:17Z,OWNER,"That's a really good call, thanks @rclement - environment variable configuration totally makes sense here. Need to figure out the right syntax for that. Something like this perhaps: ```bash DATASETTE_CONFIG_PLUGINS='{""datasette-ripgrep"": ...}' ``` Hard to know how to make this nestable though. I considered this: ```bash DATASETTE_CONFIG_PLUGINS_DATASETTE_RIPGREP_PATH='/path/to/code/' ``` But that doesn't work, because how does the processing code know that it should split on `_` for most of the tokens but NOT split `DATASETTE_RIPGREP`, instead treating that as `datasette-ripgrep`? I checked and `-` is not a valid character in an environment variable, at least in zsh on macOS: ``` % export FOO_BAR-BAZ=1 export: not valid in this context: FOO_BAR-BAZ ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/pull/2154#issuecomment-1691845306,https://api.github.com/repos/simonw/datasette/issues/2154,1691845306,IC_kwDOBm6k_c5k14K6,9599,2023-08-24T14:57:39Z,2023-08-24T14:57:39Z,OWNER,"Notes on manual testing so far - it looks like this might be working! - https://github.com/simonw/datasette/issues/2102#issuecomment-1691824713","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/issues/2102#issuecomment-1691842259,https://api.github.com/repos/simonw/datasette/issues/2102,1691842259,IC_kwDOBm6k_c5k13bT,9599,2023-08-24T14:55:54Z,2023-08-24T14:55:54Z,OWNER,"So what's needed to finish this is: - Tests that demonstrate that nothing is revealed that shouldn't be by tokens restricted in this way - Similar tests for other permissions like `create-table` that check that they work (and don't also need `view-instance` etc). - Documentation","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1691824713,https://api.github.com/repos/simonw/datasette/issues/2102,1691824713,IC_kwDOBm6k_c5k1zJJ,9599,2023-08-24T14:45:49Z,2023-08-24T14:45:49Z,OWNER,"I tested this out against a Datasette Cloud instance. I created a restricted token and tested it like this: ```bash curl -H ""Authorization: Bearer $TOKEN"" \ 'https://$INSTANCE/-/actor.json' | jq ``` ```json { ""actor"": { ""id"": ""245"", ""token"": ""dsatok"", ""token_id"": 2, ""_r"": { ""r"": { ""data"": { ""all_stocks"": [ ""vt"" ] } } } } } ``` It can access the `all_stocks` demo table: ```bash curl -H ""Authorization: Bearer $TOKEN"" \ 'https://$INSTANCE/data/all_stocks.json?_size=1' | jq ``` ```json { ""ok"": true, ""next"": ""1"", ""rows"": [ { ""rowid"": 1, ""Date"": ""2013-01-02"", ""Open"": 79.12, ""High"": 79.29, ""Low"": 77.38, ""Close"": 78.43, ""Volume"": 140124866, ""Name"": ""AAPL"" } ], ""truncated"": false } ``` Accessing the database returns just information about that table, even though other tables exist: ```bash curl -H ""Authorization: Bearer $TOKEN"" \ 'https://$INSTANCE/data.json?_size=1' ``` ```json { ""database"": ""data"", ""private"": true, ""path"": ""/data"", ""size"": 3796992, ""tables"": [ { ""name"": ""all_stocks"", ""columns"": [ ""Date"", ""Open"", ""High"", ""Low"", ""Close"", ""Volume"", ""Name"" ], ""primary_keys"": [], ""count"": 8813, ""hidden"": false, ""fts_table"": null, ""foreign_keys"": { ""incoming"": [], ""outgoing"": [] }, ""private"": true } ], ""hidden_count"": 0, ""views"": [], ""queries"": [], ""allow_execute_sql"": false, ""table_columns"": {} } ``` And hitting the top-level `/.json` thing does the same - it reveals that table but not any of the other tables or databases: ```bash curl -H ""Authorization: Bearer $TOKEN"" \ 'https://$INSTANCE/.json?_size=1' ``` ```json { ""data"": { ""name"": ""data"", ""hash"": null, ""color"": ""8d777f"", ""path"": ""/data"", ""tables_and_views_truncated"": [ { ""name"": ""all_stocks"", ""columns"": [ ""Date"", ""Open"", ""High"", ""Low"", ""Close"", ""Volume"", ""Name"" ], ""primary_keys"": [], ""count"": null, ""hidden"": false, ""fts_table"": null, ""num_relationships_for_sorting"": 0, ""private"": false } ], ""tables_and_views_more"": false, ""tables_count"": 1, ""table_rows_sum"": 0, ""show_table_row_counts"": false, ""hidden_table_rows_sum"": 0, ""hidden_tables_count"": 0, ""views_count"": 0, ""private"": false } } ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/pull/2154#issuecomment-1691788400,https://api.github.com/repos/simonw/datasette/issues/2154,1691788400,IC_kwDOBm6k_c5k1qRw,9599,2023-08-24T14:25:56Z,2023-08-24T14:25:56Z,OWNER,"Can be tested with: ```bash pip install https://github.com/simonw/datasette/archive/6d57a8c23043e99b27f7a2afbe58f4d58815fd51.zip ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865281760, https://github.com/simonw/datasette/issues/2153#issuecomment-1691779180,https://api.github.com/repos/simonw/datasette/issues/2153,1691779180,IC_kwDOBm6k_c5k1oBs,9599,2023-08-24T14:21:03Z,2023-08-24T14:21:03Z,OWNER,"`datasette serve` currently only has a `--get` - for this to be really useful it needs to grow `--post` and maybe other verbs too. Which is a good argument for moving this functionality to `datasette client get ...` instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865232341, https://github.com/simonw/datasette/pull/2152#issuecomment-1691767797,https://api.github.com/repos/simonw/datasette/issues/2152,1691767797,IC_kwDOBm6k_c5k1lP1,9599,2023-08-24T14:15:10Z,2023-08-24T14:15:10Z,OWNER,This is broken because Sphinx no longer supports Python 3.8.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865174661, https://github.com/simonw/datasette/issues/2153#issuecomment-1691763427,https://api.github.com/repos/simonw/datasette/issues/2153,1691763427,IC_kwDOBm6k_c5k1kLj,9599,2023-08-24T14:12:43Z,2023-08-24T14:12:43Z,OWNER,Annoying that `datasette client ...` makes a great name both for a plugin that executes simulated queries against a local database (thanks to its similarity to the existing `datasette.client` Python API) but is also the ideal name for a command for running commands as a client of an external Datasette instance!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865232341, https://github.com/simonw/datasette/issues/2153#issuecomment-1691761685,https://api.github.com/repos/simonw/datasette/issues/2153,1691761685,IC_kwDOBm6k_c5k1jwV,9599,2023-08-24T14:11:41Z,2023-08-24T14:11:41Z,OWNER,"Another option: implement this as a plugin, providing a new command like `datasette get ...` Or implement `datasette client get ...` as core commands or a plugin - except that clashes with the `datasette client` command that https://github.com/simonw/dclient adds (which is a tool for hitting remote Datasette instances, not running simulated queries through a local one).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865232341, https://github.com/simonw/datasette/issues/2102#issuecomment-1691758168,https://api.github.com/repos/simonw/datasette/issues/2102,1691758168,IC_kwDOBm6k_c5k1i5Y,9599,2023-08-24T14:09:45Z,2023-08-24T14:09:45Z,OWNER,I'm going to implement this in a branch to make it easier to test out.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2153#issuecomment-1691753489,https://api.github.com/repos/simonw/datasette/issues/2153,1691753489,IC_kwDOBm6k_c5k1hwR,9599,2023-08-24T14:07:25Z,2023-08-24T14:09:16Z,OWNER,"Building that `""_r""` array is the main reason this would be useful, but it's also fiddly to get right. `datasette create-token` has a design for that already: https://docs.datasette.io/en/1.0a4/authentication.html#datasette-create-token ``` datasette create-token root \ --secret mysecret \ --all view-instance \ --all view-table \ --database docs view-query \ --resource docs documents insert-row \ --resource docs documents update-row ``` Adding imitations of those options (excluding `--secret`, not needed here) to `datasette serve` would add a LOT of extra options, but it would also make it really convenient to attempt a request with a specific set of restrictions. Not sure if that would be worth the extra `--help` output or not. I feel like the names would have to have a common prefix though. Maybe something like this: ```bash datasette serve data.db --get `/data/mytable.json' \ --actor-id root \ --r-all view-instance \ --r-database data view-query \ --r-resource data documents update-row ``` Other options could be the longer `--restrict-all/--restrict-database/--restrict-resource`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1865232341, https://github.com/simonw/datasette/issues/2143#issuecomment-1691094870,https://api.github.com/repos/simonw/datasette/issues/2143,1691094870,IC_kwDOBm6k_c5kzA9W,1238873,2023-08-24T06:43:40Z,2023-08-24T06:43:40Z,NONE,"If I may, the ""path-like"" configuration is great but one thing that would be even greater: allowing the same configuration to be provided using environment variables. For instance: ``` datasette -s plugins.datasette-complex-plugin.configs '{""foo"": [1,2,3], ""bar"": ""baz""}' ``` could also be provided using: ``` export DS_PLUGINS_DATASETTE-COMPLEX-PLUGIN_CONFIGS='{""foo"": [1,2,3], ""bar"": ""baz""}' datasette ``` (I do not like mixing `-` and `_` in env vars but I do not have a best easily reversible example at the moment) FYI, you could take some inspiration from another great open source data project, Metabase: https://www.metabase.com/docs/latest/configuring-metabase/config-file https://www.metabase.com/docs/latest/configuring-metabase/environment-variables","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/issues/2102#issuecomment-1691045051,https://api.github.com/repos/simonw/datasette/issues/2102,1691045051,IC_kwDOBm6k_c5ky0y7,9599,2023-08-24T05:51:59Z,2023-08-24T05:51:59Z,OWNER,"With that fix in place, this works: ```bash datasette fixtures.db --get '/fixtures/facetable.json' --actor '{ ""_r"": { ""r"": { ""fixtures"": { ""facetable"": [ ""vt"" ] } } }, ""a"": ""user"" }' ``` But this fails, because it's for a table not explicitly listed: ```bash datasette fixtures.db --get '/fixtures/searchable.json' --actor '{ ""_r"": { ""r"": { ""fixtures"": { ""facetable"": [ ""vt"" ] } } }, ""a"": ""user"" }' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1691044283,https://api.github.com/repos/simonw/datasette/issues/2102,1691044283,IC_kwDOBm6k_c5ky0m7,9599,2023-08-24T05:51:02Z,2023-08-24T05:51:02Z,OWNER,"Also need to confirm that permissions like `insert-row`, `delete-row`, `create-table` etc don't also need special cases to ensure they get through the `view-instance` etc checks, if those exist for those actions.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1691043475,https://api.github.com/repos/simonw/datasette/issues/2102,1691043475,IC_kwDOBm6k_c5ky0aT,9599,2023-08-24T05:50:04Z,2023-08-24T05:50:04Z,OWNER,"On first test this seems to work! ```diff diff --git a/datasette/default_permissions.py b/datasette/default_permissions.py index 63a66c3c..9303dac8 100644 --- a/datasette/default_permissions.py +++ b/datasette/default_permissions.py @@ -187,6 +187,30 @@ def permission_allowed_actor_restrictions(datasette, actor, action, resource): return None _r = actor.get(""_r"") + # Special case for view-instance: it's allowed if there are any view-database + # or view-table permissions defined + if action == ""view-instance"": + database_rules = _r.get(""d"") or {} + for rules in database_rules.values(): + if ""vd"" in rules or ""view-database"" in rules: + return None + # Now check resources + resource_rules = _r.get(""r"") or {} + for _database, resources in resource_rules.items(): + for rules in resources.values(): + if ""vt"" in rules or ""view-table"" in rules: + return None + + # Special case for view-database: it's allowed if there are any view-table permissions + # defined within that database + if action == ""view-database"": + database_name = resource + resource_rules = _r.get(""r"") or {} + resources_in_database = resource_rules.get(database_name) or {} + for rules in resources_in_database.values(): + if ""vt"" in rules or ""view-table"" in rules: + return None + # Does this action have an abbreviation? to_check = {action} permission = datasette.permissions.get(action) ``` Needs a LOT of testing to make sure what it's doing is sensible though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1691037971,https://api.github.com/repos/simonw/datasette/issues/2102,1691037971,IC_kwDOBm6k_c5kyzET,9599,2023-08-24T05:42:47Z,2023-08-24T05:42:47Z,OWNER,"I applied a fun trick to help test this out: ```diff diff --git a/datasette/cli.py b/datasette/cli.py index 58f89c1c..830f47ef 100644 --- a/datasette/cli.py +++ b/datasette/cli.py @@ -445,6 +445,10 @@ def uninstall(packages, yes): ""--token"", help=""API token to send with --get requests"", ) +@click.option( + ""--actor"", + help=""Actor to use for --get requests"", +) @click.option(""--version-note"", help=""Additional note to show on /-/versions"") @click.option(""--help-settings"", is_flag=True, help=""Show available settings"") @click.option(""--pdb"", is_flag=True, help=""Launch debugger on any errors"") @@ -499,6 +503,7 @@ def serve( root, get, token, + actor, version_note, help_settings, pdb, @@ -611,7 +616,10 @@ def serve( headers = {} if token: headers[""Authorization""] = ""Bearer {}"".format(token) - response = client.get(get, headers=headers) + cookies = {} + if actor: + cookies[""ds_actor""] = client.actor_cookie(json.loads(actor)) + response = client.get(get, headers=headers, cookies=cookies) click.echo(response.text) exit_code = 0 if response.status == 200 else 1 sys.exit(exit_code) ``` This adds a `--actor` option to `datasette ... --get /path` which makes it easy to test an API endpoint using a fake actor with a set of `_r` restrictions. With that in place I can try this, with a token that has view-instance and view-database and view-table: ```bash datasette fixtures.db --get '/fixtures/facetable.json' --actor '{ ""_r"": { ""a"": [ ""vi"" ], ""d"": { ""fixtures"": [ ""vd"" ] }, ""r"": { ""fixtures"": { ""facetable"": [ ""vt"" ] } } }, ""a"": ""user"" }' ``` Or this, with a token that just has view-table but is missing the view-database and view-instance: ```bash datasette fixtures.db --get '/fixtures/facetable.json' --actor '{ ""_r"": { ""r"": { ""fixtures"": { ""facetable"": [ ""vt"" ] } } }, ""a"": ""user"" }' ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1691036559,https://api.github.com/repos/simonw/datasette/issues/2102,1691036559,IC_kwDOBm6k_c5kyyuP,9599,2023-08-24T05:40:53Z,2023-08-24T05:40:53Z,OWNER,"There might be an easier way to solve this. Here's some permission checks that run when hitting `/fixtures/facetable.json`: ``` permission_allowed: action=view-table, resource=('fixtures', 'facetable'), actor={'_r': {'a': ['vi'], 'd': {'fixtures': ['vd']}, 'r': {'fixtures': {'facetable': ['vt']}}}, 'a': 'user'} File ""/datasette/views/table.py"", line 727, in table_view_traced view_data = await table_view_data( File ""/datasette/views/table.py"", line 875, in table_view_data visible, private = await datasette.check_visibility( File ""/datasette/app.py"", line 890, in check_visibility await self.ensure_permissions(actor, permissions) permission_allowed: action=view-database, resource=fixtures, actor={'_r': {'a': ['vi'], 'd': {'fixtures': ['vd']}, 'r': {'fixtures': {'facetable': ['vt']}}}, 'a': 'user'} File ""/datasette/views/table.py"", line 727, in table_view_traced view_data = await table_view_data( File ""/datasette/views/table.py"", line 875, in table_view_data visible, private = await datasette.check_visibility( File ""/datasette/app.py"", line 890, in check_visibility await self.ensure_permissions(actor, permissions) permission_allowed: action=view-instance, resource=, actor={'_r': {'a': ['vi'], 'd': {'fixtures': ['vd']}, 'r': {'fixtures': {'facetable': ['vt']}}}, 'a': 'user'} File ""/datasette/views/table.py"", line 727, in table_view_traced view_data = await table_view_data( File ""/datasette/views/table.py"", line 875, in table_view_data visible, private = await datasette.check_visibility( File ""/datasette/app.py"", line 890, in check_visibility await self.ensure_permissions(actor, permissions) ``` That's with a token that has the view instance, view database and view table permissions required. But... what if the restrictions logic said that if you have view-table you automatically also get view-database and view-instance? Would that actually let people do anything they shouldn't be able to do? I don't think it would even let them see a list of tables that they weren't allowed to visit, so it might be OK. I'll try that and see how it works.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2147#issuecomment-1690955706,https://api.github.com/repos/simonw/datasette/issues/2147,1690955706,IC_kwDOBm6k_c5kye-6,18899,2023-08-24T03:54:35Z,2023-08-24T03:54:35Z,NONE,"That's fair. The best idea I can think of is that if a plugin wanted to limit intensive queries, it could add LIMITs or something. A hook that gives you visibility of queries and maybe the option to reject felt a little more limited than the existing plugin hooks, so I was trying to think of what else one might want to do while looking at to-be-run queries. But without a real motivating example, I see why you don't want to add that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1858228057, https://github.com/simonw/datasette/issues/2143#issuecomment-1690800119,https://api.github.com/repos/simonw/datasette/issues/2143,1690800119,IC_kwDOBm6k_c5kx4_3,9599,2023-08-24T00:10:32Z,2023-08-24T00:39:00Z,OWNER,"Something notable about this design is that, because the values in the key-value pairs are treated as JSON first and then strings only if they don't parse cleanly as JSON, it's possible to represent any structure (including nesting structures) using this syntax. You can do things like this if you need to (settings for an imaginary plugin): ```bash datasette data.db \ -s plugins.datasette-complex-plugin.configs '{""foo"": [1,2,3], ""bar"": ""baz""}' ``` Which would be equivalent to: ```yaml plugins: datasette-complex-plugin: configs: foo: - 1 - 2 - 3 bar: baz ``` This is a bit different from a previous attempt I made at the same problem: https://github.com/simonw/json-flatten - that used syntax like `foo.bar.[0]$int = 1` to specify an integer as the first item of an array, which is much more complex. That previous design was meant to support round-trips, so you could take any nested JSON object and turn it into an HTMl form or query string where every value can have its own form field, then turn the result back again. For the `datasette -s key value` feature we don't need round-tripping with individual values each editable on their own, so we can go with something much simpler.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/issues/2143#issuecomment-1690800641,https://api.github.com/repos/simonw/datasette/issues/2143,1690800641,IC_kwDOBm6k_c5kx5IB,9599,2023-08-24T00:11:16Z,2023-08-24T00:11:16Z,OWNER,"> @simonw, FWIW, I do exactly the same thing for one of my projects (both to allow multiple configuration files to be passed on the command line and setting individual values) and it works quite well for me and my users. I even use the same parameter name for both (https://studio.zerobrane.com/doc-configuration#configuration-via-command-line), but I understand why you may want to use different ones for files and individual values. There is one small difference that I accept code snippets, but I don't think it matters much in this case. That's a neat example thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/issues/2143#issuecomment-1690799608,https://api.github.com/repos/simonw/datasette/issues/2143,1690799608,IC_kwDOBm6k_c5kx434,77071,2023-08-24T00:09:47Z,2023-08-24T00:10:41Z,NONE,"@simonw, FWIW, I do exactly the same thing for one of my projects (both to allow multiple configuration files to be passed on the command line and setting individual values) and it works quite well for me and my users. I even use the same parameter name for both (https://studio.zerobrane.com/doc-configuration#configuration-via-command-line), but I understand why you may want to use different ones for files and individual values. There is one small difference that I accept code snippets, but I don't think it matters much in this case.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/issues/2143#issuecomment-1690792514,https://api.github.com/repos/simonw/datasette/issues/2143,1690792514,IC_kwDOBm6k_c5kx3JC,9599,2023-08-24T00:00:16Z,2023-08-24T00:02:55Z,OWNER,"I've been thinking about what it might look like to allow command-line arguments to be used to define _any_ of the configuration options in `datasette.yml`, as alternative and more convenient syntax. Here's what I've come up with: ``` datasette \ -s settings.sql_time_limit_ms 1000 \ -s plugins.datasette-auth-tokens.manage_tokens true \ -s plugins.datasette-auth-tokens.manage_tokens_database tokens \ -s plugins.datasette-ripgrep.path ""/home/simon/code-to-search"" \ -s databases.mydatabase.tables.example_table.sort created \ mydatabase.db tokens.db ``` Which would be equivalent to `datasette.yml` containing this: ```yaml plugins: datasette-auth-tokens: manage_tokens: true manage_tokens_database: tokens datasette-ripgrep: path: /home/simon/code-to-search databases: mydatabase: tables: example_table: sort: created settings: sql_time_limit_ms: 1000 ``` Here's a prototype implementation of this: ```python import json from typing import Any, List, Tuple def _handle_pair(key: str, value: str) -> dict: """""" Turn a key-value pair into a nested dictionary. foo, bar => {'foo': 'bar'} foo.bar, baz => {'foo': {'bar': 'baz'}} foo.bar, [1, 2, 3] => {'foo': {'bar': [1, 2, 3]}} foo.bar, ""baz"" => {'foo': {'bar': 'baz'}} foo.bar, '{""baz"": ""qux""}' => {'foo': {'bar': ""{'baz': 'qux'}""}} """""" try: value = json.loads(value) except json.JSONDecodeError: # If it doesn't parse as JSON, treat it as a string pass keys = key.split('.') result = current_dict = {} for k in keys[:-1]: current_dict[k] = {} current_dict = current_dict[k] current_dict[keys[-1]] = value return result def _combine(base: dict, update: dict) -> dict: """""" Recursively merge two dictionaries. """""" for key, value in update.items(): if isinstance(value, dict) and key in base and isinstance(base[key], dict): base[key] = _combine(base[key], value) else: base[key] = value return base def handle_pairs(pairs: List[Tuple[str, Any]]) -> dict: """""" Parse a list of key-value pairs into a nested dictionary. """""" result = {} for key, value in pairs: parsed_pair = _handle_pair(key, value) result = _combine(result, parsed_pair) return result ``` Exercised like this: ```python print(json.dumps(handle_pairs([ (""settings.sql_time_limit_ms"", ""1000""), (""plugins.datasette-auth-tokens.manage_tokens"", ""true""), (""plugins.datasette-auth-tokens.manage_tokens_database"", ""tokens""), (""plugins.datasette-ripgrep.path"", ""/home/simon/code-to-search""), (""databases.mydatabase.tables.example_table.sort"", ""created""), ]), indent=4)) ``` Output: ```json { ""settings"": { ""sql_time_limit_ms"": 1000 }, ""plugins"": { ""datasette-auth-tokens"": { ""manage_tokens"": true, ""manage_tokens_database"": ""tokens"" }, ""datasette-ripgrep"": { ""path"": ""/home/simon/code-to-search"" } }, ""databases"": { ""mydatabase"": { ""tables"": { ""example_table"": { ""sort"": ""created"" } } } } } ``` Note that `-s` isn't currently an option for `datasette serve`. `--setting key value` IS an existing option, but it isn't completely compatible with this because it maps directly just to settings. Although... we could keep compatibility by saying that if you call `--setting known_setting value` and that `known_setting` is in this list then we treat it as if you said `-s settings.known_setting value` instead: https://github.com/simonw/datasette/blob/bdf59eb7db42559e538a637bacfe86d39e5d17ca/datasette/app.py#L114-L204","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/issues/2143#issuecomment-1690787394,https://api.github.com/repos/simonw/datasette/issues/2143,1690787394,IC_kwDOBm6k_c5kx15C,9599,2023-08-23T23:52:02Z,2023-08-23T23:52:02Z,OWNER,"> This also makes it simple to separate out secrets. > > `datasette --config settings.yaml --config secrets.yaml --config db-docs.yaml --config db-fixtures.yaml` Having multiple configs that combine in that way is a really interesting direction. > To chime in from a poweruser perspective: I'm worried that this is an overengineering trap. Yes, the current solution is somewhat messy. But there are datasette-wide settings, there are database-scope settings, there are table-scope settings etc, but then there are database-scope metadata and table-scope metadata. Trying to cleanly separate ""settings"" from ""configuration"" is, I believe, an uphill fight. I'm very keen on separating out the ""metadata"" - where metadata is the slimmest possible set of things, effectively the data license and the source and the column and table descriptions - from everything else, mainly because I want metadata to be able to travel with the data. One idea that's been discussed before is having an optional mechanism for storing metadata in the SQLite database file itself - potentially in a `_datasette_metadata` table. That way you could distribute a DB file and anyone who opened it in Datasette would also see the correct metadata about it. That's why I'm so keen on splitting out metadata from all of the other stuff - settings and plugin configuration and authentication rules. So really it becomes ""true metadata"" v.s. ""all of the other junk that's accumulated in metadata and `settings.json`"".","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1855885427, https://github.com/simonw/datasette/pull/2151#issuecomment-1690746371,https://api.github.com/repos/simonw/datasette/issues/2151,1690746371,IC_kwDOBm6k_c5kxr4D,22429695,2023-08-23T22:51:57Z,2023-08-23T22:58:13Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2151?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch and project coverage have no change. > Comparison is base [(`bdf59eb`)](https://app.codecov.io/gh/simonw/datasette/commit/bdf59eb7db42559e538a637bacfe86d39e5d17ca?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.73% compared to head [(`b895cd2`)](https://app.codecov.io/gh/simonw/datasette/pull/2151?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.73%.
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2151 +/- ## ======================================= Coverage 92.73% 92.73% ======================================= Files 40 40 Lines 5919 5919 ======================================= Hits 5489 5489 Misses 430 430 ```
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2151?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1864112887, https://github.com/simonw/datasette/issues/2102#issuecomment-1690705243,https://api.github.com/repos/simonw/datasette/issues/2102,1690705243,IC_kwDOBm6k_c5kxh1b,9599,2023-08-23T22:03:54Z,2023-08-23T22:03:54Z,OWNER,Idea: `datasette-permissions-debug` plugin which simply prints out a stacktrace for every permission check so you can see where in the code they are.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1690703764,https://api.github.com/repos/simonw/datasette/issues/2102,1690703764,IC_kwDOBm6k_c5kxheU,9599,2023-08-23T22:02:14Z,2023-08-23T22:02:14Z,OWNER,"Built this new test: ```python @pytest.mark.asyncio async def test_view_table_token_can_access_table(perms_ds): actor = { ""id"": ""restricted-token"", ""token"": ""dstok"", # Restricted to just view-table on perms_ds_two/t1 ""_r"": {""r"": {""perms_ds_two"": {""t1"": [""vt""]}}}, } cookies = {""ds_actor"": perms_ds.client.actor_cookie(actor)} response = await perms_ds.client.get(""/perms_ds_two/t1.json"", cookies=cookies) assert response.status_code == 200 ``` The test fails. Running it with `pytest --pdb` let me do this: ``` (Pdb) from pprint import pprint (Pdb) pprint(perms_ds._permission_checks) deque([{'action': 'view-table', 'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}}, 'id': 'restricted-token', 'token': 'dstok'}, 'resource': ('perms_ds_two', 't1'), 'result': None, 'used_default': True, 'when': '2023-08-23T21:59:45.117155'}, {'action': 'view-database', 'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}}, 'id': 'restricted-token', 'token': 'dstok'}, 'resource': 'perms_ds_two', 'result': False, 'used_default': False, 'when': '2023-08-23T21:59:45.117189'}, {'action': 'view-instance', 'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}}, 'id': 'restricted-token', 'token': 'dstok'}, 'resource': None, 'result': False, 'used_default': False, 'when': '2023-08-23T21:59:45.126751'}, {'action': 'debug-menu', 'actor': {'_r': {'r': {'perms_ds_two': {'t1': ['vt']}}}, 'id': 'restricted-token', 'token': 'dstok'}, 'resource': None, 'result': False, 'used_default': False, 'when': '2023-08-23T21:59:45.126777'}], maxlen=200) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/2102#issuecomment-1690693830,https://api.github.com/repos/simonw/datasette/issues/2102,1690693830,IC_kwDOBm6k_c5kxfDG,9599,2023-08-23T21:51:52Z,2023-08-23T21:52:58Z,OWNER,"This is the hook in question: https://github.com/simonw/datasette/blob/bdf59eb7db42559e538a637bacfe86d39e5d17ca/datasette/hookspecs.py#L108-L110 - `True` means they are allowed to access it. You only need a single`True` from a plugin to allow it. - `False` means they are not, and just one `False` from a plugin will deny it (even if another one returned `True` I think) - `None` means that the plugin has no opinion on this question.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1805076818, https://github.com/simonw/datasette/issues/950#issuecomment-680262437,https://api.github.com/repos/simonw/datasette/issues/950,680262437,MDEyOklzc3VlQ29tbWVudDY4MDI2MjQzNw==,9599,2020-08-25T20:49:24Z,2023-08-23T21:34:09Z,OWNER,"The alternative to this would be to use regular databases and control access to them using [Authentication and permissions](https://docs.datasette.io/en/stable/authentication.html). My concern there is that it's just too easy for someone to mess up their configuration, which would be really bad. I like the idea of a much stronger defense mechanism specifically designed for secrets that should not be exposed. Outside of secrets, passwords and tokens this mechanism could also be useful for the use-case of using Datasette to power websites - as seen on https://www.niche-museums.com/ and https://www.rockybeaches.com/ - maybe those sites don't want to expose their data through their API but still want to use `datasette-template-sql` and the `graphql()` template tag in `datasette-graphql` to render data.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",685806511, https://github.com/simonw/datasette/issues/2150#issuecomment-1690438270,https://api.github.com/repos/simonw/datasette/issues/2150,1690438270,IC_kwDOBm6k_c5kwgp-,9599,2023-08-23T18:27:47Z,2023-08-23T18:27:47Z,OWNER,"I added `outline: 3px dotted pink` to that `form label` block to help spot where else it's being used. Oh interesting, looks like it's over-ridden here too: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783, https://github.com/simonw/datasette/issues/2150#issuecomment-1690435866,https://api.github.com/repos/simonw/datasette/issues/2150,1690435866,IC_kwDOBm6k_c5kwgEa,9599,2023-08-23T18:25:51Z,2023-08-23T18:25:51Z,OWNER,"Looks like that affects a few forms: The search form on the table page over-rides it already: ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783, https://github.com/simonw/datasette/issues/2150#issuecomment-1690432928,https://api.github.com/repos/simonw/datasette/issues/2150,1690432928,IC_kwDOBm6k_c5kwfWg,9599,2023-08-23T18:23:26Z,2023-08-23T18:23:26Z,OWNER,"That should be scoped just to the query filters form on the table page. I'll fix it in `main` but I'm still going to ship a fix for those plugins separately so they work well before people upgrade to the next Datasette release with this change in it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783, https://github.com/simonw/datasette/issues/2150#issuecomment-1690431509,https://api.github.com/repos/simonw/datasette/issues/2150,1690431509,IC_kwDOBm6k_c5kwfAV,9599,2023-08-23T18:22:47Z,2023-08-23T18:22:47Z,OWNER,https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/static/app.css#L485-L489,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1863810783, https://github.com/simonw/datasette/issues/2091#issuecomment-1690423878,https://api.github.com/repos/simonw/datasette/issues/2091,1690423878,IC_kwDOBm6k_c5kwdJG,9599,2023-08-23T18:18:18Z,2023-08-23T18:18:18Z,OWNER,"Dupe of: - #2097 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1781022369, https://github.com/simonw/datasette/pull/2148#issuecomment-1689133247,https://api.github.com/repos/simonw/datasette/issues/2148,1689133247,IC_kwDOBm6k_c5kriC_,22429695,2023-08-23T01:36:42Z,2023-08-23T03:09:43Z,NONE,"## [Codecov](https://app.codecov.io/gh/simonw/datasette/pull/2148?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report Patch coverage has no change and project coverage change: **`+0.01%`** :tada: > Comparison is base [(`17ec309`)](https://app.codecov.io/gh/simonw/datasette/commit/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.71% compared to head [(`a651d3c`)](https://app.codecov.io/gh/simonw/datasette/pull/2148?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) 92.73%. > Report is 2 commits behind head on main.
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2148 +/- ## ========================================== + Coverage 92.71% 92.73% +0.01% ========================================== Files 40 40 Lines 5919 5919 ========================================== + Hits 5488 5489 +1 + Misses 431 430 -1 ``` [see 1 file with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2148/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/simonw/datasette/pull/2148?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334, https://github.com/simonw/datasette/issues/2123#issuecomment-1689207309,https://api.github.com/repos/simonw/datasette/issues/2123,1689207309,IC_kwDOBm6k_c5kr0IN,9599,2023-08-23T03:07:27Z,2023-08-23T03:07:27Z,OWNER,"> I'm happy to debug and land a patch if it's welcome. Yes please! What an odd bug.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1825007061, https://github.com/simonw/datasette/issues/2147#issuecomment-1689206768,https://api.github.com/repos/simonw/datasette/issues/2147,1689206768,IC_kwDOBm6k_c5krz_w,9599,2023-08-23T03:06:32Z,2023-08-23T03:06:32Z,OWNER,"I'm less convinced by the ""rewrite the query in some way"" optional idea. What kind of use-cases can you imagine for that? My hunch is that it's much more likely to cause weird breakages than it is to allow for useful plugin extensions, but I'm willing to be convinced otherwise.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1858228057, https://github.com/simonw/datasette/issues/2147#issuecomment-1689206170,https://api.github.com/repos/simonw/datasette/issues/2147,1689206170,IC_kwDOBm6k_c5krz2a,9599,2023-08-23T03:05:32Z,2023-08-23T03:05:32Z,OWNER,"Interestingly enough there's actually a mechanism that looks like that a bit already: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/views/database.py#L496-L508 That `validate_sql_select()` function is defined here: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/utils/__init__.py#L256-L265 Against these constants: https://github.com/simonw/datasette/blob/64fd1d788eeed2624f107ac699f2370590ae1aa3/datasette/utils/__init__.py#L223-L253 Which isn't a million miles away from your suggestion to have a hook that can say if the query should be executed or not.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1858228057, https://github.com/simonw/datasette/pull/2148#issuecomment-1689198413,https://api.github.com/repos/simonw/datasette/issues/2148,1689198413,IC_kwDOBm6k_c5krx9N,49699333,2023-08-23T02:57:55Z,2023-08-23T02:57:55Z,CONTRIBUTOR,"Looks like this PR has been edited by someone other than Dependabot. That means Dependabot can't rebase it - sorry! If you're happy for Dependabot to recreate it from scratch, overwriting any edits, you can request `@dependabot recreate`. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334, https://github.com/simonw/datasette/pull/2148#issuecomment-1689198368,https://api.github.com/repos/simonw/datasette/issues/2148,1689198368,IC_kwDOBm6k_c5krx8g,9599,2023-08-23T02:57:53Z,2023-08-23T02:57:53Z,OWNER,@dependabot rebase,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334, https://github.com/simonw/datasette/pull/2148#issuecomment-1689177556,https://api.github.com/repos/simonw/datasette/issues/2148,1689177556,IC_kwDOBm6k_c5krs3U,9599,2023-08-23T02:44:29Z,2023-08-23T02:44:29Z,OWNER,"Simplest possible solution is to only run the `pip install .[docs]` bit under Python 3.9+, ditto for the docs tests. I think I'll try that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1859415334,