{"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724084199", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724084199, "node_id": "IC_kwDOBm6k_c5mw2_n", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:47:01Z", "updated_at": "2023-09-18T17:47:01Z", "author_association": "OWNER", "body": "I managed to trigger it by loading `http://127.0.0.1:8045/airtable_refs/airtable_refs` - which worked - and then hitting refresh on that page a bunch of times until it hung.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724083324", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724083324, "node_id": "IC_kwDOBm6k_c5mw2x8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:46:21Z", "updated_at": "2023-09-18T17:46:21Z", "author_association": "OWNER", "body": "Sometimes it takes a few clicks for the bug to occur, but it does seem to always be within the in-memory database.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724081909", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724081909, "node_id": "IC_kwDOBm6k_c5mw2b1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:45:27Z", "updated_at": "2023-09-18T17:45:27Z", "author_association": "OWNER", "body": "Maybe it's not related to faceting - I just got it on a hit to `http://127.0.0.1:8045/airtable_refs/airtable_refs` instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724072390", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724072390, "node_id": "IC_kwDOBm6k_c5mw0HG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:39:06Z", "updated_at": "2023-09-18T17:39:06Z", "author_association": "OWNER", "body": "Landing a version of that test anyway.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724064440", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724064440, "node_id": "IC_kwDOBm6k_c5mwyK4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:36:00Z", "updated_at": "2023-09-18T17:36:00Z", "author_association": "OWNER", "body": "I wrote this test, but it passes:\r\n```python\r\n@pytest.mark.asyncio\r\nasync def test_facet_against_in_memory_database():\r\n ds = Datasette()\r\n db = ds.add_memory_database(\"mem\")\r\n await db.execute_write(\"create table t (id integer primary key, name text)\")\r\n await db.execute_write_many(\r\n \"insert into t (name) values (?)\", [[\"one\"], [\"one\"], [\"two\"]]\r\n )\r\n response1 = await ds.client.get(\"/mem/t.json\")\r\n assert response1.status_code == 200\r\n response2 = await ds.client.get(\"/mem/t.json?_facet=name\")\r\n assert response2.status_code == 200\r\n assert response2.json() == {\r\n \"ok\": True,\r\n \"next\": None,\r\n \"facet_results\": {\r\n \"results\": {\r\n \"name\": {\r\n \"name\": \"name\",\r\n \"type\": \"column\",\r\n \"hideable\": True,\r\n \"toggle_url\": \"/mem/t.json\",\r\n \"results\": [\r\n {\r\n \"value\": \"one\",\r\n \"label\": \"one\",\r\n \"count\": 2,\r\n \"toggle_url\": \"http://localhost/mem/t.json?_facet=name&name=one\",\r\n \"selected\": False,\r\n },\r\n {\r\n \"value\": \"two\",\r\n \"label\": \"two\",\r\n \"count\": 1,\r\n \"toggle_url\": \"http://localhost/mem/t.json?_facet=name&name=two\",\r\n \"selected\": False,\r\n },\r\n ],\r\n \"truncated\": False,\r\n }\r\n },\r\n \"timed_out\": [],\r\n },\r\n \"rows\": [\r\n {\"id\": 1, \"name\": \"one\"},\r\n {\"id\": 2, \"name\": \"one\"},\r\n {\"id\": 3, \"name\": \"two\"},\r\n ],\r\n \"truncated\": False,\r\n }\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724055823", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724055823, "node_id": "IC_kwDOBm6k_c5mwwEP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:31:10Z", "updated_at": "2023-09-18T17:31:10Z", "author_association": "OWNER", "body": "That line was added in https://github.com/simonw/datasette/commit/942411ef946e9a34a2094944d3423cddad27efd3 which first shipped in 0.62a0.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724051886", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724051886, "node_id": "IC_kwDOBm6k_c5mwvGu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:28:20Z", "updated_at": "2023-09-18T17:30:30Z", "author_association": "OWNER", "body": "The bug exhibits when I try to add a facet. I think it's caused by the parallel query execution I added to facets at some point.\r\n\r\nhttp://127.0.0.1:8045/airtable_refs/airtable_refs - no error\r\nhttp://127.0.0.1:8045/airtable_refs/airtable_refs?_facet=table_name#facet-table_name - hangs the server\r\n\r\nCrucial line in the traceback:\r\n```\r\n await gather(execute_facets(), execute_suggested_facets())\r\n```\r\nFrom here: https://github.com/simonw/datasette/blob/917272c864ad7b8a00c48c77f5c2944093babb4e/datasette/views/table.py#L568", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724049538", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724049538, "node_id": "IC_kwDOBm6k_c5mwuiC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:26:44Z", "updated_at": "2023-09-18T17:26:44Z", "author_association": "OWNER", "body": "Just managed to get this exception trace:\r\n```\r\n return await self.route_path(scope, receive, send, path)\r\n File \"/Users/simon/.local/share/virtualenvs/airtable-export-Ca4U-3qk/lib/python3.8/site-packages/datasette/app.py\", line 1354, in route_path\r\n response = await view(request, send)\r\n File \"/Users/simon/.local/share/virtualenvs/airtable-export-Ca4U-3qk/lib/python3.8/site-packages/datasette/views/base.py\", line 134, in view\r\n return await self.dispatch_request(request)\r\n File \"/Users/simon/.local/share/virtualenvs/airtable-export-Ca4U-3qk/lib/python3.8/site-packages/datasette/views/base.py\", line 91, in dispatch_request\r\n return await handler(request)\r\n File \"/Users/simon/.local/share/virtualenvs/airtable-export-Ca4U-3qk/lib/python3.8/site-packages/datasette/views/base.py\", line 361, in get\r\n response_or_template_contexts = await self.data(request, **data_kwargs)\r\n File \"/Users/simon/.local/share/virtualenvs/airtable-export-Ca4U-3qk/lib/python3.8/site-packages/datasette/views/table.py\", line 158, in data\r\n return await self._data_traced(request, default_labels, _next, _size)\r\n File \"/Users/simon/.local/share/virtualenvs/airtable-export-Ca4U-3qk/lib/python3.8/site-packages/datasette/views/table.py\", line 568, in _data_traced\r\n await gather(execute_facets(), execute_suggested_facets())\r\n File \"/Users/simon/.local/share/virtualenvs/airtable-export-Ca4U-3qk/lib/python3.8/site-packages/datasette/views/table.py\", line 177, in _gather_parallel\r\n return await asyncio.gather(*args)\r\nasyncio.exceptions.CancelledError\r\nINFO: 127.0.0.1:64109 - \"GET /airtable_refs/airtable_refs?_facet=table_name&table_name=Sessions HTTP/1.1\" 500 Internal Server Error\r\n^CError in atexit._run_exitfuncs:\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.pyenv/versions/3.8.17/lib/python3.8/concurrent/futures/thread.py\", line 40, in _python_exit\r\n t.join()\r\n File \"/Users/simon/.pyenv/versions/3.8.17/lib/python3.8/threading.py\", line 1011, in join\r\n self._wait_for_tstate_lock()\r\n File \"/Users/simon/.pyenv/versions/3.8.17/lib/python3.8/threading.py\", line 1027, in _wait_for_tstate_lock\r\n elif lock.acquire(block, timeout):\r\nKeyboardInterrupt\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724048314", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724048314, "node_id": "IC_kwDOBm6k_c5mwuO6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:25:55Z", "updated_at": "2023-09-18T17:25:55Z", "author_association": "OWNER", "body": "The good news is that this bug is currently unlikely to affect most users since named in-memory databases (created using `datasette.add_memory_database(\"airtable_refs\")` ([docs](https://docs.datasette.io/en/stable/internals.html#add-memory-database-name)) are a pretty obscure feature, only available to plugins.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724045748", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724045748, "node_id": "IC_kwDOBm6k_c5mwtm0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:24:07Z", "updated_at": "2023-09-18T17:24:07Z", "author_association": "OWNER", "body": "I need reliable steps to reproduce, then I can bisect and figure out which exact version of Datasette introduced the problem.\r\n\r\nI have a hunch that it relates to changes made to the `datasette/database.py` module, maybe one of these changes here: https://github.com/simonw/datasette/compare/0.61...0.63.1#diff-4e20309c969326a0008dc9237f6807f48d55783315fbfc1e7dfa480b550e16f9", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2188#issuecomment-1722662413", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2188", "id": 1722662413, "node_id": "IC_kwDOBm6k_c5mrb4N", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T02:01:34Z", "updated_at": "2023-09-18T02:01:34Z", "author_association": "OWNER", "body": "I'm not interested in these in Datasette core itself, but I think they have a ton of potential for plugins.\r\n\r\nI wonder what the best way to handle that would be?\r\n\r\nRight now it's possible to write a plugin that [adds extra routes](https://docs.datasette.io/en/stable/plugin_hooks.html#register-routes-datasette), so someone could build a `/dbname/-/prql?query=xxx` endpoint.\r\n\r\nIf this could return JSON, they could add JavaScript to the `/dbname` page that provided a UI for kicking off one of those queries.\r\n\r\nSomething that could make that more ergonomic might be the plugin hook that allows plugins to add extra HTML to different core database pages - e.g. adding a \"Query this database using PRQL\" button or link or even a full form at the top of that database page. That's this issue here:\r\n\r\n- #1191", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1900026059, "label": "Plugin Hooks for \"compile to SQL\" languages"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1722323967", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1722323967, "node_id": "IC_kwDOBm6k_c5mqJP_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-16T21:54:33Z", "updated_at": "2023-09-16T21:54:33Z", "author_association": "OWNER", "body": "Just found this migration guide: https://importlib-metadata.readthedocs.io/en/latest/migration.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1722266942", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1722266942, "node_id": "IC_kwDOBm6k_c5mp7U-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-16T16:38:27Z", "updated_at": "2023-09-16T16:38:27Z", "author_association": "OWNER", "body": "The `importlib.metadata.entry_points()` function is pretty interesting:\r\n```pycon\r\n>>> import importlib.metadata\r\n>>> from pprint import pprint\r\n>>> pprint(importlib.metadata.entry_points())\r\n{'babel.checkers': [EntryPoint(name='num_plurals', value='babel.messages.checkers:num_plurals', group='babel.checkers'),\r\n EntryPoint(name='python_format', value='babel.messages.checkers:python_format', group='babel.checkers')],\r\n 'babel.extractors': [EntryPoint(name='jinja2', value='jinja2.ext:babel_extract[i18n]', group='babel.extractors'),\r\n EntryPoint(name='ignore', value='babel.messages.extract:extract_nothing', group='babel.extractors'),\r\n EntryPoint(name='javascript', value='babel.messages.extract:extract_javascript', group='babel.extractors'),\r\n EntryPoint(name='python', value='babel.messages.extract:extract_python', group='babel.extractors')],\r\n 'console_scripts': [EntryPoint(name='datasette', value='datasette.cli:cli', group='console_scripts'),\r\n EntryPoint(name='normalizer', value='charset_normalizer.cli.normalizer:cli_detect', group='console_scripts'),\r\n EntryPoint(name='pypprint', value='pprintpp:console', group='console_scripts'),\r\n EntryPoint(name='cog', value='cogapp:main', group='console_scripts'),\r\n EntryPoint(name='icdiff', value='icdiff:start', group='console_scripts'),\r\n EntryPoint(name='pycodestyle', value='pycodestyle:_main', group='console_scripts'),\r\n EntryPoint(name='sphinx-autobuild', value='sphinx_autobuild.__main__:main', group='console_scripts'),\r\n EntryPoint(name='sphinx-apidoc', value='sphinx.ext.apidoc:main', group='console_scripts'),\r\n EntryPoint(name='sphinx-autogen', value='sphinx.ext.autosummary.generate:main', group='console_scripts'),\r\n EntryPoint(name='sphinx-build', value='sphinx.cmd.build:main', group='console_scripts'),\r\n EntryPoint(name='sphinx-quickstart', value='sphinx.cmd.quickstart:main', group='console_scripts'),\r\n EntryPoint(name='sphinx-to-sqlite', value='sphinx_to_sqlite.cli:cli', group='console_scripts'),\r\n EntryPoint(name='pybabel', value='babel.messages.frontend:main', group='console_scripts'),\r\n EntryPoint(name='docutils', value='docutils.__main__:main', group='console_scripts'),\r\n EntryPoint(name='isort', value='isort.main:main', group='console_scripts'),\r\n EntryPoint(name='isort-identify-imports', value='isort.main:identify_imports_main', group='console_scripts'),\r\n EntryPoint(name='hupper', value='hupper.cli:main', group='console_scripts'),\r\n EntryPoint(name='sqlite-utils', value='sqlite_utils.cli:cli', group='console_scripts'),\r\n EntryPoint(name='py.test', value='pytest:console_main', group='console_scripts'),\r\n EntryPoint(name='pytest', value='pytest:console_main', group='console_scripts'),\r\n EntryPoint(name='pyflakes', value='pyflakes.api:main', group='console_scripts'),\r\n EntryPoint(name='livereload', value='livereload.cli:main', group='console_scripts'),\r\n EntryPoint(name='uvicorn', value='uvicorn.main:main', group='console_scripts'),\r\n EntryPoint(name='httpx', value='httpx:main', group='console_scripts'),\r\n EntryPoint(name='flake8', value='flake8.main.cli:main', group='console_scripts'),\r\n EntryPoint(name='blacken-docs', value='blacken_docs:main', group='console_scripts'),\r\n EntryPoint(name='pip', value='pip._internal.cli.main:main', group='console_scripts'),\r\n EntryPoint(name='pip3', value='pip._internal.cli.main:main', group='console_scripts'),\r\n EntryPoint(name='pip3.10', value='pip._internal.cli.main:main', group='console_scripts'),\r\n EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts'),\r\n EntryPoint(name='pygmentize', value='pygments.cmdline:main', group='console_scripts'),\r\n EntryPoint(name='black', value='black:patched_main', group='console_scripts'),\r\n EntryPoint(name='blackd', value='blackd:patched_main [d]', group='console_scripts'),\r\n EntryPoint(name='codespell', value='codespell_lib:_script_main', group='console_scripts'),\r\n EntryPoint(name='tabulate', value='tabulate:_main', group='console_scripts')],\r\n 'datasette': [EntryPoint(name='debug_permissions', value='datasette_debug_permissions', group='datasette'),\r\n EntryPoint(name='codespaces', value='datasette_codespaces', group='datasette'),\r\n EntryPoint(name='vega', value='datasette_vega', group='datasette'),\r\n EntryPoint(name='x_forwarded_host', value='datasette_x_forwarded_host', group='datasette'),\r\n EntryPoint(name='json_html', value='datasette_json_html', group='datasette'),\r\n EntryPoint(name='datasette_write_ui', value='datasette_write_ui', group='datasette'),\r\n EntryPoint(name='pretty_json', value='datasette_pretty_json', group='datasette'),\r\n EntryPoint(name='graphql', value='datasette_graphql', group='datasette')],\r\n 'distutils.commands': [EntryPoint(name='compile_catalog', value='babel.messages.frontend:compile_catalog', group='distutils.commands'),\r\n EntryPoint(name='extract_messages', value='babel.messages.frontend:extract_messages', group='distutils.commands'),\r\n EntryPoint(name='init_catalog', value='babel.messages.frontend:init_catalog', group='distutils.commands'),\r\n EntryPoint(name='update_catalog', value='babel.messages.frontend:update_catalog', group='distutils.commands'),\r\n EntryPoint(name='isort', value='isort.setuptools_commands:ISortCommand', group='distutils.commands'),\r\n EntryPoint(name='alias', value='setuptools.command.alias:alias', group='distutils.commands'),\r\n EntryPoint(name='bdist_egg', value='setuptools.command.bdist_egg:bdist_egg', group='distutils.commands'),\r\n EntryPoint(name='bdist_rpm', value='setuptools.command.bdist_rpm:bdist_rpm', group='distutils.commands'),\r\n EntryPoint(name='build', value='setuptools.command.build:build', group='distutils.commands'),\r\n EntryPoint(name='build_clib', value='setuptools.command.build_clib:build_clib', group='distutils.commands'),\r\n EntryPoint(name='build_ext', value='setuptools.command.build_ext:build_ext', group='distutils.commands'),\r\n EntryPoint(name='build_py', value='setuptools.command.build_py:build_py', group='distutils.commands'),\r\n EntryPoint(name='develop', value='setuptools.command.develop:develop', group='distutils.commands'),\r\n EntryPoint(name='dist_info', value='setuptools.command.dist_info:dist_info', group='distutils.commands'),\r\n EntryPoint(name='easy_install', value='setuptools.command.easy_install:easy_install', group='distutils.commands'),\r\n EntryPoint(name='editable_wheel', value='setuptools.command.editable_wheel:editable_wheel', group='distutils.commands'),\r\n EntryPoint(name='egg_info', value='setuptools.command.egg_info:egg_info', group='distutils.commands'),\r\n EntryPoint(name='install', value='setuptools.command.install:install', group='distutils.commands'),\r\n EntryPoint(name='install_egg_info', value='setuptools.command.install_egg_info:install_egg_info', group='distutils.commands'),\r\n EntryPoint(name='install_lib', value='setuptools.command.install_lib:install_lib', group='distutils.commands'),\r\n EntryPoint(name='install_scripts', value='setuptools.command.install_scripts:install_scripts', group='distutils.commands'),\r\n EntryPoint(name='rotate', value='setuptools.command.rotate:rotate', group='distutils.commands'),\r\n EntryPoint(name='saveopts', value='setuptools.command.saveopts:saveopts', group='distutils.commands'),\r\n EntryPoint(name='sdist', value='setuptools.command.sdist:sdist', group='distutils.commands'),\r\n EntryPoint(name='setopt', value='setuptools.command.setopt:setopt', group='distutils.commands'),\r\n EntryPoint(name='test', value='setuptools.command.test:test', group='distutils.commands'),\r\n EntryPoint(name='upload_docs', value='setuptools.command.upload_docs:upload_docs', group='distutils.commands'),\r\n EntryPoint(name='bdist_wheel', value='wheel.bdist_wheel:bdist_wheel', group='distutils.commands')],\r\n 'distutils.setup_keywords': [EntryPoint(name='message_extractors', value='babel.messages.frontend:check_message_extractors', group='distutils.setup_keywords'),\r\n EntryPoint(name='cffi_modules', value='cffi.setuptools_ext:cffi_modules', group='distutils.setup_keywords'),\r\n EntryPoint(name='dependency_links', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),\r\n EntryPoint(name='eager_resources', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),\r\n EntryPoint(name='entry_points', value='setuptools.dist:check_entry_points', group='distutils.setup_keywords'),\r\n EntryPoint(name='exclude_package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),\r\n EntryPoint(name='extras_require', value='setuptools.dist:check_extras', group='distutils.setup_keywords'),\r\n EntryPoint(name='include_package_data', value='setuptools.dist:assert_bool', group='distutils.setup_keywords'),\r\n EntryPoint(name='install_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),\r\n EntryPoint(name='namespace_packages', value='setuptools.dist:check_nsp', group='distutils.setup_keywords'),\r\n EntryPoint(name='package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),\r\n EntryPoint(name='packages', value='setuptools.dist:check_packages', group='distutils.setup_keywords'),\r\n EntryPoint(name='python_requires', value='setuptools.dist:check_specifier', group='distutils.setup_keywords'),\r\n EntryPoint(name='setup_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),\r\n EntryPoint(name='test_loader', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),\r\n EntryPoint(name='test_runner', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),\r\n EntryPoint(name='test_suite', value='setuptools.dist:check_test_suite', group='distutils.setup_keywords'),\r\n EntryPoint(name='tests_require', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),\r\n EntryPoint(name='use_2to3', value='setuptools.dist:invalid_unless_false', group='distutils.setup_keywords'),\r\n EntryPoint(name='zip_safe', value='setuptools.dist:assert_bool', group='distutils.setup_keywords')],\r\n 'egg_info.writers': [EntryPoint(name='PKG-INFO', value='setuptools.command.egg_info:write_pkg_info', group='egg_info.writers'),\r\n EntryPoint(name='dependency_links.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),\r\n EntryPoint(name='depends.txt', value='setuptools.command.egg_info:warn_depends_obsolete', group='egg_info.writers'),\r\n EntryPoint(name='eager_resources.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),\r\n EntryPoint(name='entry_points.txt', value='setuptools.command.egg_info:write_entries', group='egg_info.writers'),\r\n EntryPoint(name='namespace_packages.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),\r\n EntryPoint(name='requires.txt', value='setuptools.command.egg_info:write_requirements', group='egg_info.writers'),\r\n EntryPoint(name='top_level.txt', value='setuptools.command.egg_info:write_toplevel_names', group='egg_info.writers')],\r\n 'flake8.extension': [EntryPoint(name='C90', value='mccabe:McCabeChecker', group='flake8.extension'),\r\n EntryPoint(name='E', value='flake8.plugins.pycodestyle:pycodestyle_logical', group='flake8.extension'),\r\n EntryPoint(name='F', value='flake8.plugins.pyflakes:FlakesChecker', group='flake8.extension'),\r\n EntryPoint(name='W', value='flake8.plugins.pycodestyle:pycodestyle_physical', group='flake8.extension')],\r\n 'flake8.report': [EntryPoint(name='default', value='flake8.formatting.default:Default', group='flake8.report'),\r\n EntryPoint(name='pylint', value='flake8.formatting.default:Pylint', group='flake8.report'),\r\n EntryPoint(name='quiet-filename', value='flake8.formatting.default:FilenameOnly', group='flake8.report'),\r\n EntryPoint(name='quiet-nothing', value='flake8.formatting.default:Nothing', group='flake8.report')],\r\n 'pylama.linter': [EntryPoint(name='isort', value='isort.pylama_isort:Linter', group='pylama.linter')],\r\n 'pytest11': [EntryPoint(name='icdiff', value='pytest_icdiff', group='pytest11'),\r\n EntryPoint(name='asyncio', value='pytest_asyncio.plugin', group='pytest11'),\r\n EntryPoint(name='xdist', value='xdist.plugin', group='pytest11'),\r\n EntryPoint(name='xdist.looponfail', value='xdist.looponfail', group='pytest11'),\r\n EntryPoint(name='timeout', value='pytest_timeout', group='pytest11'),\r\n EntryPoint(name='anyio', value='anyio.pytest_plugin', group='pytest11')],\r\n 'setuptools.finalize_distribution_options': [EntryPoint(name='keywords', value='setuptools.dist:Distribution._finalize_setup_keywords', group='setuptools.finalize_distribution_options'),\r\n EntryPoint(name='parent_finalize', value='setuptools.dist:_Distribution.finalize_options', group='setuptools.finalize_distribution_options')],\r\n 'sphinx.html_themes': [EntryPoint(name='alabaster', value='alabaster', group='sphinx.html_themes'),\r\n EntryPoint(name='basic-ng', value='sphinx_basic_ng', group='sphinx.html_themes'),\r\n EntryPoint(name='furo', value='furo', group='sphinx.html_themes')],\r\n 'sqlite_utils': [EntryPoint(name='hello_world', value='sqlite_utils_hello_world', group='sqlite_utils')]}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1722266513", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1722266513, "node_id": "IC_kwDOBm6k_c5mp7OR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-16T16:36:09Z", "updated_at": "2023-09-16T16:36:09Z", "author_association": "OWNER", "body": "Now I need to switch out `pkg_resources` in `plugins.py`:\r\n\r\nhttps://github.com/simonw/datasette/blob/852f5014853943fa27f43ddaa2d442545b3259fb/datasette/plugins.py#L33-L74", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1722265848", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1722265848, "node_id": "IC_kwDOBm6k_c5mp7D4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-16T16:32:42Z", "updated_at": "2023-09-16T16:32:42Z", "author_association": "OWNER", "body": "Here's the exception it uses:\r\n```pycon\r\n>>> importlib.metadata.version(\"datasette\")\r\n'1.0a6'\r\n>>> importlib.metadata.version(\"datasette2\")\r\nTraceback (most recent call last):\r\n File \"\", line 1, in \r\n File \"/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py\", line 996, in version\r\n return distribution(distribution_name).version\r\n File \"/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py\", line 969, in distribution\r\n return Distribution.from_name(distribution_name)\r\n File \"/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py\", line 548, in from_name\r\n raise PackageNotFoundError(name)\r\nimportlib.metadata.PackageNotFoundError: No package metadata was found for datasette2\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1722258980", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1722258980, "node_id": "IC_kwDOBm6k_c5mp5Yk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-16T15:56:45Z", "updated_at": "2023-09-16T15:56:45Z", "author_association": "OWNER", "body": "Weird, I still can't get the warning to show even with this:\r\n```python\r\n@pytest.mark.asyncio\r\nasync def test_plugin_is_installed():\r\n datasette = Datasette(memory=True)\r\n\r\n class DummyPlugin:\r\n __name__ = \"DummyPlugin\"\r\n\r\n @hookimpl\r\n def actors_from_ids(self, datasette, actor_ids):\r\n return {}\r\n\r\n try:\r\n pm.register(DummyPlugin(), name=\"DummyPlugin\")\r\n response = await datasette.client.get(\"/-/plugins.json\")\r\n assert response.status_code == 200\r\n installed_plugins = {p[\"name\"] for p in response.json()}\r\n assert \"DummyPlugin\" in installed_plugins\r\n\r\n finally:\r\n pm.unregister(name=\"ReturnNothingPlugin\")\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1722257328", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1722257328, "node_id": "IC_kwDOBm6k_c5mp4-w", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-16T15:47:32Z", "updated_at": "2023-09-16T15:47:32Z", "author_association": "OWNER", "body": "Frustrating that this warning doesn't show up in the Datasette test suite itself. It shows up in plugin test suites that run this test:\r\n```python\r\n@pytest.mark.asyncio\r\nasync def test_plugin_is_installed():\r\n datasette = Datasette(memory=True)\r\n response = await datasette.client.get(\"/-/plugins.json\")\r\n assert response.status_code == 200\r\n installed_plugins = {p[\"name\"] for p in response.json()}\r\n assert \"datasette-chronicle\" in installed_plugins\r\n```\r\nIf you run that test inside Datasette core `installed_plugins` is an empty set, which presumably is why the warning doesn't get triggered there.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2186#issuecomment-1721742055", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2186", "id": 1721742055, "node_id": "IC_kwDOBm6k_c5mn7Ln", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-15T19:27:59Z", "updated_at": "2023-09-15T19:27:59Z", "author_association": "OWNER", "body": "This feels like it might be quite a nice pattern generally - providing optional arguments to plugins and views that can be `await get_x()` called to run an extra calculation.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1898927976, "label": "Mechanism for register_output_renderer hooks to access full count"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2186#issuecomment-1721740872", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2186", "id": 1721740872, "node_id": "IC_kwDOBm6k_c5mn65I", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-15T19:26:51Z", "updated_at": "2023-09-15T19:27:19Z", "author_association": "OWNER", "body": "Here's where it's called at the moment: https://github.com/simonw/datasette/blob/16f0b6d8222d06682a31b904d0a402c391ae1c1c/datasette/views/base.py#L297-L313\r\n\r\nAnd the docs: https://github.com/simonw/datasette/blob/1.0a6/docs/plugin_hooks.rst#register-output-renderer-datasette\r\n\r\nI'm tempted to add a `get_count` argument which, when awaited, returns the full count. Then plugins could do this:\r\n\r\n```python\r\nasync def render_notebook(datasette, request, get_count, rows):\r\n count = await get_count()\r\n # ...\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1898927976, "label": "Mechanism for register_output_renderer hooks to access full count"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2183#issuecomment-1718316733", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2183", "id": 1718316733, "node_id": "IC_kwDOBm6k_c5ma269", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-13T21:05:36Z", "updated_at": "2023-09-13T21:05:36Z", "author_association": "OWNER", "body": "I'm going to land this and make any further documentation tweaks on `main`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1891212159, "label": "`datasette.yaml` plugin support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/594#issuecomment-1714920708", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/594", "id": 1714920708, "node_id": "IC_kwDOCGYnMM5mN50E", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-12T03:51:13Z", "updated_at": "2023-09-12T03:51:13Z", "author_association": "OWNER", "body": "Changing this without breaking backwards compatibility (and forcing a 4.0 release) will be tricky, because `ForeignKey()` is a `namedtuple`:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/622c3a5a7dd53a09c029e2af40c2643fe7579340/sqlite_utils/db.py#L148-L150\r\n\r\nI could swap it out for a `dataclass` and add those extra columns, but I need to make sure that code like this still works:\r\n```python\r\nfor table, column, other_table, other_column in table.foreign_keys:\r\n # ...\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1891614971, "label": "Represent compound foreign keys in table.foreign_keys output"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/594#issuecomment-1714919806", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/594", "id": 1714919806, "node_id": "IC_kwDOCGYnMM5mN5l-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-12T03:49:41Z", "updated_at": "2023-09-12T03:49:41Z", "author_association": "OWNER", "body": "Digging in a bit more:\r\n```pycon\r\n>>> pprint(list(db.query('PRAGMA foreign_key_list(courses)')))\r\n[{'from': 'campus_name',\r\n 'id': 0,\r\n 'match': 'NONE',\r\n 'on_delete': 'NO ACTION',\r\n 'on_update': 'NO ACTION',\r\n 'seq': 0,\r\n 'table': 'departments',\r\n 'to': 'campus_name'},\r\n {'from': 'dept_code',\r\n 'id': 0,\r\n 'match': 'NONE',\r\n 'on_delete': 'NO ACTION',\r\n 'on_update': 'NO ACTION',\r\n 'seq': 1,\r\n 'table': 'departments',\r\n 'to': 'dept_code'}]\r\n```\r\nI think the way you tell it's a compound foreign key is that both of those have the same `id` value - of `0` - but they then have two different `seq` values of `0` and `1`.\r\n\r\nRight now I ignore those columns entirely: https://github.com/simonw/sqlite-utils/blob/622c3a5a7dd53a09c029e2af40c2643fe7579340/sqlite_utils/db.py#L1523-L1540", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1891614971, "label": "Represent compound foreign keys in table.foreign_keys output"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2183#issuecomment-1714699724", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2183", "id": 1714699724, "node_id": "IC_kwDOBm6k_c5mND3M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-11T23:01:36Z", "updated_at": "2023-09-11T23:02:30Z", "author_association": "OWNER", "body": "On thinking about this further, I'm fine releasing it as another alpha provided it causes Datasette to error loudly with an explanatory message if you attempt to load `-m metadata.json` at a metadata file that includes `\"plugins\"` configuration.\r\n\r\nSomething like this:\r\n```bash\r\ndatasette -m metadata.json\r\n```\r\nOutputs:\r\n\r\n> Datasette no longer accepts plugin configuration in --metadata. Move your `\"plugins\"` configuration blocks to a separate file - we suggest calling that `datasette.yaml` - and start Datasette with `datasette -c datasette.yaml`.\r\n\r\nFor added usability points, let's have it suggest `datasette.json` if they used `metadata.json` and `datasette.yaml` if they tried to use `metadata.yaml`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1891212159, "label": "`datasette.yaml` plugin support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2168#issuecomment-1712897194", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2168", "id": 1712897194, "node_id": "IC_kwDOBm6k_c5mGLyq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-10T17:54:07Z", "updated_at": "2023-09-10T17:54:07Z", "author_association": "OWNER", "body": "This looks relevant:\r\n\r\nhttps://pluggy.readthedocs.io/en/stable/#wrappers\r\n\r\n> A *hookimpl* can be marked with the `\"wrapper\"` option, which indicates that the function will be called to *wrap* (or surround) all other normal *hookimpl* calls. A *hook wrapper* can thus execute some code ahead and after the execution of all corresponding non-wrappper *hookimpls*.\r\n\r\nThis could be the perfect mechanism for implementing this hook, although I still need to figure out how it interacts with streaming.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1876353656, "label": "Consider a request/response wrapping hook slightly higher level than asgi_wrapper()"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1712895580", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1712895580, "node_id": "IC_kwDOCGYnMM5mGLZc", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-10T17:46:41Z", "updated_at": "2023-09-10T17:46:41Z", "author_association": "OWNER", "body": "In working on this I learned that `rowid` values in SQLite are way less stable than I had thought - in particular, they are often entirely rewritten on a `VACUUM`:\r\n\r\nhttps://www.sqlite.org/lang_vacuum.html#how_vacuum_works\r\n\r\n> The VACUUM command may change the [ROWIDs](https://www.sqlite.org/lang_createtable.html#rowid) of entries in any tables that do not have an explicit [INTEGER PRIMARY KEY](https://www.sqlite.org/lang_createtable.html#rowid).\r\n\r\nSo this fix wasn't as valuable as I thought. I need to move away from ever assuming that a `rowid` is a useful foreign key for anything.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2180#issuecomment-1712895084", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2180", "id": 1712895084, "node_id": "IC_kwDOBm6k_c5mGLRs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-10T17:44:14Z", "updated_at": "2023-09-10T17:44:14Z", "author_association": "OWNER", "body": "Used by `datasette-short-links` here: https://github.com/datasette/datasette-short-links/blob/468c3e25dbe06a8dcba8edda59bc16a18e126a51/datasette_short_links/__init__.py#L108-L115", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886791100, "label": "Plugin hook: `actors_from_ids()`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2180#issuecomment-1711057080", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2180", "id": 1711057080, "node_id": "IC_kwDOBm6k_c5l_Ki4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T04:28:03Z", "updated_at": "2023-09-08T04:28:03Z", "author_association": "OWNER", "body": "Landed:\r\n\r\n- https://docs.datasette.io/en/latest/plugin_hooks.html#actors-from-ids-datasette-actor-ids\r\n- https://docs.datasette.io/en/latest/internals.html#await-actors-from-ids-actor-ids", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886791100, "label": "Plugin hook: `actors_from_ids()`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2181#issuecomment-1711054840", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2181", "id": 1711054840, "node_id": "IC_kwDOBm6k_c5l_J_4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T04:23:47Z", "updated_at": "2023-09-08T04:23:47Z", "author_association": "OWNER", "body": "I've implemented this hook once now in:\r\n- https://github.com/datasette/datasette-remote-actors\r\n\r\nAnd built and tested a debug tool for it in:\r\n- https://github.com/datasette/datasette-debug-actors-from-ids\r\n\r\nI'm now confident in the design of this plugin hook, I'm going to land it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886812002, "label": "actors_from_ids plugin hook and datasette.actors_from_ids() method"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2180#issuecomment-1711054624", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2180", "id": 1711054624, "node_id": "IC_kwDOBm6k_c5l_J8g", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T04:23:23Z", "updated_at": "2023-09-08T04:23:23Z", "author_association": "OWNER", "body": "I've implemented this hook once now in:\r\n- https://github.com/datasette/datasette-remote-actors\r\n\r\nAnd built and tested a debug tool for it in:\r\n- https://github.com/datasette/datasette-debug-actors-from-ids\r\n\r\nI'm now confident in the design of this plugin hook, I'm going to land it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886791100, "label": "Plugin hook: `actors_from_ids()`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2180#issuecomment-1711028355", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2180", "id": 1711028355, "node_id": "IC_kwDOBm6k_c5l_DiD", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T03:36:02Z", "updated_at": "2023-09-08T03:36:02Z", "author_association": "OWNER", "body": "I shipped the first version of \r\n\r\n- https://github.com/datasette/datasette-remote-actors/issues/1\r\n\r\nWhen I land this plugin in a Datasette release I need to update that repo to depend on the new alpha.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886791100, "label": "Plugin hook: `actors_from_ids()`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2181#issuecomment-1710969448", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2181", "id": 1710969448, "node_id": "IC_kwDOBm6k_c5l-1Jo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T01:53:27Z", "updated_at": "2023-09-08T01:53:27Z", "author_association": "OWNER", "body": "Documentation preview:\r\n- https://datasette--2181.org.readthedocs.build/en/2181/internals.html#await-actors-from-ids-actor-ids\r\n- https://datasette--2181.org.readthedocs.build/en/2181/plugin_hooks.html#plugin-hook-actors-from-ids", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886812002, "label": "actors_from_ids plugin hook and datasette.actors_from_ids() method"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2180#issuecomment-1710969339", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2180", "id": 1710969339, "node_id": "IC_kwDOBm6k_c5l-1H7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T01:53:16Z", "updated_at": "2023-09-08T01:53:16Z", "author_association": "OWNER", "body": "Documentation preview:\r\n- https://datasette--2181.org.readthedocs.build/en/2181/internals.html#await-actors-from-ids-actor-ids\r\n- https://datasette--2181.org.readthedocs.build/en/2181/plugin_hooks.html#plugin-hook-actors-from-ids", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886791100, "label": "Plugin hook: `actors_from_ids()`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2180#issuecomment-1710947637", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2180", "id": 1710947637, "node_id": "IC_kwDOBm6k_c5l-v01", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T01:17:28Z", "updated_at": "2023-09-08T01:17:28Z", "author_association": "OWNER", "body": "I think this is both a plugin hook and a `await datasette.actors_from_ids(actor_ids)` internal API function that calls it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886791100, "label": "Plugin hook: `actors_from_ids()`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710935270", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710935270, "node_id": "IC_kwDOCGYnMM5l-szm", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:55:30Z", "updated_at": "2023-09-08T00:55:30Z", "author_association": "OWNER", "body": "Yes! That recreated the bug:\r\n```\r\n> assert previous_rows == next_rows\r\nE AssertionError: assert equals failed\r\nE [ [ \r\nE (1, '1', 'Paris'), (1, '1', 'Paris'), \r\nE (3, '3', 'New York'), (2, '3', 'New York'), \r\nE (4, '4', 'London'), (3, '4', 'London'), \r\nE ] ...\r\nE \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710934448", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710934448, "node_id": "IC_kwDOCGYnMM5l-smw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:54:03Z", "updated_at": "2023-09-08T00:54:03Z", "author_association": "OWNER", "body": "Oh! Maybe the row ID preservation here is a coincidence because the tables are created from scratch and count 1, 2, 3.\r\n\r\nIf I delete a row from the table and then insert some more - breaking the `rowid` sequence - it might show the bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710933716", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710933716, "node_id": "IC_kwDOCGYnMM5l-sbU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:52:42Z", "updated_at": "2023-09-08T00:52:42Z", "author_association": "OWNER", "body": "I just noticed that the table where I encountered this bug wasn't actually a `rowid` table after all - it had an `id` column that was a text primary key.\r\n\r\nThe reason the `rowid` was important is that's how the FTS mechanism in Datasette relates FTS entries to their rows.\r\n\r\nBut I tried this test and it passed, too:\r\n```python\r\ndef test_transform_preserves_rowids(fresh_db):\r\n fresh_db[\"places\"].insert_all(\r\n [\r\n {\"id\": \"1\", \"name\": \"Paris\", \"country\": \"France\"},\r\n {\"id\": \"2\", \"name\": \"London\", \"country\": \"UK\"},\r\n {\"id\": \"3\", \"name\": \"New York\", \"country\": \"USA\"},\r\n ],\r\n pk=\"id\",\r\n )\r\n previous_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, id, name from places\")\r\n )\r\n # Transform it\r\n fresh_db[\"places\"].transform(column_order=(\"country\", \"name\"))\r\n # Should be the same\r\n next_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, id, name from places\")\r\n )\r\n assert previous_rows == next_rows\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710931605", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710931605, "node_id": "IC_kwDOCGYnMM5l-r6V", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:49:02Z", "updated_at": "2023-09-08T00:49:02Z", "author_association": "OWNER", "body": "I tried bumping that up to 10,000 rows instead of just 3 but the test still passed.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710930934", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/592", "id": 1710930934, "node_id": "IC_kwDOCGYnMM5l-rv2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-08T00:47:57Z", "updated_at": "2023-09-08T00:47:57Z", "author_association": "OWNER", "body": "That's odd, I wrote a test for this just now and it passes already:\r\n```python\r\ndef test_transform_preserves_rowids(fresh_db):\r\n # Create a rowid table\r\n fresh_db[\"places\"].insert_all(\r\n (\r\n {\"name\": \"Paris\", \"country\": \"France\"},\r\n {\"name\": \"London\", \"country\": \"UK\"},\r\n {\"name\": \"New York\", \"country\": \"USA\"},\r\n ),\r\n )\r\n assert fresh_db[\"places\"].use_rowid\r\n previous_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, name from places\")\r\n )\r\n # Transform it\r\n fresh_db[\"places\"].transform(column_order=(\"country\", \"name\"))\r\n # Should be the same\r\n next_rows = list(\r\n tuple(row) for row in fresh_db.execute(\"select rowid, name from places\")\r\n )\r\n assert previous_rows == next_rows\r\n```\r\nSo maybe I'm wrong about the cause of that bug?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886771493, "label": "`table.transform()` should preserve `rowid` values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2174#issuecomment-1708727204", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2174", "id": 1708727204, "node_id": "IC_kwDOBm6k_c5l2Ruk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-06T16:35:39Z", "updated_at": "2023-09-07T23:45:42Z", "author_association": "OWNER", "body": "We can use this here: https://click.palletsprojects.com/en/8.1.x/options/#values-from-environment-variables\r\n```python\r\n@click.option(..., envvar=\"DATASETTE_INTERNAL\")\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884330740, "label": "Use $DATASETTE_INTERNAL in absence of --internal"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2178#issuecomment-1710879239", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2178", "id": 1710879239, "node_id": "IC_kwDOBm6k_c5l-fIH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-07T23:20:32Z", "updated_at": "2023-09-07T23:20:32Z", "author_association": "OWNER", "body": "To test that locally, use this YAML instead:\r\n```yaml\r\ndatabases:\r\n content:\r\n allow:\r\n id: root\r\n tables:\r\n releases:\r\n allow: true\r\n```\r\nAnd:\r\n```yaml\r\nallow:\r\n id: root\r\ndatabases:\r\n content:\r\n tables:\r\n releases:\r\n allow: true", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886350562, "label": "Don't show foreign key links to tables the user cannot access"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2178#issuecomment-1710878391", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2178", "id": 1710878391, "node_id": "IC_kwDOBm6k_c5l-e63", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-07T23:19:05Z", "updated_at": "2023-09-07T23:19:05Z", "author_association": "OWNER", "body": "This fix didn't work on Datasette Cloud. I used `/-/permissions` to debug it and saw this:\r\n\r\n![image](https://github.com/simonw/datasette/assets/9599/61d2bc5f-1f96-41ea-8658-91dfbcb6610c)\r\n\r\nOnly checking `view-table` is not enough: for my instance on Datasette Cloud the view permission check that should have failed was for the database or instance.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886350562, "label": "Don't show foreign key links to tables the user cannot access"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2178#issuecomment-1710871095", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2178", "id": 1710871095, "node_id": "IC_kwDOBm6k_c5l-dI3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-07T23:07:16Z", "updated_at": "2023-09-07T23:07:16Z", "author_association": "OWNER", "body": "I ran this:\r\n\r\n datasette content.db -p 8043 -m fk-auth.yml --root\r\n\r\nAgainst this YAML:\r\n```yaml\r\ndatabases:\r\n content:\r\n tables:\r\n users:\r\n allow:\r\n id: root\r\n```\r\nAnd it worked as it should - here's a screenshot of an anonymous user and a root user viewing the same page:\r\n\r\n![CleanShot 2023-09-07 at 16 05 34@2x](https://github.com/simonw/datasette/assets/9599/3e91da08-107c-421c-8a00-aa650b960c58)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886350562, "label": "Don't show foreign key links to tables the user cannot access"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2178#issuecomment-1710567329", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2178", "id": 1710567329, "node_id": "IC_kwDOBm6k_c5l9S-h", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-07T17:59:59Z", "updated_at": "2023-09-07T17:59:59Z", "author_association": "OWNER", "body": "Should I put the permission check in that undocumented `datasette.expand_foreign_keys()` method? I think so - it should accept `request.actor` as one of its arguments.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886350562, "label": "Don't show foreign key links to tables the user cannot access"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2178#issuecomment-1710565268", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2178", "id": 1710565268, "node_id": "IC_kwDOBm6k_c5l9SeU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-07T17:58:04Z", "updated_at": "2023-09-07T17:59:06Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/fbcb103c0cb6668018ace539a01a6a1f156e8d6a/datasette/views/table.py#L1132-L1149\r\n\r\nWhich calls this undocumented method:\r\n\r\nhttps://github.com/simonw/datasette/blob/fbcb103c0cb6668018ace539a01a6a1f156e8d6a/datasette/app.py#L938-L973", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1886350562, "label": "Don't show foreign key links to tables the user cannot access"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2177#issuecomment-1708777964", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2177", "id": 1708777964, "node_id": "IC_kwDOBm6k_c5l2eHs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-06T17:04:30Z", "updated_at": "2023-09-06T17:04:30Z", "author_association": "OWNER", "body": "Here's the thinking: setting the`DATASETTE_INTERNAL` environment variable for your entire machine will cause any time you run `datasette x.db` to result in a process that shares the same `internal.db` database as other instances.\r\n\r\nYou might run more than one instance at once (I often have 4 or 5 going). This would currently break, because they would over-write each other's catalog tables:\r\n\r\nhttps://github.com/simonw/datasette/blob/e4abae3fd7a828625d00c35c316852ffbaa5ef2f/datasette/utils/internal_db.py#L5-L62\r\n\r\nThe breaking wouldn't be obvious because the catalog tables aren't used by any features yet, but it's still bad.\r\n\r\nThis convinced us that actually we should move those `catalog_` tables OUT of `internal.db`. The `_internal` database will be reserved for plugins that want to use it for caching, storing progress, etc.\r\n\r\nI think we move them to an in-memory `_catalog` database which is excluded from `ds.databases` (like `_internal` is ) but can be accessed using `datasette.get_catalog_database()` - similar to `datasette.get_internal_database()`.\r\n\r\nSo each instance of Datasette gets its own truly private `_catalog`, which is in-memory and so gets cleared at the end of each process.\r\n\r\nAn interesting thing that came up about a shared `_internal` database is that it provides opportunities for things like a notes plugin which allows you to attach notes to any row in any table in a database... where those notes become available to multiple Datasette instances that you might launch on the same laptop.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884408624, "label": "Move schema tables from _internal to _catalog"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2174#issuecomment-1708728926", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2174", "id": 1708728926, "node_id": "IC_kwDOBm6k_c5l2SJe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-06T16:36:56Z", "updated_at": "2023-09-06T16:36:56Z", "author_association": "OWNER", "body": "`DATASETTE_INTERNAL` would be more consistent with `DATASETTE_SECRET`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884330740, "label": "Use $DATASETTE_INTERNAL in absence of --internal"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/591#issuecomment-1708695907", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/591", "id": 1708695907, "node_id": "IC_kwDOCGYnMM5l2KFj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-06T16:15:59Z", "updated_at": "2023-09-06T16:19:14Z", "author_association": "OWNER", "body": "The test failure was while installing `numpy`, relating to importing `distutils` - maybe relevant:\r\n- https://github.com/pypa/setuptools/issues/3661\r\n\r\n```\r\n25h Installing build dependencies: started\r\n Installing build dependencies: finished with status 'done'\r\n Getting requirements to build wheel: started\r\n Getting requirements to build wheel: finished with status 'done'\r\nERROR: Exception:\r\nTraceback (most recent call last):\r\n...\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_internal/utils/misc.py\", line 697, in get_requires_for_build_wheel\r\n return super().get_requires_for_build_wheel(config_settings=cs)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py\", line 166, in get_requires_for_build_wheel\r\n return self._call_hook('get_requires_for_build_wheel', {\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py\", line 321, in _call_hook\r\n raise BackendUnavailable(data.get('traceback', ''))\r\npip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py\", line 77, in _build_backend\r\n obj = import_module(mod_path)\r\n ^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"/opt/hostedtoolcache/Python/3.12.0-rc.2/x64/lib/python3.12/importlib/__init__.py\", line 90, in import_module\r\n return _bootstrap._gcd_import(name[level:], package, level)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File \"\", line 1381, in _gcd_import\r\n File \"\", line 1354, in _find_and_load\r\n File \"\", line 1304, in _find_and_load_unlocked\r\n File \"\", line 488, in _call_with_frames_removed\r\n File \"\", line 1381, in _gcd_import\r\n File \"\", line 1354, in _find_and_load\r\n File \"\", line 1325, in _find_and_load_unlocked\r\n File \"\", line 929, in _load_unlocked\r\n File \"\", line 994, in exec_module\r\n File \"\", line 488, in _call_with_frames_removed\r\n File \"/tmp/pip-build-env-x9nyg3kd/overlay/lib/python3.12/site-packages/setuptools/__init__.py\", line 10, in \r\n import distutils.core\r\nModuleNotFoundError: No module named 'distutils'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884335789, "label": "Test against Python 3.12 preview"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2173#issuecomment-1707570378", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2173", "id": 1707570378, "node_id": "IC_kwDOBm6k_c5lx3TK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-06T02:40:31Z", "updated_at": "2023-09-06T02:40:31Z", "author_association": "OWNER", "body": "Surprising error:\r\n```\r\n blacken-docs -l 60 docs/*.rst\r\n shell: /usr/bin/bash -e {0}\r\n env:\r\n pythonLocation: /opt/hostedtoolcache/Python/3.11.4/x64\r\n PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.4/x64/lib/pkgconfig\r\n Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64\r\n Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64\r\n Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.4/x64\r\n LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.4/x64/lib\r\ndocs/writing_plugins.rst:365: code block parse error Cannot parse: 8:0: \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1883055640, "label": "click-default-group>=1.2.3"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2173#issuecomment-1707565495", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2173", "id": 1707565495, "node_id": "IC_kwDOBm6k_c5lx2G3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-06T02:33:56Z", "updated_at": "2023-09-06T02:33:56Z", "author_association": "OWNER", "body": "Running tests to see if Pyodide works - that was the reason I switched to my `click-default-group-wheel` package originally.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1883055640, "label": "click-default-group>=1.2.3"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/590#issuecomment-1704387161", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/590", "id": 1704387161, "node_id": "IC_kwDOCGYnMM5lluJZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:50:36Z", "updated_at": "2023-09-03T19:50:36Z", "author_association": "OWNER", "body": "Maybe just populate `db.memory: bool` and `db.memory_name: Optional[str]` for this, then document them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879214365, "label": "Ability to tell if a Database is an in-memory one"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704384393", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/589", "id": 1704384393, "node_id": "IC_kwDOCGYnMM5llteJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:36:34Z", "updated_at": "2023-09-03T19:36:34Z", "author_association": "OWNER", "body": "Here's a prototype: https://github.com/simonw/sqlite-utils/commit/62f673835c4a66f87cf6f949eaff43c8b014619b\r\n\r\nStill needs tests and documentation (and some more thought to make sure it's doing the right thing).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879209560, "label": "Mechanism for de-registering registered SQL functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704384111", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/589", "id": 1704384111, "node_id": "IC_kwDOCGYnMM5lltZv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:35:03Z", "updated_at": "2023-09-03T19:35:03Z", "author_association": "OWNER", "body": "Normally in Python/`sqlite3` you de-register a function by passing `None` to it.\r\n\r\nYou can't do that with `db.register_function()` at the moment because a `fn` of `None` does something else:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L461-L464", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879209560, "label": "Mechanism for de-registering registered SQL functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/589#issuecomment-1704383901", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/589", "id": 1704383901, "node_id": "IC_kwDOCGYnMM5lltWd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-03T19:34:05Z", "updated_at": "2023-09-03T19:34:05Z", "author_association": "OWNER", "body": "For that particular case I realized I'd quite like to have a mechanism for applying functions for a block of code and then de-registering them at the end - a context manager.\r\n\r\nI played with this idea a bit:\r\n\r\n```python\r\nwith db.register_functions(md5, md5_random):\r\n db.query(...)\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1879209560, "label": "Mechanism for de-registering registered SQL functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1765#issuecomment-1701894468", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1765", "id": 1701894468, "node_id": "IC_kwDOBm6k_c5lcNlE", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-31T23:08:24Z", "updated_at": "2023-08-31T23:08:24Z", "author_association": "OWNER", "body": "https://docs.datasette.io/en/latest/writing_plugins.html#plugins-that-define-new-plugin-hooks", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1292370469, "label": "Document plugins providing new plugin hook-"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2168#issuecomment-1701831013", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2168", "id": 1701831013, "node_id": "IC_kwDOBm6k_c5lb-Fl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-31T21:51:12Z", "updated_at": "2023-08-31T21:52:15Z", "author_association": "OWNER", "body": "Need to make sure the design of this takes streaming responses into account. Those could be pretty tricky here.\r\n\r\nI nice thing about `asgi_wrapper()` is that it handles those already.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1876353656, "label": "Consider a request/response wrapping hook slightly higher level than asgi_wrapper()"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2168#issuecomment-1701830241", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2168", "id": 1701830241, "node_id": "IC_kwDOBm6k_c5lb95h", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-31T21:50:18Z", "updated_at": "2023-08-31T21:50:18Z", "author_association": "OWNER", "body": "The hook could be called `register_middleware()` and could work like `register_routes()` and `register_commands()`:\r\n\r\n```python\r\n@hookspec\r\ndef register_middleware(datasette):\r\n \"\"\"Register middleware: returns a list of async def middleware functions\"\"\"\r\n\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1876353656, "label": "Consider a request/response wrapping hook slightly higher level than asgi_wrapper()"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2168#issuecomment-1701828197", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2168", "id": 1701828197, "node_id": "IC_kwDOBm6k_c5lb9Zl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-31T21:48:00Z", "updated_at": "2023-08-31T21:48:57Z", "author_association": "OWNER", "body": "A pattern like this could be interesting:\r\n```python\r\nasync def my_middleware(datasette, request, get_response):\r\n # Mess with request here if neccessary\r\n response = await get_response(request)\r\n # mess with response\r\n return response\r\n```\r\nThe Django pattern is more complicated but does have that mechanism for running one-time configuration prior to defining the `middleware()` function, which is neat.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1876353656, "label": "Consider a request/response wrapping hook slightly higher level than asgi_wrapper()"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2168#issuecomment-1701826521", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2168", "id": 1701826521, "node_id": "IC_kwDOBm6k_c5lb8_Z", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-31T21:46:13Z", "updated_at": "2023-08-31T21:46:13Z", "author_association": "OWNER", "body": "This could even be a pair of hooks - `process_request()` and `process_response()`.\r\n\r\nOr could take a leaf from Django, which redesigned middleware to use this pattern instead:\r\n\r\n```python\r\ndef simple_middleware(get_response):\r\n # One-time configuration and initialization.\r\n def middleware(request):\r\n # Code to be executed for each request before\r\n # the view (and later middleware) are called.\r\n response = get_response(request)\r\n # Code to be executed for each request/response after\r\n # the view is called.\r\n return response\r\n return middleware\r\n```\r\nOr even borrow an idea from `pytest` where fixtures can `yield` in the middle, like this:\r\n```python\r\n@pytest.fixture\r\ndef sending_user(mail_admin):\r\n user = mail_admin.create_user()\r\n yield user\r\n mail_admin.delete_user(user)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1876353656, "label": "Consider a request/response wrapping hook slightly higher level than asgi_wrapper()"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2168#issuecomment-1701823609", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2168", "id": 1701823609, "node_id": "IC_kwDOBm6k_c5lb8R5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-31T21:43:06Z", "updated_at": "2023-08-31T21:44:13Z", "author_association": "OWNER", "body": "Not sure what to call this. Maybe `app_wrapper()`?\r\n\r\nOr perhaps it's simpler than that, something like this:\r\n\r\n```python\r\n@hookspec\r\ndef process_response(datasette, request, response):\r\n \"\"\"Last chance to modify the response before it is returned to the client\"\"\"\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1876353656, "label": "Consider a request/response wrapping hook slightly higher level than asgi_wrapper()"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1092#issuecomment-1699926384", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1092", "id": 1699926384, "node_id": "IC_kwDOBm6k_c5lUtFw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T22:17:42Z", "updated_at": "2023-08-30T22:17:42Z", "author_association": "OWNER", "body": "This is implemented now:\r\n- https://docs.datasette.io/en/stable/internals.html#await-ensure-permissions-actor-permissions", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 742041667, "label": "Make cascading permission checks available to plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1190#issuecomment-1699925224", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1190", "id": 1699925224, "node_id": "IC_kwDOBm6k_c5lUszo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T22:16:38Z", "updated_at": "2023-08-30T22:16:38Z", "author_association": "OWNER", "body": "This is going to happen in this tool instead:\r\n- https://github.com/simonw/dclient", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 787098146, "label": "`datasette publish upload` mechanism for uploading databases to an existing Datasette instance"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2165#issuecomment-1699910555", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2165", "id": 1699910555, "node_id": "IC_kwDOBm6k_c5lUpOb", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T22:05:14Z", "updated_at": "2023-08-30T22:05:14Z", "author_association": "OWNER", "body": "Documentation preview: https://github.com/simonw/datasette/blob/6321c9c055a640ed6ea98e231dc5813dcde1f773/docs/plugins.rst#controlling-which-plugins-are-loaded", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1874327336, "label": "DATASETTE_LOAD_PLUGINS environment variable for loading specific plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2165#issuecomment-1699884314", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2165", "id": 1699884314, "node_id": "IC_kwDOBm6k_c5lUi0a", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T21:38:29Z", "updated_at": "2023-08-30T21:39:51Z", "author_association": "OWNER", "body": "Here's the reason for that name disparity:\r\n\r\nhttps://github.com/simonw/datasette/blob/30b28c8367a9c6870386ea10a202705b40862457/datasette/plugins.py#L54-L65\r\n\r\nNote how the `distinfo.project_name` name is used when available. That seems to work for regularly installed plugins but not for plugins loaded via `DATASETTE_LOAD_PLUGINS`.\r\n\r\nAnd that's looking things up in `plugin_to_distinfo` which is populated here:\r\n\r\nhttps://github.com/simonw/datasette/blob/30b28c8367a9c6870386ea10a202705b40862457/datasette/plugins.py#L37", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1874327336, "label": "DATASETTE_LOAD_PLUGINS environment variable for loading specific plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2165#issuecomment-1699811810", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2165", "id": 1699811810, "node_id": "IC_kwDOBm6k_c5lURHi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T20:42:41Z", "updated_at": "2023-08-30T20:42:41Z", "author_association": "OWNER", "body": "The `load_setuptools_entrypoints()` function in Pluggy [does this](https://github.com/pytest-dev/pluggy/blob/0b41c9766508a46ae666cf281684df3164b3e2a9/src/pluggy/_manager.py#L376):\r\n\r\n```python\r\n for ep in dist.entry_points:\r\n if (\r\n ep.group != group\r\n or (name is not None and ep.name != name)\r\n # already registered\r\n or self.get_plugin(ep.name)\r\n or self.is_blocked(ep.name)\r\n ):\r\n continue\r\n plugin = ep.load()\r\n self.register(plugin, name=ep.name)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1874327336, "label": "DATASETTE_LOAD_PLUGINS environment variable for loading specific plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2165#issuecomment-1699809688", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2165", "id": 1699809688, "node_id": "IC_kwDOBm6k_c5lUQmY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T20:41:06Z", "updated_at": "2023-08-30T20:41:06Z", "author_association": "OWNER", "body": "Slight weirdness: I noticed that the output from the `datasette plugins` command looks like this for plugins loaded with the new environment variable:\r\n```json\r\n {\r\n \"name\": \"datasette_pretty_json\",\r\n \"static\": false,\r\n \"templates\": false,\r\n \"version\": null,\r\n \"hooks\": [\r\n \"render_cell\"\r\n ]\r\n },\r\n```\r\nThat should ideally be `datasette-pretty-json`, not `datasette_pretty_json`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1874327336, "label": "DATASETTE_LOAD_PLUGINS environment variable for loading specific plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2165#issuecomment-1699802028", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2165", "id": 1699802028, "node_id": "IC_kwDOBm6k_c5lUOus", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T20:35:02Z", "updated_at": "2023-08-30T20:35:02Z", "author_association": "OWNER", "body": "Testing this is going to be a bit of a pain.\r\n\r\nI think I'll add a whole separate test block to CI which installs a couple of plugins and then exercises this feature using `datasette plugins`.\r\n\r\nI'll use `datasette-init` and `datasette-json-html` just because they are small and simple.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1874327336, "label": "DATASETTE_LOAD_PLUGINS environment variable for loading specific plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2164#issuecomment-1699728102", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2164", "id": 1699728102, "node_id": "IC_kwDOBm6k_c5lT8rm", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-30T19:35:32Z", "updated_at": "2023-08-30T19:35:47Z", "author_association": "OWNER", "body": "Figured this out so far with the help of GPT-4: https://chat.openai.com/share/0e785865-621b-4fb3-ba05-7449e57c8496\r\n\r\nNow this works:\r\n```bash\r\nDATASETTE_LOAD_PLUGINS=datasette-write-ui datasette plugins\r\n```\r\n```json\r\n[\r\n {\r\n \"name\": \"datasette_write_ui\",\r\n \"static\": true,\r\n \"templates\": true,\r\n \"version\": null,\r\n \"hooks\": [\r\n \"extra_template_vars\",\r\n \"register_routes\"\r\n ]\r\n }\r\n]\r\n```\r\nOr multiple plugins:\r\n```bash\r\nDATASETTE_LOAD_PLUGINS=datasette-write-ui,datasette-pretty-json datasette plugins\r\n```\r\nOutputs:\r\n```json\r\n[\r\n {\r\n \"name\": \"datasette_pretty_json\",\r\n \"static\": false,\r\n \"templates\": false,\r\n \"version\": null,\r\n \"hooks\": [\r\n \"render_cell\"\r\n ]\r\n },\r\n {\r\n \"name\": \"datasette_write_ui\",\r\n \"static\": true,\r\n \"templates\": true,\r\n \"version\": null,\r\n \"hooks\": [\r\n \"extra_template_vars\",\r\n \"register_routes\"\r\n ]\r\n }\r\n]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1874255116, "label": "Ability to only load a specific list of plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2163#issuecomment-1697818917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2163", "id": 1697818917, "node_id": "IC_kwDOBm6k_c5lMqkl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T16:56:20Z", "updated_at": "2023-08-29T16:56:20Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/blob/50da908213a0fc405ecd7a40090dfea7a2e7395c/datasette/utils/internal_db.py#L8-L62", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1872043170, "label": "Rename core_X to catalog_X in the internals"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2154#issuecomment-1697725150", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2154", "id": 1697725150, "node_id": "IC_kwDOBm6k_c5lMTre", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T15:54:12Z", "updated_at": "2023-08-29T15:54:12Z", "author_association": "OWNER", "body": "In that last commit I also upgraded `Permission` from a named tuple to a dataclass, and added a `implies_can_view=True` private (not documented) option to it.\r\n\r\nLast step is to refactor the code to use that new property.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865281760, "label": "Cascade for restricted token view-table/view-database/view-instance operations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2154#issuecomment-1697612168", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2154", "id": 1697612168, "node_id": "IC_kwDOBm6k_c5lL4GI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T14:57:54Z", "updated_at": "2023-08-29T14:57:54Z", "author_association": "OWNER", "body": "The code to refactor is this: https://github.com/simonw/datasette/blob/d64a9896f743f87b673d58859a0ec16685594e79/datasette/default_permissions.py#L181-L280\r\n\r\nI'm going to turn that into a more general `restrictions_allow_action` function.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865281760, "label": "Cascade for restricted token view-table/view-database/view-instance operations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2162#issuecomment-1696710911", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2162", "id": 1696710911, "node_id": "IC_kwDOBm6k_c5lIcD_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T03:23:44Z", "updated_at": "2023-08-29T03:23:44Z", "author_association": "OWNER", "body": "I'm going to merge this so we can see how it feels.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1870672704, "label": "Add new `--internal internal.db` option, deprecate legacy `_internal` database"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2162#issuecomment-1696709110", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2162", "id": 1696709110, "node_id": "IC_kwDOBm6k_c5lIbn2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T03:20:40Z", "updated_at": "2023-08-29T03:22:47Z", "author_association": "OWNER", "body": "> However, one important notes about those new `core_` tables: If a `--internal` DB is passed in, that means those `core_` tables will persist across multiple Datasette instances. This wasn't the case before, since `_internal` was always an in-memory database created from scratch.\r\n\r\nI'm completely happy for the `core_*` tables (or `datasette_*` or some other name) to live in the persisted-to-disk `internal.db` database, even though they're effectively meant to be an in-memory cache.\r\n\r\nI don't think it causes any harm, and it could even be quite useful to have them visible on disk - other applications could read the `internal.db` database while Datasette itself is running, should they have some weird reason to want to do that!\r\n\r\nHaving those tables stick around in `internal.db` after Datasette shuts down could be useful for other debugging activities as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1870672704, "label": "Add new `--internal internal.db` option, deprecate legacy `_internal` database"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2162#issuecomment-1696707458", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2162", "id": 1696707458, "node_id": "IC_kwDOBm6k_c5lIbOC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T03:17:54Z", "updated_at": "2023-08-29T03:17:54Z", "author_association": "OWNER", "body": "Documentation preview: https://datasette--2162.org.readthedocs.build/en/2162/internals.html#datasette-s-internal-database", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1870672704, "label": "Add new `--internal internal.db` option, deprecate legacy `_internal` database"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2154#issuecomment-1696644066", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2154", "id": 1696644066, "node_id": "IC_kwDOBm6k_c5lILvi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T01:37:47Z", "updated_at": "2023-08-29T03:00:25Z", "author_association": "OWNER", "body": "Code for this might be cleaner with a `Restrictions()` class that takes a `\"_r\"` dictionary to the constructor and can then answer questions like `.any_resource_has_permission(\"view-table\")` - where it can resolve aliases etc as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865281760, "label": "Cascade for restricted token view-table/view-database/view-instance operations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2154#issuecomment-1696642671", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2154", "id": 1696642671, "node_id": "IC_kwDOBm6k_c5lILZv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T01:35:37Z", "updated_at": "2023-08-29T01:35:37Z", "author_association": "OWNER", "body": "Reminder that I also need to confirm that `insert-row` works if you have it at the instance level, the database level or the resource level in `_r`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865281760, "label": "Cascade for restricted token view-table/view-database/view-instance operations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2154#issuecomment-1696618784", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2154", "id": 1696618784, "node_id": "IC_kwDOBm6k_c5lIFkg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T00:58:34Z", "updated_at": "2023-08-29T00:58:34Z", "author_association": "OWNER", "body": "Should this have `implies_can_view=True` too? Probably: https://github.com/simonw/datasette/blob/d64a9896f743f87b673d58859a0ec16685594e79/datasette/default_permissions.py#L20-L22", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865281760, "label": "Cascade for restricted token view-table/view-database/view-instance operations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2160#issuecomment-1696595326", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2160", "id": 1696595326, "node_id": "IC_kwDOBm6k_c5lH_1-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T00:20:08Z", "updated_at": "2023-08-29T00:20:08Z", "author_association": "OWNER", "body": "Cog failed!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1869807874, "label": "Bump sphinx, furo, blacken-docs dependencies"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2160#issuecomment-1696592763", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2160", "id": 1696592763, "node_id": "IC_kwDOBm6k_c5lH_N7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T00:16:38Z", "updated_at": "2023-08-29T00:16:38Z", "author_association": "OWNER", "body": "Since this bumps Sphinx I'm manually reviewing the ReadTheDocs preview a bit, looks good to me: https://datasette--2160.org.readthedocs.build/en/2160/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1869807874, "label": "Bump sphinx, furo, blacken-docs dependencies"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2148#issuecomment-1696591943", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2148", "id": 1696591943, "node_id": "IC_kwDOBm6k_c5lH_BH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T00:15:27Z", "updated_at": "2023-08-29T00:15:27Z", "author_association": "OWNER", "body": "Now solving this here:\r\n- #2160 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1859415334, "label": "Bump sphinx, furo, blacken-docs dependencies"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2160#issuecomment-1696586767", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2160", "id": 1696586767, "node_id": "IC_kwDOBm6k_c5lH9wP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T00:08:26Z", "updated_at": "2023-08-29T00:08:35Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/issues/2148#issuecomment-1689177556\r\n\r\n> Simplest possible solution is to only run the `pip install .[docs]` bit under Python 3.9+, ditto for the docs tests. I think I'll try that.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1869807874, "label": "Bump sphinx, furo, blacken-docs dependencies"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2160#issuecomment-1696586213", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2160", "id": 1696586213, "node_id": "IC_kwDOBm6k_c5lH9nl", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-29T00:07:46Z", "updated_at": "2023-08-29T00:07:46Z", "author_association": "OWNER", "body": "I figured out why this was failing in:\r\n- https://github.com/simonw/datasette/pull/2148\r\n\r\nIt's because Sphinx dropped support for Python 3.8.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1869807874, "label": "Bump sphinx, furo, blacken-docs dependencies"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2102#issuecomment-1696378239", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2102", "id": 1696378239, "node_id": "IC_kwDOBm6k_c5lHK1_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-28T20:38:01Z", "updated_at": "2023-08-28T20:38:01Z", "author_association": "OWNER", "body": "I want to test \"for this set of restrictions, does a GET/POST to this path return 200 or 403\"?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1805076818, "label": "API tokens with view-table but not view-database/view-instance cannot access the table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2102#issuecomment-1696361304", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2102", "id": 1696361304, "node_id": "IC_kwDOBm6k_c5lHGtY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-28T20:23:47Z", "updated_at": "2023-08-28T20:24:35Z", "author_association": "OWNER", "body": "Here's an existing relevant test:\r\n\r\nhttps://github.com/simonw/datasette/blob/2e2825869fc2655b5fcadc743f6f9dec7a49bc65/tests/test_permissions.py#L616-L666\r\n\r\nIt's not quite right for this new set of tests though, since they need to be exercising actual endpoints (`/.json` etc) in order to check that this works correctly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1805076818, "label": "API tokens with view-table but not view-database/view-instance cannot access the table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2153#issuecomment-1696355634", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2153", "id": 1696355634, "node_id": "IC_kwDOBm6k_c5lHFUy", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-28T20:19:15Z", "updated_at": "2023-08-28T20:19:15Z", "author_association": "OWNER", "body": "Documentation:\r\n- https://docs.datasette.io/en/latest/cli-reference.html#datasette-get\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865232341, "label": "Datasette --get --actor option"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2156#issuecomment-1696260446", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2156", "id": 1696260446, "node_id": "IC_kwDOBm6k_c5lGuFe", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-28T19:33:05Z", "updated_at": "2023-08-28T19:33:05Z", "author_association": "OWNER", "body": "Now a PR:\r\n- #2161 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865649347, "label": "datasette -s/--setting option for setting nested configuration options"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/588#issuecomment-1694823972", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/588", "id": 1694823972, "node_id": "IC_kwDOCGYnMM5lBPYk", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-28T00:41:54Z", "updated_at": "2023-08-28T00:41:54Z", "author_association": "OWNER", "body": "Tips on typing `**kwargs`: https://adamj.eu/tech/2021/05/11/python-type-hints-args-and-kwargs/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1868713944, "label": "`table.get(column=value)` option for retrieving things not by their primary key"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/950#issuecomment-1692494455", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/950", "id": 1692494455, "node_id": "IC_kwDOBm6k_c5k4Wp3", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T22:26:08Z", "updated_at": "2023-08-24T22:26:08Z", "author_association": "OWNER", "body": "Closing this issue in favour of this one:\r\n- #2157 ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 685806511, "label": "Private/secret databases: database files that are only visible to plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2157#issuecomment-1692465763", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2157", "id": 1692465763, "node_id": "IC_kwDOBm6k_c5k4Ppj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T21:54:29Z", "updated_at": "2023-08-24T21:54:29Z", "author_association": "OWNER", "body": "But yes, I'm a big +1 on this whole plan.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865869205, "label": "Proposal: Make the `_internal` database persistent, customizable, and hidden"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2157#issuecomment-1692465334", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2157", "id": 1692465334, "node_id": "IC_kwDOBm6k_c5k4Pi2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T21:54:09Z", "updated_at": "2023-08-24T21:54:09Z", "author_association": "OWNER", "body": "We discussed this in-person this morning and these notes reflect what we talked about perfectly.\r\n\r\nI've had so many bugs with plugins that I've written myself that have forgotten to special-case the `_internal` database when looping through `datasette.databases.keys()` - removing it from there entirely would help a lot.\r\n\r\nJust one tiny disagreement: for `datasette-comments` I think having it store things in `_internal` could be an option, but in most cases I expect users to chose NOT to do that - because being able to join against those tables for more advanced queries is going to be super useful.\r\n\r\nShow me all rows in `foia_requests` with at least one associated comment in `datasette_comments.comments` kind of tihng.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865869205, "label": "Proposal: Make the `_internal` database persistent, customizable, and hidden"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2143#issuecomment-1692210044", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2143", "id": 1692210044, "node_id": "IC_kwDOBm6k_c5k3RN8", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T18:28:27Z", "updated_at": "2023-08-24T18:28:27Z", "author_association": "OWNER", "body": "Just spotted this: https://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/datasette/app.py#L328-L332\r\n\r\nhttps://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/datasette/app.py#L359-L360\r\n\r\nLooks to me like that second bit of code doesn't yet handle `datasette.yml`\r\n\r\nThis code does though:\r\n\r\nhttps://github.com/simonw/datasette/blob/17ec309e14f9c2e90035ba33f2f38ecc5afba2fa/datasette/app.py#L333-L335\r\n\r\n`parse_metadata()` is clearly a bad name for this function:\r\n\r\nhttps://github.com/simonw/datasette/blob/d97e82df3c8a3f2e97038d7080167be9bb74a68d/datasette/utils/__init__.py#L980-L990\r\n\r\nThat ` @documented` decorator indicates that it's part of the documented API used by plugin authors: https://docs.datasette.io/en/1.0a4/internals.html#parse-metadata-content\r\n\r\nSo we should rename it to something better like `parse_json_or_yaml()` but keep `parse_metadata` as an undocumented alias for that to avoid any unnecessary plugin breaks.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855885427, "label": "De-tangling Metadata before Datasette 1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2156#issuecomment-1692206200", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2156", "id": 1692206200, "node_id": "IC_kwDOBm6k_c5k3QR4", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T18:25:23Z", "updated_at": "2023-08-24T18:25:23Z", "author_association": "OWNER", "body": "Ran out of time for this, I'll look at the next step next week.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865649347, "label": "datasette -s/--setting option for setting nested configuration options"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2156#issuecomment-1692201647", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2156", "id": 1692201647, "node_id": "IC_kwDOBm6k_c5k3PKv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T18:21:53Z", "updated_at": "2023-08-24T18:21:53Z", "author_association": "OWNER", "body": "Oops, that was meant to be a PR. It's just a utility function though so it's safe to land already. I'll do a PR for the actual integration of it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865649347, "label": "datasette -s/--setting option for setting nested configuration options"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2156#issuecomment-1692186522", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2156", "id": 1692186522, "node_id": "IC_kwDOBm6k_c5k3Lea", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T18:10:04Z", "updated_at": "2023-08-24T18:10:04Z", "author_association": "OWNER", "body": "I have an implementation in https://github.com/simonw/datasette/issues/2143#issuecomment-1690792514 too - I'm going to land that as a PR.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865649347, "label": "datasette -s/--setting option for setting nested configuration options"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2143#issuecomment-1692182910", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2143", "id": 1692182910, "node_id": "IC_kwDOBm6k_c5k3Kl-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T18:06:57Z", "updated_at": "2023-08-24T18:08:17Z", "author_association": "OWNER", "body": "The other thing that could work is something like this:\r\n```bash\r\nexport AUTH_TOKENS_DB=\"tokens\"\r\ndatasette \\\r\n -s settings.sql_time_limit_ms 1000 \\\r\n -s plugins.datasette-auth-tokens.manage_tokens true \\\r\n -e plugins.datasette-auth-tokens.manage_tokens_database AUTH_TOKENS_DB\r\n```\r\nSo `-e` is an alternative version of `-s` which reads from the named environment variable instead of having the value provided directly as the second value in the pair.\r\n\r\nI quite like this, because it could replace the really ugly `$ENV` pattern we have in plugin configuration at the moment: https://docs.datasette.io/en/1.0a4/plugins.html#secret-configuration-values\r\n```yaml\r\nplugins:\r\n datasette-auth-github:\r\n client_secret:\r\n $env: GITHUB_CLIENT_SECRET\r\n```", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855885427, "label": "De-tangling Metadata before Datasette 1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2143#issuecomment-1692180683", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2143", "id": 1692180683, "node_id": "IC_kwDOBm6k_c5k3KDL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T18:05:17Z", "updated_at": "2023-08-24T18:05:17Z", "author_association": "OWNER", "body": "That's a really good call, thanks @rclement - environment variable configuration totally makes sense here.\r\n\r\nNeed to figure out the right syntax for that. Something like this perhaps:\r\n\r\n```bash\r\nDATASETTE_CONFIG_PLUGINS='{\"datasette-ripgrep\": ...}'\r\n```\r\nHard to know how to make this nestable though. I considered this:\r\n```bash\r\nDATASETTE_CONFIG_PLUGINS_DATASETTE_RIPGREP_PATH='/path/to/code/'\r\n```\r\nBut that doesn't work, because how does the processing code know that it should split on `_` for most of the tokens but NOT split `DATASETTE_RIPGREP`, instead treating that as `datasette-ripgrep`?\r\n\r\nI checked and `-` is not a valid character in an environment variable, at least in zsh on macOS:\r\n```\r\n% export FOO_BAR-BAZ=1\r\nexport: not valid in this context: FOO_BAR-BAZ\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1855885427, "label": "De-tangling Metadata before Datasette 1.0"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2154#issuecomment-1691845306", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2154", "id": 1691845306, "node_id": "IC_kwDOBm6k_c5k14K6", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T14:57:39Z", "updated_at": "2023-08-24T14:57:39Z", "author_association": "OWNER", "body": "Notes on manual testing so far - it looks like this might be working!\r\n- https://github.com/simonw/datasette/issues/2102#issuecomment-1691824713", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865281760, "label": "Cascade for restricted token view-table/view-database/view-instance operations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2102#issuecomment-1691842259", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2102", "id": 1691842259, "node_id": "IC_kwDOBm6k_c5k13bT", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T14:55:54Z", "updated_at": "2023-08-24T14:55:54Z", "author_association": "OWNER", "body": "So what's needed to finish this is:\r\n- Tests that demonstrate that nothing is revealed that shouldn't be by tokens restricted in this way\r\n- Similar tests for other permissions like `create-table` that check that they work (and don't also need `view-instance` etc).\r\n- Documentation", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1805076818, "label": "API tokens with view-table but not view-database/view-instance cannot access the table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2102#issuecomment-1691824713", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2102", "id": 1691824713, "node_id": "IC_kwDOBm6k_c5k1zJJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T14:45:49Z", "updated_at": "2023-08-24T14:45:49Z", "author_association": "OWNER", "body": "I tested this out against a Datasette Cloud instance. I created a restricted token and tested it like this:\r\n```bash\r\ncurl -H \"Authorization: Bearer $TOKEN\" \\\r\n 'https://$INSTANCE/-/actor.json' | jq\r\n```\r\n```json\r\n{\r\n \"actor\": {\r\n \"id\": \"245\",\r\n \"token\": \"dsatok\",\r\n \"token_id\": 2,\r\n \"_r\": {\r\n \"r\": {\r\n \"data\": {\r\n \"all_stocks\": [\r\n \"vt\"\r\n ]\r\n }\r\n }\r\n }\r\n }\r\n}\r\n```\r\nIt can access the `all_stocks` demo table:\r\n```bash\r\ncurl -H \"Authorization: Bearer $TOKEN\" \\\r\n 'https://$INSTANCE/data/all_stocks.json?_size=1' | jq\r\n```\r\n```json\r\n{\r\n \"ok\": true,\r\n \"next\": \"1\",\r\n \"rows\": [\r\n {\r\n \"rowid\": 1,\r\n \"Date\": \"2013-01-02\",\r\n \"Open\": 79.12,\r\n \"High\": 79.29,\r\n \"Low\": 77.38,\r\n \"Close\": 78.43,\r\n \"Volume\": 140124866,\r\n \"Name\": \"AAPL\"\r\n }\r\n ],\r\n \"truncated\": false\r\n}\r\n```\r\nAccessing the database returns just information about that table, even though other tables exist:\r\n```bash\r\ncurl -H \"Authorization: Bearer $TOKEN\" \\\r\n 'https://$INSTANCE/data.json?_size=1'\r\n```\r\n```json\r\n{\r\n \"database\": \"data\",\r\n \"private\": true,\r\n \"path\": \"/data\",\r\n \"size\": 3796992,\r\n \"tables\": [\r\n {\r\n \"name\": \"all_stocks\",\r\n \"columns\": [\r\n \"Date\",\r\n \"Open\",\r\n \"High\",\r\n \"Low\",\r\n \"Close\",\r\n \"Volume\",\r\n \"Name\"\r\n ],\r\n \"primary_keys\": [],\r\n \"count\": 8813,\r\n \"hidden\": false,\r\n \"fts_table\": null,\r\n \"foreign_keys\": {\r\n \"incoming\": [],\r\n \"outgoing\": []\r\n },\r\n \"private\": true\r\n }\r\n ],\r\n \"hidden_count\": 0,\r\n \"views\": [],\r\n \"queries\": [],\r\n \"allow_execute_sql\": false,\r\n \"table_columns\": {}\r\n}\r\n```\r\nAnd hitting the top-level `/.json` thing does the same - it reveals that table but not any of the other tables or databases:\r\n```bash\r\ncurl -H \"Authorization: Bearer $TOKEN\" \\\r\n 'https://$INSTANCE/.json?_size=1'\r\n```\r\n```json\r\n{\r\n \"data\": {\r\n \"name\": \"data\",\r\n \"hash\": null,\r\n \"color\": \"8d777f\",\r\n \"path\": \"/data\",\r\n \"tables_and_views_truncated\": [\r\n {\r\n \"name\": \"all_stocks\",\r\n \"columns\": [\r\n \"Date\",\r\n \"Open\",\r\n \"High\",\r\n \"Low\",\r\n \"Close\",\r\n \"Volume\",\r\n \"Name\"\r\n ],\r\n \"primary_keys\": [],\r\n \"count\": null,\r\n \"hidden\": false,\r\n \"fts_table\": null,\r\n \"num_relationships_for_sorting\": 0,\r\n \"private\": false\r\n }\r\n ],\r\n \"tables_and_views_more\": false,\r\n \"tables_count\": 1,\r\n \"table_rows_sum\": 0,\r\n \"show_table_row_counts\": false,\r\n \"hidden_table_rows_sum\": 0,\r\n \"hidden_tables_count\": 0,\r\n \"views_count\": 0,\r\n \"private\": false\r\n }\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1805076818, "label": "API tokens with view-table but not view-database/view-instance cannot access the table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2154#issuecomment-1691788400", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2154", "id": 1691788400, "node_id": "IC_kwDOBm6k_c5k1qRw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T14:25:56Z", "updated_at": "2023-08-24T14:25:56Z", "author_association": "OWNER", "body": "Can be tested with:\r\n```bash\r\npip install https://github.com/simonw/datasette/archive/6d57a8c23043e99b27f7a2afbe58f4d58815fd51.zip\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865281760, "label": "Cascade for restricted token view-table/view-database/view-instance operations"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2153#issuecomment-1691779180", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2153", "id": 1691779180, "node_id": "IC_kwDOBm6k_c5k1oBs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-08-24T14:21:03Z", "updated_at": "2023-08-24T14:21:03Z", "author_association": "OWNER", "body": "`datasette serve` currently only has a `--get` - for this to be really useful it needs to grow `--post` and maybe other verbs too.\r\n\r\nWhich is a good argument for moving this functionality to `datasette client get ...` instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1865232341, "label": "Datasette --get --actor option"}, "performed_via_github_app": null}