{"html_url": "https://github.com/simonw/sqlite-utils/pull/604#issuecomment-1846560096", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/604", "id": 1846560096, "node_id": "IC_kwDOCGYnMM5uEEVg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:16:44Z", "updated_at": "2023-12-08T05:17:20Z", "author_association": "OWNER", "body": "Also tested this manually like so:\r\n\r\n```bash\r\nsqlite-utils create-table strict.db strictint id integer size integer --strict\r\nsqlite-utils create-table strict.db notstrictint id integer size integer \r\nsqlite-utils install sqlite-utils-shell\r\nsqlite-utils shell strict.db \r\n```\r\n```\r\nAttached to strict.db\r\nType 'exit' to exit.\r\nsqlite-utils> insert into strictint (size) values (4);\r\n1 row affected\r\nsqlite-utils> insert into strictint (size) values ('four');\r\nAn error occurred: cannot store TEXT value in INTEGER column strictint.size\r\nsqlite-utils> insert into notstrictint (size) values ('four');\r\n1 row affected\r\nsqlite-utils> commit;\r\nDone\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2001006157, "label": "Add more STRICT table support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/603#issuecomment-1846555822", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/603", "id": 1846555822, "node_id": "IC_kwDOCGYnMM5uEDSu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:09:55Z", "updated_at": "2023-12-08T05:10:31Z", "author_association": "OWNER", "body": "I'm unable to replicate this issue. This is with a fresh install of `sqlite-utils==3.35.2`:\r\n```\r\n(base) ~ python3.12\r\nPython 3.12.0 (v3.12.0:0fb18b02c8, Oct 2 2023, 09:45:56) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin\r\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\r\n>>> import sqlite_utils\r\n>>> db = sqlite_utils.Database(memory=True)\r\n>>> db[\"foo\"].insert({\"bar\": 1})\r\n\r\n>>> import sys\r\n>>> sys.version\r\n'3.12.0 (v3.12.0:0fb18b02c8, Oct 2 2023, 09:45:56) [Clang 13.0.0 (clang-1300.0.29.30)]'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1988525411, "label": "Pyhton 3.12 Bug report"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/605#issuecomment-1846554637", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/605", "id": 1846554637, "node_id": "IC_kwDOCGYnMM5uEDAN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-08T05:07:54Z", "updated_at": "2023-12-08T05:07:54Z", "author_association": "OWNER", "body": "Thanks for opening an issue - this should help future Google searchers figure out what's going on here.\r\n\r\nAnother approach here could be to store large integers as `TEXT` in SQLite (or even as `BLOB`).\r\n\r\nBoth storing as `REAL` and storing as `TEXT/BLOB` feel nasty to me, but it looks like SQLite has a hard upper limit of 9223372036854775807 for integers.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2007893839, "label": "Insert fails with `Error: Python int too large to convert to SQLite INTEGER`; can we use `NUMERIC` here?"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/604#issuecomment-1843586503", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/604", "id": 1843586503, "node_id": "IC_kwDOCGYnMM5t4uXH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T19:49:10Z", "updated_at": "2023-12-06T19:49:29Z", "author_association": "OWNER", "body": "This looks really great on first glance - design is good, implementation is solid, tests and documentation look great.\r\n\r\nLooks like a couple of `mypy` failures in the tests at the moment:\r\n\r\n```\r\n mypy sqlite_utils tests\r\n\r\nsqlite_utils/db.py:543: error: Incompatible types in assignment (expression has type \"type[Table]\", variable has type \"type[View]\") [assignment]\r\ntests/test_lookup.py:156: error: Name \"test_lookup_new_table\" already defined on line 5 [no-redef]\r\nFound 2 errors in 2 files (checked 54 source files)\r\nError: Process completed with exit code 1.\r\n```\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2001006157, "label": "Add more STRICT table support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/606#issuecomment-1843579184", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606", "id": 1843579184, "node_id": "IC_kwDOCGYnMM5t4skw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T19:43:55Z", "updated_at": "2023-12-06T19:43:55Z", "author_association": "OWNER", "body": "Updated documentation:\r\n- https://sqlite-utils.datasette.io/en/latest/cli.html#cli-add-column\r\n- https://sqlite-utils.datasette.io/en/latest/cli-reference.html#add-column", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2029161033, "label": "str and int as aliases for text and integer"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/606#issuecomment-1843465748", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606", "id": 1843465748, "node_id": "IC_kwDOCGYnMM5t4Q4U", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-12-06T18:36:51Z", "updated_at": "2023-12-06T18:36:51Z", "author_association": "OWNER", "body": "I'll add `bytes` too - `float` already works. This makes sense because when you are working with the Python API you use `str` and `float` and `bytes` and `int` to specify column types.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 2029161033, "label": "str and int as aliases for text and integer"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2205#issuecomment-1794054390", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2205", "id": 1794054390, "node_id": "IC_kwDOBm6k_c5q7xj2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-06T04:09:43Z", "updated_at": "2023-11-06T04:10:34Z", "author_association": "OWNER", "body": "That `keep_blank_values=True` is from https://github.com/simonw/datasette/commit/0934844c0b6d124163d0185fb6a41ba5a71433da\r\n\r\nCommit message:\r\n> request.post_vars() no longer discards empty values \r\n\r\nRelevant test:\r\n\r\nhttps://github.com/simonw/datasette/blob/452a587e236ef642cbc6ae345b58767ea8420cb5/tests/test_internals_request.py#L19-L27", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1978023780, "label": "request.post_vars() method obliterates form keys with multiple values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2205#issuecomment-1794052079", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2205", "id": 1794052079, "node_id": "IC_kwDOBm6k_c5q7w_v", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-06T04:06:05Z", "updated_at": "2023-11-06T04:08:50Z", "author_association": "OWNER", "body": "It should return a `MultiParams`:\r\n\r\nhttps://github.com/simonw/datasette/blob/452a587e236ef642cbc6ae345b58767ea8420cb5/datasette/utils/__init__.py#L900-L917\r\n\r\nChange needs to be made before 1.0.\r\n\r\n```python\r\nreturn MultiParams(urllib.parse.parse_qs(body.decode(\"utf-8\")))\r\n```\r\nNeed to remember why I was using `keep_blank_values=True` there and check that using `MultiParams` doesn't conflict with that reason.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1978023780, "label": "request.post_vars() method obliterates form keys with multiple values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2205#issuecomment-1793880738", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2205", "id": 1793880738, "node_id": "IC_kwDOBm6k_c5q7HKi", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-05T23:26:14Z", "updated_at": "2023-11-05T23:26:14Z", "author_association": "OWNER", "body": "I found this problem while trying to use WTForms with this pattern:\r\n\r\n```python\r\nchoices = [(col, col) for col in await db.table_columns(table)]\r\n\r\nclass ConfigForm(Form):\r\n template = TextAreaField(\"Template\")\r\n api_token = PasswordField(\"OpenAI API token\")\r\n columns = MultiCheckboxField('Columns', choices=choices)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1978023780, "label": "request.post_vars() method obliterates form keys with multiple values"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/591#issuecomment-1793278279", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/591", "id": 1793278279, "node_id": "IC_kwDOCGYnMM5q40FH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:58:03Z", "updated_at": "2023-11-04T00:58:03Z", "author_association": "OWNER", "body": "I'm going to abandon this PR and ship the 3.12 testing change directly to `main`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1884335789, "label": "Test against Python 3.12 preview"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/596#issuecomment-1793274869", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/596", "id": 1793274869, "node_id": "IC_kwDOCGYnMM5q4zP1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:47:55Z", "updated_at": "2023-11-04T00:47:55Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1919296686, "label": "Fixes mapping for time fields related to mysql, closes #522"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/598#issuecomment-1793274485", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/598", "id": 1793274485, "node_id": "IC_kwDOCGYnMM5q4zJ1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:46:55Z", "updated_at": "2023-11-04T00:46:55Z", "author_association": "OWNER", "body": "Manually tested. Before:\r\n\r\n![cursor-bug](https://github.com/simonw/sqlite-utils/assets/9599/3bdd30ea-1a54-4fec-b42d-793130a17bc1)\r\n\r\nAfter:\r\n\r\n![cursor-fix](https://github.com/simonw/sqlite-utils/assets/9599/015d4e4e-b40c-4a93-81f5-1a4adef69b11)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1926729132, "label": "Fixed issue #433 - CLI eats cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1793274350", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/433", "id": 1793274350, "node_id": "IC_kwDOCGYnMM5q4zHu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:46:30Z", "updated_at": "2023-11-04T00:46:30Z", "author_association": "OWNER", "body": "And a GIF of the fix after applying:\r\n- #598\r\n\r\n![cursor-fix](https://github.com/simonw/sqlite-utils/assets/9599/35829aec-c9ac-4925-a8e6-ffe7c2ab0d96)\r\n", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239034903, "label": "CLI eats my cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1793273968", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/433", "id": 1793273968, "node_id": "IC_kwDOCGYnMM5q4zBw", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:45:19Z", "updated_at": "2023-11-04T00:45:19Z", "author_association": "OWNER", "body": "Here's an animated GIF that demonstrates the bug:\r\n\r\n![cursor-bug](https://github.com/simonw/sqlite-utils/assets/9599/18e54b38-a466-4384-9162-114eb01526a2)\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1239034903, "label": "CLI eats my cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/598#issuecomment-1793272429", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/598", "id": 1793272429, "node_id": "IC_kwDOCGYnMM5q4ypt", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:40:34Z", "updated_at": "2023-11-04T00:40:34Z", "author_association": "OWNER", "body": "Thanks!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1926729132, "label": "Fixed issue #433 - CLI eats cursor"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793269219", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793269219, "node_id": "IC_kwDOCGYnMM5q4x3j", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:34:33Z", "updated_at": "2023-11-04T00:34:33Z", "author_association": "OWNER", "body": "The GIS tests now pass in that container too:\r\n```bash\r\npytest tests/test_gis.py\r\n```\r\n```\r\n======================== test session starts =========================\r\nplatform linux -- Python 3.10.12, pytest-7.4.3, pluggy-1.3.0\r\nrootdir: /tmp/sqlite-utils\r\nplugins: hypothesis-6.88.1\r\ncollected 12 items \r\n\r\ntests/test_gis.py ............ [100%]\r\n\r\n========================= 12 passed in 0.48s =========================\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/599#issuecomment-1793268750", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/599", "id": 1793268750, "node_id": "IC_kwDOCGYnMM5q4xwO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:33:25Z", "updated_at": "2023-11-04T00:33:25Z", "author_association": "OWNER", "body": "See details of how I tested this here:\r\n- https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793268126\r\n\r\nShort version: having applied this fix, the following command (on simulated `aarch64`):\r\n```bash\r\nsqlite-utils memory \"select spatialite_version()\" --load-extension=spatialite\r\n```\r\nOutputs:\r\n```json\r\n[{\"spatialite_version()\": \"5.0.1\"}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1976986318, "label": "Cannot find spatialite on arm64 linux"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793268126", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793268126, "node_id": "IC_kwDOCGYnMM5q4xme", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:31:34Z", "updated_at": "2023-11-04T00:31:34Z", "author_association": "OWNER", "body": "Testing this manually on macOS using Docker Desk top like this:\r\n```bash\r\ndocker run -it --rm arm64v8/ubuntu /bin/bash\r\n```\r\nThen inside the container:\r\n```bash\r\nuname -m\r\n```\r\nOutputs: `aarch64`\r\n\r\nThen:\r\n```bash\r\napt install spatialite-bin libsqlite3-mod-spatialite git python3 python3-venv -y\r\ncd /tmp\r\ngit clone https://github.com/simonw/sqlite-utils\r\ncd sqlite-utils\r\npython3 -m venv venv\r\nsource venv/bin/activate\r\npip install -e '.[test]'\r\nsqlite-utils memory \"select spatialite_version()\" --load-extension=spatialite\r\n```\r\nWhich output:\r\n```\r\nTraceback (most recent call last):\r\n File \"/tmp/sqlite-utils/venv/bin/sqlite-utils\", line 33, in \r\n sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')())\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1157, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1078, in main\r\n rv = self.invoke(ctx)\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1688, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 1434, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py\", line 783, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/tmp/sqlite-utils/sqlite_utils/cli.py\", line 1959, in memory\r\n _load_extensions(db, load_extension)\r\n File \"/tmp/sqlite-utils/sqlite_utils/cli.py\", line 3232, in _load_extensions\r\n if \":\" in ext:\r\nTypeError: argument of type 'NoneType' is not iterable\r\n```\r\nThen I ran this:\r\n```bash\r\ngit checkout -b MikeCoats-spatialite-paths-linux-arm main\r\ngit pull https://github.com/MikeCoats/sqlite-utils.git spatialite-paths-linux-arm\r\n```\r\nAnd now:\r\n```bash\r\nsqlite-utils memory \"select spatialite_version()\" --load-extension=spatialite\r\n```\r\nOutputs:\r\n```json\r\n[{\"spatialite_version()\": \"5.0.1\"}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793265952", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793265952, "node_id": "IC_kwDOCGYnMM5q4xEg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:25:34Z", "updated_at": "2023-11-04T00:25:34Z", "author_association": "OWNER", "body": "The tests failed because they found a spelling mistake in a completely unrelated area of the code - not sure why that had not been caught before.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/600#issuecomment-1793263638", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/600", "id": 1793263638, "node_id": "IC_kwDOCGYnMM5q4wgW", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-04T00:19:58Z", "updated_at": "2023-11-04T00:19:58Z", "author_association": "OWNER", "body": "Thanks for this!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1977004379, "label": "Add spatialite arm64 linux path"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/949#issuecomment-1791911093", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/949", "id": 1791911093, "node_id": "IC_kwDOBm6k_c5qzmS1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-11-03T05:28:09Z", "updated_at": "2023-11-03T05:28:58Z", "author_association": "OWNER", "body": "Datasette is using that now, see:\r\n- #1893", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 684961449, "label": "Try out CodeMirror SQL hints"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2052#issuecomment-1760552652", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2052", "id": 1760552652, "node_id": "IC_kwDOBm6k_c5o7-bM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T23:59:21Z", "updated_at": "2023-10-12T23:59:21Z", "author_association": "OWNER", "body": "I'm landing this despite the cog failures. I'll fix them on main if I have to.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1651082214, "label": "feat: Javascript Plugin API (Custom panels, column menu items with JS actions)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2052#issuecomment-1760545012", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2052", "id": 1760545012, "node_id": "IC_kwDOBm6k_c5o78j0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T23:48:16Z", "updated_at": "2023-10-12T23:48:16Z", "author_association": "OWNER", "body": "Oh! I think I broke Cog on `main` and these tests are running against this branch rebased against main.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1651082214, "label": "feat: Javascript Plugin API (Custom panels, column menu items with JS actions)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2052#issuecomment-1760542865", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2052", "id": 1760542865, "node_id": "IC_kwDOBm6k_c5o78CR", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T23:44:53Z", "updated_at": "2023-10-12T23:45:15Z", "author_association": "OWNER", "body": "Weird, the `cog` check is failing in CI.\r\n\r\n```\r\nRun cog --check docs/*.rst\r\n cog --check docs/*.rst\r\n shell: /usr/bin/bash -e {0}\r\n env:\r\n pythonLocation: /opt/hostedtoolcache/Python/3.9.18/x64\r\n PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.9.18/x64/lib/pkgconfig\r\n Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.18/x64\r\n Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.18/x64\r\n Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.18/x64\r\n LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.18/x64/lib\r\nCheck failed\r\nChecking docs/authentication.rst\r\nChecking docs/binary_data.rst\r\nChecking docs/changelog.rst\r\nChecking docs/cli-reference.rst\r\nChecking docs/configuration.rst (changed)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1651082214, "label": "feat: Javascript Plugin API (Custom panels, column menu items with JS actions)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2199#issuecomment-1760441535", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2199", "id": 1760441535, "node_id": "IC_kwDOBm6k_c5o7jS_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T22:08:42Z", "updated_at": "2023-10-12T22:08:42Z", "author_association": "OWNER", "body": "Pushed that incomplete code here: https://github.com/datasette/datasette-upgrade", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1940346034, "label": "Detailed upgrade instructions for metadata.yaml -> datasette.yaml"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2196#issuecomment-1760417555", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2196", "id": 1760417555, "node_id": "IC_kwDOBm6k_c5o7dcT", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T21:54:52Z", "updated_at": "2023-10-12T21:54:52Z", "author_association": "OWNER", "body": "I can't replicate this bug. Closing, but please re-open if it's still happening.\r\n\r\nAs an aside, the link I promote is https://datasette.io/discord which redirects:\r\n\r\n```\r\ncurl -i 'https://datasette.io/discord'\r\nHTTP/2 301 \r\nlocation: https://discord.gg/ktd74dm5mw\r\ncontent-type: text/plain\r\nx-cloud-trace-context: 8dcfd08d3d1fa44f7ee78568e0f5305e;o=1\r\ndate: Thu, 12 Oct 2023 21:54:17 GMT\r\nserver: Google Frontend\r\ncontent-length: 0\r\n```\r\n```\r\ncurl -i 'https://discord.gg/ktd74dm5mw'\r\nHTTP/2 301 \r\ndate: Thu, 12 Oct 2023 21:54:28 GMT\r\ncontent-type: text/plain;charset=UTF-8\r\ncontent-length: 0\r\nlocation: https://discord.com/invite/ktd74dm5mw\r\nstrict-transport-security: max-age=31536000; includeSubDomains; preload\r\npermissions-policy: interest-cohort=()\r\nx-content-type-options: nosniff\r\nx-frame-options: DENY\r\nx-robots-tag: noindex, nofollow, noarchive, nocache, noimageindex, noodp\r\nx-xss-protection: 1; mode=block\r\nreport-to: {\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=Dzzrf%2FgGkfFxtzSAQ46slMVDLcFjsH9fsvVkzHtgUUiZ891rXAa6LvTRpHK%2BdSMSQ54F57hS9z1mZXXklIbONZW1bfBuFjSK9J4XmjjLjsFUulMXvpjfCLkB6PI%3D\"}],\"group\":\"cf-nel\",\"max_age\":604800}\r\nnel: {\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}\r\nserver: cloudflare\r\ncf-ray: 815294ddff282511-SJC\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1910269679, "label": "Discord invite link returns 401"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2199#issuecomment-1760413191", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2199", "id": 1760413191, "node_id": "IC_kwDOBm6k_c5o7cYH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T21:52:25Z", "updated_at": "2023-10-12T21:52:25Z", "author_association": "OWNER", "body": "Demo of that logic:\r\n```\r\n$ datasette upgrade metadata-to-config ../datasette/metadata.json\r\nUpgrading ../datasette/metadata.json to new metadata.yaml format\r\nNew metadata.yaml file will be written to metadata-new-1.yaml\r\nNew datasette.yaml file will be written to datasette.yaml\r\n$ touch metadata-new-1.yaml\r\n$ datasette upgrade metadata-to-config ../datasette/metadata.json\r\nUpgrading ../datasette/metadata.json to new metadata.yaml format\r\nNew metadata.yaml file will be written to metadata-new-2.yaml\r\nNew datasette.yaml file will be written to datasette.yaml\r\n$ touch datasette.yaml\r\n$ datasette upgrade metadata-to-config ../datasette/metadata.json\r\nUpgrading ../datasette/metadata.json to new metadata.yaml format\r\nNew metadata.yaml file will be written to metadata-new-2.yaml\r\nNew datasette.yaml file will be written to datasette-new.yaml\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1940346034, "label": "Detailed upgrade instructions for metadata.yaml -> datasette.yaml"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2199#issuecomment-1760412424", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2199", "id": 1760412424, "node_id": "IC_kwDOBm6k_c5o7cMI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T21:51:44Z", "updated_at": "2023-10-12T21:51:44Z", "author_association": "OWNER", "body": "Started playing with this plugin idea, now tearing myself away to work on something more important:\r\n```python\r\nfrom datasette import hookimpl\r\nimport click\r\nimport pathlib\r\n\r\n\r\n@hookimpl\r\ndef register_commands(cli):\r\n @cli.group()\r\n def upgrade():\r\n \"\"\"\r\n Apply configuration upgrades to an existing Datasette instance\r\n \"\"\"\r\n pass\r\n\r\n\r\n @upgrade.command()\r\n @click.argument(\r\n \"metadata\", type=click.Path(exists=True)\r\n )\r\n @click.option(\r\n \"new_metadata\", \"-m\", \"--new-metadata\", help=\"Path to new metadata.yaml file\", type=click.Path(exists=False)\r\n )\r\n @click.option(\r\n \"new_datasette\", \"-c\", \"--new-datasette\", help=\"Path to new datasette.yaml file\", type=click.Path(exists=False)\r\n )\r\n @click.option(\r\n \"output_dir\", \"-e\", \"--output-dir\", help=\"Directory to write new files to\", type=click.Path(), default=\".\"\r\n )\r\n def metadata_to_config(metadata, new_metadata, new_datasette, output_dir):\r\n \"\"\"\r\n Upgrade an existing metadata.json/yaml file to the new metadata.yaml and\r\n datasette.yaml split introduced prior to Datasette 1.0.\r\n \"\"\"\r\n print(\"Upgrading {} to new metadata.yaml format\".format(metadata))\r\n output_dir = pathlib.Path(output_dir)\r\n if not new_metadata:\r\n # Pick a filename for the new metadata.yaml file that does not yet exist\r\n new_metadata = pick_filename(\"metadata\", output_dir)\r\n if not new_datasette:\r\n new_datasette = pick_filename(\"datasette\", output_dir)\r\n print(\"New metadata.yaml file will be written to {}\".format(new_metadata))\r\n print(\"New datasette.yaml file will be written to {}\".format(new_datasette))\r\n\r\n\r\ndef pick_filename(base, output_dir):\r\n options = [\"{}.yaml\".format(base), \"{}-new.yaml\".format(base)]\r\n i = 0\r\n while True:\r\n option = options.pop(0)\r\n option_path = output_dir / option\r\n if not option_path.exists():\r\n return option_path\r\n # If we ran out\r\n if not options:\r\n i += 1\r\n options = [\"{}-new-{}.yaml\".format(base, i)]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1940346034, "label": "Detailed upgrade instructions for metadata.yaml -> datasette.yaml"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2199#issuecomment-1760411937", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2199", "id": 1760411937, "node_id": "IC_kwDOBm6k_c5o7cEh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T21:51:16Z", "updated_at": "2023-10-12T21:51:16Z", "author_association": "OWNER", "body": "I think I'm OK with not preserving comments, just because it adds a level of complexity to the tool which I don't think is worth the value it provides.\r\n\r\nIf people want to keep their comments I'm happy to leave them to copy those over by hand.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1940346034, "label": "Detailed upgrade instructions for metadata.yaml -> datasette.yaml"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2199#issuecomment-1760396195", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2199", "id": 1760396195, "node_id": "IC_kwDOBm6k_c5o7YOj", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T21:36:25Z", "updated_at": "2023-10-12T21:36:25Z", "author_association": "OWNER", "body": "Related idea: how about a `datasette-upgrade` plugin which adds a `datasette upgrade` command that can be used to automate this process?\r\n\r\nMaybe something like this:\r\n```bash\r\ndatasette install datasette-upgrade\r\ndatasette upgrade metadata-to-config metadata.json\r\n```\r\nThis would output two new files: `metadata.yaml` and `datasette.yaml`. If files with those names existed already in the current directory they would be called `metadata-new.yaml` and `datasette-new.yaml`.\r\n\r\nThe command would tell you what it did:\r\n```\r\nYour metadata.json file has been rewritten as two files:\r\n\r\n metadata-new.yaml\r\n datasette.yaml\r\n\r\nStart Datasette like this to try them out:\r\n\r\n datasette -m metadata-new.yaml -c datasette.yaml\r\n```\r\nThe command is `datasette upgrade metadata-to-config` because `metadata-to-config` is the name of the upgrade recipe. The first version of the plugin would only have that single recipe, but we could add more recipes in the future for other upgrades.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1940346034, "label": "Detailed upgrade instructions for metadata.yaml -> datasette.yaml"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2199#issuecomment-1759952247", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2199", "id": 1759952247, "node_id": "IC_kwDOBm6k_c5o5r13", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T16:23:10Z", "updated_at": "2023-10-12T16:23:10Z", "author_association": "OWNER", "body": "Some options for where this could go:\r\n- Directly in the release notes? I'm not sure about that, those are getting pretty long already. I think the release notes should link to relevant upgrade guides.\r\n- On a new page? We could have a \"upgrade instructions\" page in the documentation.\r\n- At the bottom of the new https://docs.datasette.io/en/latest/configuration.html page\r\n\r\nI'm leaning towards the third option at the moment.\r\n\r\nBut... we may also need to provide upgrade instructions for plugin authors. Those could live in a separate area of the documentation though, since issues affecting end-users who configure Datasette and issues affecting plugin authors are unlikely to overlap much.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1940346034, "label": "Detailed upgrade instructions for metadata.yaml -> datasette.yaml"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2190#issuecomment-1759948683", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2190", "id": 1759948683, "node_id": "IC_kwDOBm6k_c5o5q-L", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T16:20:41Z", "updated_at": "2023-10-12T16:20:41Z", "author_association": "OWNER", "body": "I'm going to land this and open a new issue for the upgrade instructions.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901483874, "label": "Raise an exception if a \"plugins\" block exists in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2190#issuecomment-1759947534", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2190", "id": 1759947534, "node_id": "IC_kwDOBm6k_c5o5qsO", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T16:19:59Z", "updated_at": "2023-10-12T16:19:59Z", "author_association": "OWNER", "body": "It would be nice if we could catch that and turn that into a less intimidating Click exception too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901483874, "label": "Raise an exception if a \"plugins\" block exists in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2190#issuecomment-1759947021", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2190", "id": 1759947021, "node_id": "IC_kwDOBm6k_c5o5qkN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-12T16:19:38Z", "updated_at": "2023-10-12T16:19:38Z", "author_association": "OWNER", "body": "\r\n\r\nThis looks good and works well. The error from this currently looks like:\r\n\r\n```\r\ndatasette -m metadata.json -p 8844\r\nTraceback (most recent call last):\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/datasette\", line 33, in \r\n sys.exit(load_entry_point('datasette', 'console_scripts', 'datasette')())\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py\", line 1130, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py\", line 1055, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py\", line 1657, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py\", line 1404, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.10/site-packages/click/core.py\", line 760, in invoke\r\n return __callback(*args, **kwargs)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/cli.py\", line 98, in wrapped\r\n return fn(*args, **kwargs)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/cli.py\", line 546, in serve\r\n metadata_data = fail_if_plugins_in_metadata(parse_metadata(metadata.read()))\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/utils/__init__.py\", line 1282, in fail_if_plugins_in_metadata\r\n raise Exception(\r\nException: Datasette no longer accepts plugin configuration in --metadata. Move your \"plugins\" configuration blocks to a separate file - we suggest calling that datasette..json - and start Datasette with datasette -c datasette..json. See https://docs.datasette.io/en/latest/configuration.html for more details.\r\n```\r\nWith wrapping:\r\n\r\n`Exception: Datasette no longer accepts plugin configuration in --metadata. Move your \"plugins\" configuration blocks to a separate file - we suggest calling that datasette..json - and start Datasette with datasette -c datasette..json. See https://docs.datasette.io/en/latest/configuration.html for more details.`\r\n\r\nI think we should link directly to documentation that tells people how to perform this upgrade.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901483874, "label": "Raise an exception if a \"plugins\" block exists in metadata.json"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2197#issuecomment-1752096661", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2197", "id": 1752096661, "node_id": "IC_kwDOBm6k_c5obt-V", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-08T16:17:04Z", "updated_at": "2023-10-08T16:17:29Z", "author_association": "OWNER", "body": "https://lite.datasette.io/?install=datasette-packages#/-/packages confirms that Datasette Lite still works (`click-default-group-wheel` was originally built to allow it to run) and that it's now using these packages:\r\n```\r\n \"click-default-group\": \"1.2.4\",\r\n \"datasette\": \"0.64.5\",\r\n```\r\nFull list:\r\n```json\r\n{\r\n \"aiofiles\": \"23.2.1\",\r\n \"anyio\": \"3.7.1\",\r\n \"asgi-csrf\": \"0.9\",\r\n \"asgiref\": \"3.7.2\",\r\n \"certifi\": \"2022.12.7\",\r\n \"click\": \"8.1.3\",\r\n \"click-default-group\": \"1.2.4\",\r\n \"datasette\": \"0.64.5\",\r\n \"datasette-packages\": \"0.2\",\r\n \"h11\": \"0.12.0\",\r\n \"httpcore\": \"0.15.0\",\r\n \"httpx\": \"0.23.0\",\r\n \"hupper\": \"1.12\",\r\n \"idna\": \"3.4\",\r\n \"itsdangerous\": \"2.1.2\",\r\n \"janus\": \"1.0.0\",\r\n \"Jinja2\": \"3.1.2\",\r\n \"MarkupSafe\": \"2.1.2\",\r\n \"mergedeep\": \"1.3.4\",\r\n \"micropip\": \"0.3.0\",\r\n \"packaging\": \"23.0\",\r\n \"Pint\": \"0.22\",\r\n \"pip\": \"23.2.1\",\r\n \"pluggy\": \"1.0.0\",\r\n \"pyparsing\": \"3.0.9\",\r\n \"python-multipart\": \"0.0.6\",\r\n \"PyYAML\": \"6.0\",\r\n \"rfc3986\": \"1.5.0\",\r\n \"setuptools\": \"67.6.1\",\r\n \"sniffio\": \"1.3.0\",\r\n \"typing-extensions\": \"4.5.0\",\r\n \"uvicorn\": \"0.23.2\"\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1930008379, "label": "click-default-group-wheel dependency conflict"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2197#issuecomment-1752095961", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2197", "id": 1752095961, "node_id": "IC_kwDOBm6k_c5obtzZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-08T16:13:42Z", "updated_at": "2023-10-08T16:14:39Z", "author_association": "OWNER", "body": "Confirmed - I ran this in a fresh virtual environment:\r\n```bash\r\npip install --no-cache datasette\r\n```\r\nAnd now:\r\n```bash\r\npip freeze | grep click\r\n```\r\n```\r\nclick==8.1.7\r\nclick-default-group==1.2.4\r\n```\r\n```bash\r\ndatasette --version\r\n```\r\n```\r\ndatasette, version 0.64.5\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1930008379, "label": "click-default-group-wheel dependency conflict"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2197#issuecomment-1752093039", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2197", "id": 1752093039, "node_id": "IC_kwDOBm6k_c5obtFv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-10-08T15:59:53Z", "updated_at": "2023-10-08T15:59:53Z", "author_association": "OWNER", "body": "Replicated this myself:\r\n```bash\r\ncd /tmp\r\nmkdir dddd\r\ncd dddd\r\npipenv shell\r\npip install datasette\r\npip freeze | grep click\r\n```\r\n```\r\nclick==8.1.7\r\nclick-default-group==1.2.4\r\nclick-default-group-wheel==1.2.3\r\n```\r\nYeah this is bad, I'll ship a `0.64.5` release.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1930008379, "label": "click-default-group-wheel dependency conflict"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1168#issuecomment-1739816358", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1168", "id": 1739816358, "node_id": "IC_kwDOBm6k_c5ns32m", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-28T18:29:05Z", "updated_at": "2023-09-28T18:29:05Z", "author_association": "OWNER", "body": "Datasette Cloud really wants this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777333388, "label": "Mechanism for storing metadata in _metadata tables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730458954", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730458954, "node_id": "IC_kwDOBm6k_c5nJLVK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:57:39Z", "updated_at": "2023-09-21T22:57:48Z", "author_association": "OWNER", "body": "Worth noting that it already sets `--cors` automatically without you needing to specify it:\r\n\r\nhttps://github.com/simonw/datasette/blob/d97e82df3c8a3f2e97038d7080167be9bb74a68d/datasette/utils/__init__.py#L374-L374\r\n\r\nI wonder if that's actually surprising behaviour that we should change before 1.0.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730457374", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730457374, "node_id": "IC_kwDOBm6k_c5nJK8e", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:56:18Z", "updated_at": "2023-09-21T22:56:18Z", "author_association": "OWNER", "body": "Maybe I should add `--cors` and `--crossdb` to `datasette publish cloudrun` as well?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730446937", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730446937, "node_id": "IC_kwDOBm6k_c5nJIZZ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:46:42Z", "updated_at": "2023-09-21T22:46:52Z", "author_association": "OWNER", "body": "Found more when I [searched for YAML](https://github.com/search?q=datasette+publish+extra-options++language%3AYAML&type=code).\r\n\r\nHere's the most interesting: https://github.com/labordata/warehouse/blob/0029a72fc1ceae9091932da6566f891167179012/.github/workflows/build.yml#L59\r\n\r\n`--extra-options=\"--crossdb --setting sql_time_limit_ms 100000 --cors --setting facet_time_limit_ms 500 --setting allow_facet off --setting trace_debug 1\"`\r\n\r\nUses both `--cors` and `--crossdb`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730441613", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730441613, "node_id": "IC_kwDOBm6k_c5nJHGN", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:42:12Z", "updated_at": "2023-09-21T22:42:12Z", "author_association": "OWNER", "body": "https://github.com/search?q=datasette+publish+extra-options+language%3AShell&type=code&l=Shell shows 17 matches, I'll copy in illustrative examples here:\r\n\r\n```\r\n--extra-options=\"--setting sql_time_limit_ms 5000\"\r\n--extra-options=\"--config default_cache_ttl:3600 --config hash_urls:1\"\r\n--extra-options \"--setting sql_time_limit_ms 3500 --setting default_page_size 20 --setting trace_debug 1\"\r\n--extra-options=\"--config default_page_size:50 --config sql_time_limit_ms:30000 --config facet_time_limit_ms:10000\"\r\n--extra-options=\"--setting sql_time_limit_ms 5000\"\r\n--extra-options \"--setting suggest_facets off --setting allow_download on --setting truncate_cells_html 0 --setting max_csv_mb 0 --setting sql_time_limit_ms 2000\"\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730438503", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730438503, "node_id": "IC_kwDOBm6k_c5nJGVn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:38:10Z", "updated_at": "2023-09-21T22:38:10Z", "author_association": "OWNER", "body": "I'd really like to remove `--extra-options`. I think the new design makes that completely obsolete?\r\n\r\nMaybe it doesn't. You still need `--extra-options` for the `--crossdb` option for example.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730437934", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730437934, "node_id": "IC_kwDOBm6k_c5nJGMu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:37:22Z", "updated_at": "2023-09-21T22:37:22Z", "author_association": "OWNER", "body": "Here's the full help for Cloud Run at the moment:\r\n```bash\r\ndatasette publish cloudrun --help\r\n```\r\n```\r\nUsage: datasette publish cloudrun [OPTIONS] [FILES]...\r\n\r\n Publish databases to Datasette running on Cloud Run\r\n\r\nOptions:\r\n -m, --metadata FILENAME Path to JSON/YAML file containing metadata\r\n to publish\r\n --extra-options TEXT Extra options to pass to datasette serve\r\n --branch TEXT Install datasette from a GitHub branch e.g.\r\n main\r\n --template-dir DIRECTORY Path to directory containing custom\r\n templates\r\n --plugins-dir DIRECTORY Path to directory containing custom plugins\r\n --static MOUNT:DIRECTORY Serve static files from this directory at\r\n /MOUNT/...\r\n --install TEXT Additional packages (e.g. plugins) to\r\n install\r\n --plugin-secret ...\r\n Secrets to pass to plugins, e.g. --plugin-\r\n secret datasette-auth-github client_id xxx\r\n --version-note TEXT Additional note to show on /-/versions\r\n --secret TEXT Secret used for signing secure values, such\r\n as signed cookies\r\n --title TEXT Title for metadata\r\n --license TEXT License label for metadata\r\n --license_url TEXT License URL for metadata\r\n --source TEXT Source label for metadata\r\n --source_url TEXT Source URL for metadata\r\n --about TEXT About label for metadata\r\n --about_url TEXT About URL for metadata\r\n -n, --name TEXT Application name to use when building\r\n --service TEXT Cloud Run service to deploy (or over-write)\r\n --spatialite Enable SpatialLite extension\r\n --show-files Output the generated Dockerfile and\r\n metadata.json\r\n --memory TEXT Memory to allocate in Cloud Run, e.g. 1Gi\r\n --cpu [1|2|4] Number of vCPUs to allocate in Cloud Run\r\n --timeout INTEGER Build timeout in seconds\r\n --apt-get-install TEXT Additional packages to apt-get install\r\n --max-instances INTEGER Maximum Cloud Run instances\r\n --min-instances INTEGER Minimum Cloud Run instances\r\n --help Show this message and exit.\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730437237", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730437237, "node_id": "IC_kwDOBm6k_c5nJGB1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:36:22Z", "updated_at": "2023-09-21T22:36:22Z", "author_association": "OWNER", "body": "I think the actual design of this is pretty simple. Current help starts like this:\r\n\r\n```\r\nUsage: datasette publish cloudrun [OPTIONS] [FILES]...\r\n\r\n Publish databases to Datasette running on Cloud Run\r\n\r\nOptions:\r\n -m, --metadata FILENAME Path to JSON/YAML file containing metadata\r\n to publish\r\n --extra-options TEXT Extra options to pass to datasette serve\r\n```\r\nThe `-s` and `-c` short options are not being used.\r\n\r\nSo I think `-c/--config` can point to a JSON or YAML `datasette.yaml` file, and `-s/--setting key value` can mirror the new `-s/--setting` option in `datasette serve` itself (a shortcut for populating the config file directly from the CLI).\r\n\r\nHere's the relevant help section from `datasette serve`:\r\n```\r\n -m, --metadata FILENAME Path to JSON/YAML file containing\r\n license/source metadata\r\n -c, --config FILENAME Path to JSON/YAML Datasette configuration\r\n file\r\n -s, --setting SETTING... nested.key, value setting to use in\r\n Datasette configuration\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1730388418", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1730388418, "node_id": "IC_kwDOBm6k_c5nI6HC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:26:19Z", "updated_at": "2023-09-21T22:26:19Z", "author_association": "OWNER", "body": "1.0a7 is out with this fix as well now: https://docs.datasette.io/en/1.0a7/changelog.html#a7-2023-09-21", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730363182", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730363182, "node_id": "IC_kwDOBm6k_c5nIz8u", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:09:10Z", "updated_at": "2023-09-21T22:09:10Z", "author_association": "OWNER", "body": "Tests all pass now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730362441", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730362441, "node_id": "IC_kwDOBm6k_c5nIzxJ", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:08:19Z", "updated_at": "2023-09-21T22:08:19Z", "author_association": "OWNER", "body": "That worked\r\n\r\nhttps://github.com/simonw/datasette/commit/e4f868801a6633400045f59584cfe650961c3fa6 is the latest commit right now and https://latest.datasette.io/-/versions shows that as the deployed version.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730356422", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730356422, "node_id": "IC_kwDOBm6k_c5nIyTG", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T22:01:00Z", "updated_at": "2023-09-21T22:01:00Z", "author_association": "OWNER", "body": "Tested that locally with Python 3.9 from `pyenv` and it worked.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730353462", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730353462, "node_id": "IC_kwDOBm6k_c5nIxk2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T21:57:17Z", "updated_at": "2023-09-21T21:57:17Z", "author_association": "OWNER", "body": "Still fails in Python 3.9: https://github.com/simonw/datasette/actions/runs/6266752548/job/17018363302\r\n```\r\n plugin_info[\"name\"] = distinfo.name or distinfo.project_name\r\nAttributeError: 'PathDistribution' object has no attribute 'name'\r\nTest failed: datasette-json-html should not have been loaded\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2193#issuecomment-1730353006", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2193", "id": 1730353006, "node_id": "IC_kwDOBm6k_c5nIxdu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T21:56:43Z", "updated_at": "2023-09-21T21:56:43Z", "author_association": "OWNER", "body": "The test fails as expected now. Closing this issue, will solve the remaining problems in:\r\n- #2057", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907655261, "label": "\"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2193#issuecomment-1730352111", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2193", "id": 1730352111, "node_id": "IC_kwDOBm6k_c5nIxPv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T21:55:41Z", "updated_at": "2023-09-21T21:55:41Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/actions/runs/6267146158/job/17019594849 failed on 3.9 this time.\r\n\r\n```\r\n plugin_info[\"name\"] = distinfo.name or distinfo.project_name\r\nAttributeError: 'PathDistribution' object has no attribute 'name'\r\nTest failed: datasette-json-html should not have been loaded\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907655261, "label": "\"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730313565", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730313565, "node_id": "IC_kwDOBm6k_c5nIn1d", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T21:16:31Z", "updated_at": "2023-09-21T21:16:31Z", "author_association": "OWNER", "body": "The `@add_common_publish_arguments_and_options` decorator described here is bad. If I update it to support a new `config` option all plugins that use it will break.\r\n\r\nhttps://github.com/simonw/datasette/blob/f130c7c0a88e50cea4121ea18d1f6db2431b6fab/docs/plugin_hooks.rst#L347-L355\r\n\r\nI want to deprecate it and switch to a different, better design to address the same problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2195#issuecomment-1730312128", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2195", "id": 1730312128, "node_id": "IC_kwDOBm6k_c5nInfA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T21:15:11Z", "updated_at": "2023-09-21T21:15:11Z", "author_association": "OWNER", "body": "As soon as `datasette publish cloudrun` has this I can re-enable this bit of the demo deploy:\r\n\r\nhttps://github.com/simonw/datasette/blob/2da1a6acec915b81a16127008fd739c7d6075681/.github/workflows/deploy-latest.yml#L91-L97\r\n\r\nWhich should fix this broken demo from https://simonwillison.net/2022/Dec/2/datasette-write-api/\r\n\r\nhttps://todomvc.datasette.io/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907765514, "label": "`datasette publish` needs support for the new config/metadata split"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730305920", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730305920, "node_id": "IC_kwDOBm6k_c5nIl-A", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T21:09:21Z", "updated_at": "2023-09-21T21:09:21Z", "author_association": "OWNER", "body": "I'm going to disable this bit of the deploy for the moment, which will break the demo linked to from https://simonwillison.net/2022/Dec/2/datasette-write-api/\r\n\r\nhttps://github.com/simonw/datasette/blob/2da1a6acec915b81a16127008fd739c7d6075681/.github/workflows/deploy-latest.yml#L91-L97\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730259871", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730259871, "node_id": "IC_kwDOBm6k_c5nIauf", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:34:09Z", "updated_at": "2023-09-21T20:34:09Z", "author_association": "OWNER", "body": "... which raises the challenge that `datasette publish` doesn't yet know what to do with a config file!\r\n\r\nhttps://github.com/simonw/datasette/blob/2da1a6acec915b81a16127008fd739c7d6075681/.github/workflows/deploy-latest.yml#L114-L122", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730258302", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730258302, "node_id": "IC_kwDOBm6k_c5nIaV-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:32:53Z", "updated_at": "2023-09-21T20:33:02Z", "author_association": "OWNER", "body": "Correct usage is now:\r\n```bash\r\npython tests/fixtures.py fixtures.db fixtures-config.json fixtures-metadata.json \\\r\n plugins --extra-db-filename extra_database.db\r\n```\r\n```\r\nTest tables written to fixtures.db\r\n- metadata written to fixtures-metadata.json\r\n- config written to fixtures-config.json\r\n Wrote plugin: plugins/register_output_renderer.py\r\n Wrote plugin: plugins/view_name.py\r\n Wrote plugin: plugins/my_plugin.py\r\n Wrote plugin: plugins/messages_output_renderer.py\r\n Wrote plugin: plugins/sleep_sql_function.py\r\n Wrote plugin: plugins/my_plugin_2.py\r\nTest tables written to extra_database.db\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730256435", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730256435, "node_id": "IC_kwDOBm6k_c5nIZ4z", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:31:22Z", "updated_at": "2023-09-21T20:31:31Z", "author_association": "OWNER", "body": "New error: \"Error: Metadata should end with .json\"\r\n\r\nhttps://github.com/simonw/datasette/actions/runs/6266720924/job/17018265851\r\n\r\n\"CleanShot", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730250337", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730250337, "node_id": "IC_kwDOBm6k_c5nIYZh", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:26:12Z", "updated_at": "2023-09-21T20:26:12Z", "author_association": "OWNER", "body": "That does seem to fix the problem! ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730247545", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730247545, "node_id": "IC_kwDOBm6k_c5nIXt5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:23:47Z", "updated_at": "2023-09-21T20:23:47Z", "author_association": "OWNER", "body": "Hunch: https://pypi.org/project/importlib-metadata/ may help here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730245204", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730245204, "node_id": "IC_kwDOBm6k_c5nIXJU", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:21:42Z", "updated_at": "2023-09-21T20:21:42Z", "author_association": "OWNER", "body": "I think I see the problem - it's from here: https://github.com/simonw/datasette/commit/b2ec8717c3619260a1b535eea20e618bf95aa30b#diff-5dbc88d6e5c3615caf10e32a9d6fc6ff683f5b5814948928cb84c3ab91c038b6L770\r\n\r\nThe `config` and `metadata` Click options are the wrong way round:\r\n\r\nhttps://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/tests/fixtures.py#L785-L786\r\n\r\nhttps://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/tests/fixtures.py#L801", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730242734", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730242734, "node_id": "IC_kwDOBm6k_c5nIWiu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:19:29Z", "updated_at": "2023-09-21T20:19:29Z", "author_association": "OWNER", "body": "Maybe `plugins/` does not exist? It should have been created by this line:\r\n\r\nhttps://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/.github/workflows/deploy-latest.yml#L41-L42", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2194#issuecomment-1730241813", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2194", "id": 1730241813, "node_id": "IC_kwDOBm6k_c5nIWUV", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:18:40Z", "updated_at": "2023-09-21T20:18:40Z", "author_association": "OWNER", "body": "This looks to be the step that is failing:\r\n\r\nhttps://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/.github/workflows/deploy-latest.yml#L50-L60", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907695234, "label": "Deploy failing with \"plugins/alternative_route.py: Not a directory\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1730232308", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1730232308, "node_id": "IC_kwDOBm6k_c5nIT_0", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:11:16Z", "updated_at": "2023-09-21T20:11:16Z", "author_association": "OWNER", "body": "We're planning a breaking change in `1.0a7`:\r\n- #2191 \r\n\r\nSince that's a breaking change I'm going to ship 1.0a7 right now with this fix, then ship that breaking change as `1.0a8` instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1730231404", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1730231404, "node_id": "IC_kwDOBm6k_c5nITxs", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:10:28Z", "updated_at": "2023-09-21T20:10:28Z", "author_association": "OWNER", "body": "Release 0.64.4: https://docs.datasette.io/en/stable/changelog.html#v0-64-4", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730226107", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730226107, "node_id": "IC_kwDOBm6k_c5nISe7", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:06:19Z", "updated_at": "2023-09-21T20:06:19Z", "author_association": "OWNER", "body": "No that's not it actually, it's something else.\r\n\r\nGot to this point:\r\n```bash\r\nDATASETTE_LOAD_PLUGINS=datasette-init python -i $(which datasette) plugins\r\n```\r\nThat fails and drops me into a debugger:\r\n\r\n```\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/cli.py\", line 186, in plugins\r\n app = Datasette([], plugins_dir=plugins_dir)\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/app.py\", line 405, in __init__\r\n for plugin in get_plugins()\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/plugins.py\", line 89, in get_plugins\r\n plugin_info[\"name\"] = distinfo.name or distinfo.project_name\r\nAttributeError: 'PathDistribution' object has no attribute 'name'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730219703", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730219703, "node_id": "IC_kwDOBm6k_c5nIQ63", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T20:01:54Z", "updated_at": "2023-09-21T20:01:54Z", "author_association": "OWNER", "body": "The problem is here:\r\n```\r\n 86 \t distinfo = plugin_to_distinfo.get(plugin)\r\n 87 \t if distinfo is None:\r\n 88 \t breakpoint()\r\n 89 ->\t assert False\r\n 90 \t if distinfo.name is None:\r\n 91 \t breakpoint()\r\n 92 \t assert False\r\n 93 \t if distinfo:\r\n 94 \t plugin_info[\"version\"] = distinfo.version\r\n(Pdb) distinfo\r\n(Pdb) plugin\r\n\r\n```\r\nThat `plugin_to_distinfo` is missing some stuff.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730214654", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730214654, "node_id": "IC_kwDOBm6k_c5nIPr-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:59:51Z", "updated_at": "2023-09-21T19:59:51Z", "author_association": "OWNER", "body": "So the problem is the `get_plugins()` function returning plugins with `None` for their name:\r\n\r\nhttps://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/datasette/plugins.py#L61-L91", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730212597", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730212597, "node_id": "IC_kwDOBm6k_c5nIPL1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:58:38Z", "updated_at": "2023-09-21T19:58:38Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/80a9cd9620fddf2695d12d8386a91e7c6b145ef2/datasette/app.py#L1127-L1146", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730211445", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730211445, "node_id": "IC_kwDOBm6k_c5nIO51", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:57:44Z", "updated_at": "2023-09-21T19:57:44Z", "author_association": "OWNER", "body": "In the debugger:\r\n```\r\n>>> import pdb\r\n>>> pdb.pm()\r\n> /Users/simon/Dropbox/Development/datasette/datasette/app.py(1136)_plugins()\r\n-> ps.sort(key=lambda p: p[\"name\"])\r\n(Pdb) ps\r\n[{'name': None, 'static_path': None, 'templates_path': None, 'hooks': ['prepare_connection', 'render_cell'], 'version': '1.0.1'}, {'name': None, 'static_path': None, 'templates_path': None, 'hooks': ['startup'], 'version': '0.2'}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730210728", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730210728, "node_id": "IC_kwDOBm6k_c5nIOuo", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:57:08Z", "updated_at": "2023-09-21T19:57:08Z", "author_association": "OWNER", "body": "In my Python 3.8 environment I ran:\r\n```bash\r\ndatasette install datasette-init datasette-json-html\r\n```\r\nAnd now `datasette plugins` produces this error:\r\n```\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/cli.py\", line 192, in plugins\r\n click.echo(json.dumps(app._plugins(all=all), indent=4))\r\n File \"/Users/simon/Dropbox/Development/datasette/datasette/app.py\", line 1136, in _plugins\r\n ps.sort(key=lambda p: p[\"name\"])\r\nTypeError: '<' not supported between instances of 'NoneType' and 'NoneType'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2193#issuecomment-1730208566", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2193", "id": 1730208566, "node_id": "IC_kwDOBm6k_c5nIOM2", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:55:19Z", "updated_at": "2023-09-21T19:55:19Z", "author_association": "OWNER", "body": "Yes, the new script seems to work. On Python 3.11:\r\n\r\n```\r\ntests/test-datasette-load-plugins.sh \r\necho $?\r\n0\r\n```\r\nOn Python 3.8:\r\n```\r\ntests/test-datasette-load-plugins.sh\r\nTest failed: datasette-json-html not found\r\necho $?\r\n1\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907655261, "label": "\"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2193#issuecomment-1730206629", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2193", "id": 1730206629, "node_id": "IC_kwDOBm6k_c5nINul", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:53:39Z", "updated_at": "2023-09-21T19:53:39Z", "author_association": "OWNER", "body": "[GPT-4 says](https://chat.openai.com/share/65a95561-8f3c-4f9b-a4ab-c2508e87d2a9):\r\n\r\n> In the script, you're using a subshell `( ... )` to group commands. If you `exit 1` within the subshell, it will only exit the subshell and not the main script. This is why GitHub Actions does not see it as a failure.\r\n\r\nIt suggested doing this instead:\r\n\r\n```bash\r\n#!/bin/bash\r\n\r\nPLUGINS=$(datasette plugins)\r\nif ! echo \"$PLUGINS\" | jq 'any(.[]; .name == \"datasette-json-html\")' | grep -q true; then\r\n echo \"Test failed: datasette-json-html not found\"\r\n exit 1\r\nfi\r\n\r\nPLUGINS2=$(DATASETTE_LOAD_PLUGINS=datasette-init datasette plugins)\r\nif ! echo \"$PLUGINS2\" | jq 'any(.[]; .name == \"datasette-json-html\")' | grep -q false; then\r\n echo \"Test failed: datasette-json-html should not have been loaded\"\r\n exit 1\r\nfi\r\n\r\nif ! echo \"$PLUGINS2\" | jq 'any(.[]; .name == \"datasette-init\")' | grep -q true; then\r\n echo \"Test failed: datasette-init should have been loaded\"\r\n exit 1\r\nfi\r\n\r\nPLUGINS3=$(DATASETTE_LOAD_PLUGINS='' datasette plugins)\r\nif ! echo \"$PLUGINS3\" | grep -q '\\[\\]'; then\r\n echo \"Test failed: datasette plugins should have returned []\"\r\n exit 1\r\nfi\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907655261, "label": "\"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2193#issuecomment-1730203356", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2193", "id": 1730203356, "node_id": "IC_kwDOBm6k_c5nIM7c", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:51:04Z", "updated_at": "2023-09-21T19:51:04Z", "author_association": "OWNER", "body": "The script:\r\n\r\nhttps://github.com/simonw/datasette/blob/b0d0a0e5de8bb5b9b6c253e8af451a532266bcf1/tests/test-datasette-load-plugins.sh#L1-L29\r\n\r\nI'm not sure why those `exit 1` lines did not cause a CI failure.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907655261, "label": "\"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2193#issuecomment-1730202533", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2193", "id": 1730202533, "node_id": "IC_kwDOBm6k_c5nIMul", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:50:22Z", "updated_at": "2023-09-21T19:50:22Z", "author_association": "OWNER", "body": "Here's the failure in CI, which did not cause the workflow to fail even though it should have:\r\n\r\n\"CleanShot\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1907655261, "label": "\"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730201226", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730201226, "node_id": "IC_kwDOBm6k_c5nIMaK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:49:20Z", "updated_at": "2023-09-21T19:49:20Z", "author_association": "OWNER", "body": "That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the \"Test DATASETTE_LOAD_PLUGINS\" test shows errors but did not fail the CI run.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730188367", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730188367, "node_id": "IC_kwDOBm6k_c5nIJRP", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:38:28Z", "updated_at": "2023-09-21T19:40:38Z", "author_association": "OWNER", "body": "I'll imitate `certbot`:\r\n\r\nhttps://github.com/certbot/certbot/blob/694c758db7fcd8410b5dadcd136c61b3eb028fdc/certbot-ci/setup.py#L9\r\n\r\n```python\r\n 'importlib_resources>=1.3.1; python_version < \"3.9\"',\r\n```\r\nLooks like `1.3` is the minimum version needed for compatibility with the 3.9 standard library, according to https://github.com/python/importlib_resources/blob/main/README.rst#compatibility\r\n\r\nhttps://github.com/certbot/certbot/blob/694c758db7fcd8410b5dadcd136c61b3eb028fdc/certbot/certbot/_internal/constants.py#L13C29-L16\r\n\r\n```python\r\nif sys.version_info >= (3, 9): # pragma: no cover\r\n import importlib.resources as importlib_resources\r\nelse: # pragma: no cover\r\n import importlib_resources\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730185322", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730185322, "node_id": "IC_kwDOBm6k_c5nIIhq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:35:49Z", "updated_at": "2023-09-21T19:35:49Z", "author_association": "OWNER", "body": "I think I can fix this using https://importlib-resources.readthedocs.io/en/latest/using.html - maybe as a dependency only installed if the Python version is less than 3.9.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730183405", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730183405, "node_id": "IC_kwDOBm6k_c5nIIDt", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:34:09Z", "updated_at": "2023-09-21T19:34:09Z", "author_association": "OWNER", "body": "Confirmed: https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files\r\n\r\n> `importlib.resources.files(package)`\r\n> [...]\r\n> New in version 3.9.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2057#issuecomment-1730171241", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2057", "id": 1730171241, "node_id": "IC_kwDOBm6k_c5nIFFp", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:27:25Z", "updated_at": "2023-09-21T19:27:25Z", "author_association": "OWNER", "body": "This broke in Python 3.8:\r\n\r\n```\r\nif plugin.__name__ not in DEFAULT_PLUGINS:\r\n try:\r\n if (importlib.resources.files(plugin.__name__) / \"static\").is_dir():\r\nE AttributeError: module 'importlib.resources' has no attribute 'files'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1662951875, "label": "DeprecationWarning: pkg_resources is deprecated as an API"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1730162283", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1730162283, "node_id": "IC_kwDOBm6k_c5nIC5r", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-21T19:19:47Z", "updated_at": "2023-09-21T19:19:47Z", "author_association": "OWNER", "body": "I'm going to release this in `1.0a7`, and I'll backport it to a `0.64.4` release too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2191#issuecomment-1728504633", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2191", "id": 1728504633, "node_id": "IC_kwDOBm6k_c5nBuM5", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-20T22:24:51Z", "updated_at": "2023-09-20T22:25:16Z", "author_association": "OWNER", "body": "The `{\"units\": {\"distance\": \"m\", \"frequency\": \"Hz\"}}` bit is for the [units feature](https://docs.datasette.io/en/1.0a6/metadata.html#specifying-units-for-a-column) which I've half-disabled already and would like to remove before 1.0, though ideally turning that functionality into a plugin instead (if I can figure out how to do that).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901768721, "label": "Move `permissions`, `allow` blocks, canned queries and more out of `metadata.yaml` and into `datasette.yaml`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2191#issuecomment-1728503623", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2191", "id": 1728503623, "node_id": "IC_kwDOBm6k_c5nBt9H", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-20T22:23:33Z", "updated_at": "2023-09-20T22:24:10Z", "author_association": "OWNER", "body": "This is one of the most interesting illustrative examples in the new code:\r\n\r\nhttps://github.com/simonw/datasette/blob/f7bdedff779606466b580d8528e5a44509291002/tests/fixtures.py#L301-L349\r\n\r\nInteresting to note that it now has canned queries in it, which include this bit:\r\n\r\nhttps://github.com/simonw/datasette/blob/f7bdedff779606466b580d8528e5a44509291002/tests/fixtures.py#L341-L342\r\n\r\nIt looks like metadata, but in this case it's configuration. That blur between metadata and configuration at the canned query level still feels a little bit odd to me, but I still think we're going in the right direction with it.\r\n\r\nAlso interesting, from that same file:\r\n\r\nhttps://github.com/simonw/datasette/blob/f7bdedff779606466b580d8528e5a44509291002/tests/fixtures.py#L351-L399\r\n\r\nThere are a few things in that metadata block that are arguably configuration, not metadata - for example:\r\n\r\nhttps://github.com/simonw/datasette/blob/f7bdedff779606466b580d8528e5a44509291002/tests/fixtures.py#L360\r\n\r\nI think `extra_css_urls` is definitely configuration, not metadata.\r\n\r\nhttps://github.com/simonw/datasette/blob/f7bdedff779606466b580d8528e5a44509291002/tests/fixtures.py#L369-L395\r\n\r\nMost of that stuff is arguably configuration too, with the exception of the `roadside_attractions.columns` bit which is metadata about those columns.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901768721, "label": "Move `permissions`, `allow` blocks, canned queries and more out of `metadata.yaml` and into `datasette.yaml`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/2191#issuecomment-1728498221", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2191", "id": 1728498221, "node_id": "IC_kwDOBm6k_c5nBsot", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-20T22:17:26Z", "updated_at": "2023-09-20T22:17:26Z", "author_association": "OWNER", "body": "I tested this locally for permissions like this. `datasette.yml`:\r\n\r\n```yaml\r\ndatabases:\r\n content:\r\n allow:\r\n id: root\r\n```\r\nStarted Datasette like this:\r\n```bash\r\ndatasette --root content.db pottery2.db -c datasette.yml\r\n```\r\nAs root I could see this (note the padlock):\r\n\r\n\"CleanShot\r\n\r\nhttp://127.0.0.1:8001/-/metadata returned `{}` showing that the permissions must have come from the config file instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901768721, "label": "Move `permissions`, `allow` blocks, canned queries and more out of `metadata.yaml` and into `datasette.yaml`"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1726749355", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1726749355, "node_id": "IC_kwDOBm6k_c5m7Bqr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-20T01:28:16Z", "updated_at": "2023-09-20T01:28:16Z", "author_association": "OWNER", "body": "Added a note to that example in the documentation: https://github.com/simonw/datasette/blob/4e6a34179eaedec44c1263275d7592fd83d7e2ac/docs/internals.rst?plain=1#L1320", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724325068", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724325068, "node_id": "IC_kwDOBm6k_c5mxxzM", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T20:29:41Z", "updated_at": "2023-09-18T20:29:41Z", "author_association": "OWNER", "body": "The one other thing affected by this change is this documentation, which suggests a not-actually-safe pattern: https://github.com/simonw/datasette/blob/6ed7908580fa2ba9297c3225d85c56f8b08b9937/docs/internals.rst#L1292-L1321", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724317367", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724317367, "node_id": "IC_kwDOBm6k_c5mxv63", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T20:25:44Z", "updated_at": "2023-09-18T20:25:44Z", "author_association": "OWNER", "body": "My current hunch is that SQLite gets unhappy if multiple threads access the same underlying C object - which sometimes happens with in-memory connections and Datasette presumably because they are faster than file-backed databases.\r\n\r\nI'm going to remove the `asyncio.gather()` code from the table view. I'll ship a 0.x release with that fix too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724315591", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724315591, "node_id": "IC_kwDOBm6k_c5mxvfH", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T20:24:30Z", "updated_at": "2023-09-18T20:24:30Z", "author_association": "OWNER", "body": "[Using SQLite In Multi-Threaded Applications](https://www.sqlite.org/threadsafe.html)\r\n\r\nThat indicates that there's a SQLite option for \"Serialized\" mode where it's safe to access anything SQLite provides from multiple threads, but as far as I can tell Python doesn't give you an option to turn that mode on or off for a connection - you can read `sqlite3.threadsafet`y to see if that mode was compiled in or not, but not actually change it.\r\n\r\nOn my Mac `sqlite3.threadsafety` returns 1 which means https://docs.python.org/3/library/sqlite3.html#sqlite3.threadsafety \"Multi-thread: In this mode, SQLite can be safely used by multiple threads provided that no single database connection is used simultaneously in two or more threads.\" - it would need to return 3 for that serialized mode.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724305169", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724305169, "node_id": "IC_kwDOBm6k_c5mxs8R", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T20:16:22Z", "updated_at": "2023-09-18T20:16:36Z", "author_association": "OWNER", "body": "Looking again at this code:\r\n\r\nhttps://github.com/simonw/datasette/blob/6ed7908580fa2ba9297c3225d85c56f8b08b9937/datasette/database.py#L87-L117\r\n\r\n`check_same_thread=False` really stands out here.\r\n\r\nPython docs at https://docs.python.org/3/library/sqlite3.html\r\n\r\n> **check_same_thread** ([*bool*](https://docs.python.org/3/library/functions.html#bool \"bool\")) -- If `True` (default), [`ProgrammingError`](https://docs.python.org/3/library/sqlite3.html#sqlite3.ProgrammingError \"sqlite3.ProgrammingError\") will be raised if the database connection is used by a thread other than the one that created it. If `False`, the connection may be accessed in multiple threads; write operations may need to be serialized by the user to avoid data corruption. See [`threadsafety`](https://docs.python.org/3/library/sqlite3.html#sqlite3.threadsafety \"sqlite3.threadsafety\") for more information.\r\n\r\nI think I'm playing with fire by allowing multiple threads to access the same connection without doing my own serialization of those requests.\r\n\r\nI _do_ do that using the write connection - and in this particular case the bug isn't coming from write queries, it's coming from read queries - but perhaps SQLite has issues with threading for reads, too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724298817", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724298817, "node_id": "IC_kwDOBm6k_c5mxrZB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T20:11:26Z", "updated_at": "2023-09-18T20:11:26Z", "author_association": "OWNER", "body": "Now that I've confirmed that parallel query execution of the kind introduced in https://github.com/simonw/datasette/commit/942411ef946e9a34a2094944d3423cddad27efd3 can cause hangs (presumably some kind of locking issue) against in-memory databases, some options:\r\n\r\n1. Disable parallel execution entirely and rip out related code.\r\n2. Disable parallel execution entirely by leaving that code but having it always behave as if `_noparallel=1`\r\n3. Continue digging and try and find some way to avoid this problem\r\n\r\nThe parallel execution work is something I was playing with last year in the hope of speeding up Datasette pages like the table page which need to execute a bunch of queries - one for each facet, plus one for each column to see if it should be suggested as a facet.\r\n\r\nI wrote about this at the time here: https://simonwillison.net/2022/May/6/weeknotes/\r\n\r\nMy hope was that despite Python's GIL this optimization would still help, because the SQLite C module releases the GIL once it gets to SQLite.\r\n\r\nBut... that didn't hold up. It looked like enough work was happening in Python land with the GIL that the optimization didn't improve things.\r\n\r\nRunning the `nogil` fork of Python DID improve things though! I left the code in partly on the hope that the `nogil` fork would be accepted into Python core.\r\n\r\n... which it now has! But it will still be a year or two before it fully lands: https://discuss.python.org/t/a-steering-council-notice-about-pep-703-making-the-global-interpreter-lock-optional-in-cpython/30474\r\n\r\nSo I'm not particularly concerned about dropping the parallel execution. If I do drop it though do I leave the potentially complex code in that relates to it?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724281824", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724281824, "node_id": "IC_kwDOBm6k_c5mxnPg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T19:58:06Z", "updated_at": "2023-09-18T19:58:06Z", "author_association": "OWNER", "body": "I also confirmed that `http://127.0.0.1:8064/airtable_refs/airtable_refs?_noparallel=1` does not trigger the bug but `http://127.0.0.1:8064/airtable_refs/airtable_refs` does.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724278386", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724278386, "node_id": "IC_kwDOBm6k_c5mxmZy", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T19:55:32Z", "updated_at": "2023-09-18T19:55:32Z", "author_association": "OWNER", "body": "OK it looks like it found it!\r\n\r\n```\r\n942411ef946e9a34a2094944d3423cddad27efd3 is the first bad commit\r\ncommit \r\n\r\nAuthor: Simon Willison \r\nDate: Tue Apr 26 15:48:56 2022 -0700\r\n\r\n Execute some TableView queries in parallel\r\n \r\n Use ?_noparallel=1 to opt out (undocumented, useful for benchmark comparisons)\r\n \r\n Refs #1723, #1715\r\n\r\n datasette/views/table.py | 93 ++++++++++++++++++++++++++++++++++--------------\r\n 1 file changed, 67 insertions(+), 26 deletions(-)\r\nbisect found first bad commit\r\n```\r\nhttps://github.com/simonw/datasette/commit/942411ef946e9a34a2094944d3423cddad27efd3 does look like the cause of this problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724276917", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724276917, "node_id": "IC_kwDOBm6k_c5mxmC1", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T19:54:23Z", "updated_at": "2023-09-18T19:54:23Z", "author_association": "OWNER", "body": "Turned out I wasn't running the `datasette` from the current directory, so it was not testing what I intended.\r\n\r\nFIxed that with `pip install -e .` in the `datasette/` directory.\r\n\r\nNow I'm seeing some passes, which look like this:\r\n```\r\nrunning '../airtable-export/testit.sh'\r\nINFO: Started server process [77810]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on http://127.0.0.1:8064 (Press CTRL+C to quit)\r\n Running curls\r\nINFO: 127.0.0.1:59439 - \"GET /airtable_refs/airtable_refs HTTP/1.1\" 200 OK\r\nINFO: 127.0.0.1:59440 - \"GET /airtable_refs/airtable_refs HTTP/1.1\" 200 OK\r\nINFO: 127.0.0.1:59441 - \"GET /airtable_refs/airtable_refs HTTP/1.1\" 200 OK\r\n All curl succeeded\r\nKilling datasette server with PID 77810\r\n../airtable-export/testit.sh: line 54: 77810 Killed: 9 datasette pottery2.db -p $port\r\nAll three curls succeeded.\r\nBisecting: 4 revisions left to test after this (roughly 2 steps)\r\n[7463b051cf8d7f856df5eba9f7aa944183ebabe5] Cosmetic tweaks after blacken-docs, refs #1718\r\nrunning '../airtable-export/testit.sh'\r\nINFO: Started server process [77826]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on http://127.0.0.1:8064 (Press CTRL+C to quit)\r\n Running curls\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724257290", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724257290, "node_id": "IC_kwDOBm6k_c5mxhQK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T19:39:27Z", "updated_at": "2023-09-18T19:44:26Z", "author_association": "OWNER", "body": "I'm now trying this test script:\r\n```bash\r\n#!/bin/bash\r\n\r\nport=8064\r\n# Start datasette server in the background and get its PID\r\ndatasette pottery2.db -p $port &\r\nserver_pid=$!\r\n\r\n# Wait for a moment to ensure the server has time to start up\r\nsleep 2\r\n\r\n# Initialize counters and parameters\r\nretry_count=0\r\nmax_retries=3\r\nsuccess_count=0\r\npath=\"/airtable_refs/airtable_refs\"\r\n\r\n# Function to run curl with a timeout\r\nfunction test_curl {\r\n # Run the curl command with a timeout of 3 seconds\r\n timeout 3s curl -s \"http://localhost:${port}${path}\" > /dev/null\r\n if [ $? -eq 0 ]; then\r\n # Curl was successful\r\n ((success_count++))\r\n fi\r\n}\r\n\r\n# Try three parallel curl requests\r\nwhile [[ $retry_count -lt $max_retries ]]; do\r\n # Reset the success counter\r\n success_count=0\r\n\r\n # Run the curls in parallel\r\n echo \" Running curls\"\r\n test_curl\r\n test_curl\r\n test_curl # & test_curl & test_curl &\r\n\r\n # Wait for all curls to finish\r\n #wait\r\n\r\n # Check the success count\r\n if [[ $success_count -eq 3 ]]; then\r\n # All curls succeeded, break out of the loop\r\n echo \" All curl succeeded\"\r\n break\r\n fi\r\n\r\n ((retry_count++))\r\ndone\r\n\r\n# Kill the datasette server\r\necho \"Killing datasette server with PID $server_pid\"\r\nkill -9 $server_pid\r\nsleep 2\r\n\r\n# Print result\r\nif [[ $success_count -eq 3 ]]; then\r\n echo \"All three curls succeeded.\"\r\n exit 0\r\nelse\r\n echo \"Error: Not all curls succeeded after $retry_count attempts.\"\r\n exit 1\r\nfi\r\n```\r\nI run it like this:\r\n```bash\r\ngit bisect reset\r\ngit bisect start\r\ngit bisect good 0.59.4\r\ngit bisect bad 1.0a6\r\ngit bisect run ../airtable-export/testit.sh\r\n```\r\nBut... it's not having the desired result, I think because the bug is intermittent so each time I run it the bisect spits out a different commit as the one that is to blame.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724263390", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724263390, "node_id": "IC_kwDOBm6k_c5mxive", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T19:44:03Z", "updated_at": "2023-09-18T19:44:03Z", "author_association": "OWNER", "body": "I knocked it down to 1 retry just to see what happened.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724259229", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724259229, "node_id": "IC_kwDOBm6k_c5mxhud", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T19:40:56Z", "updated_at": "2023-09-18T19:40:56Z", "author_association": "OWNER", "body": "I tried it with a path of `/` and everything passed - so it's definitely the path of `/airtable_refs/airtable_refs` (an in-memory database created by an experimental branch of https://github.com/simonw/airtable-export) that triggers the problem.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724258279", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724258279, "node_id": "IC_kwDOBm6k_c5mxhfn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T19:40:13Z", "updated_at": "2023-09-18T19:40:13Z", "author_association": "OWNER", "body": "Output while it is running looks like this:\r\n```\r\nrunning '../airtable-export/testit.sh'\r\nINFO: Started server process [75649]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on http://127.0.0.1:8064 (Press CTRL+C to quit)\r\n Running curls\r\n Running curls\r\n Running curls\r\nKilling datasette server with PID 75649\r\n../airtable-export/testit.sh: line 54: 75649 Killed: 9 datasette pottery2.db -p $port\r\nError: Not all curls succeeded after 3 attempts.\r\nBisecting: 155 revisions left to test after this (roughly 7 steps)\r\n[247e460e08bf823142f7b84058fe44e43626787f] Update beautifulsoup4 requirement (#1703)\r\nrunning '../airtable-export/testit.sh'\r\nINFO: Started server process [75722]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on http://127.0.0.1:8064 (Press CTRL+C to quit)\r\n Running curls\r\n Running curls\r\n Running curls\r\nKilling datasette server with PID 75722\r\n../airtable-export/testit.sh: line 54: 75722 Killed: 9 datasette pottery2.db -p $port\r\nError: Not all curls succeeded after 3 attempts.\r\nBisecting: 77 revisions left to test after this (roughly 6 steps)\r\n[3ef47a0896c7e63404a34e465b7160c80eaa571d] Link rel=alternate header for tables and rows\r\nrunning '../airtable-export/testit.sh'\r\nINFO: Started server process [75818]\r\nINFO: Waiting for application startup.\r\nINFO: Application startup complete.\r\nINFO: Uvicorn running on http://127.0.0.1:8064 (Press CTRL+C to quit)\r\n Running curls\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724159882", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724159882, "node_id": "IC_kwDOBm6k_c5mxJeK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T18:32:29Z", "updated_at": "2023-09-18T18:32:29Z", "author_association": "OWNER", "body": "This worked, including on macOS even though GPT-4 thought `timeout` would not work there: https://chat.openai.com/share/cc4628e9-5240-4f35-b640-16a9c178b315\r\n```bash\r\n#!/bin/bash\r\n\r\n# Run the command with a timeout of 5 seconds\r\ntimeout 5s datasette pottery2.db -p 8045 --get /airtable_refs/airtable_refs\r\n\r\n# Check the exit code from timeout\r\nif [ $? -eq 124 ]; then\r\n echo \"Error: Command timed out after 5 seconds.\"\r\n exit 1\r\nfi\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724157182", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724157182, "node_id": "IC_kwDOBm6k_c5mxIz-", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T18:30:30Z", "updated_at": "2023-09-18T18:30:30Z", "author_association": "OWNER", "body": "OK, I can trigger the bug like this:\r\n\r\n```bash\r\ndatasette pottery2.db -p 8045 --get /airtable_refs/airtable_refs\r\n```\r\nCan I write a bash script that fails (and terminates the process) if it takes longer than X seconds?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/2189#issuecomment-1724089666", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/2189", "id": 1724089666, "node_id": "IC_kwDOBm6k_c5mw4VC", "user": {"value": 9599, "label": "simonw"}, "created_at": "2023-09-18T17:49:24Z", "updated_at": "2023-09-18T17:49:24Z", "author_association": "OWNER", "body": "I switched that particular implementation to using an on-disk database instead of an in-memory database and could no longer recreate the bug.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1901416155, "label": "Server hang on parallel execution of queries to named in-memory databases"}, "performed_via_github_app": null}